Image source: Generated by Unbounded AI

Microsoft is quietly changing the rules of the game for enterprise AI!

Now, Microsoft has quietly established the world's largest AI agent ecosystem.

Since its launch, more than 100,000 companies have used Copilot Studio to create their own AI agent.

For example, McKinsey allocates agents through an automated process, shortening the project acceptance process from 20 days to only 2 days; Pets at Home deployed fraud prevention agents in less than two weeks, saving thousands of dollars every year. Millions of dollars...

“This growth rate far exceeds our expectations and far exceeds the performance of any other cutting-edge technology we have released,” said Charles, head of Microsoft Intelligence. Lamanna revealed, "In just one quarter, we achieved a 2-fold growth."

At yesterday's Ignite conference, Microsoft made a major announcement: Enterprises can use Azure catalogs in smart bodies. Any one of the 1800 LLM models! This means that Microsoft no longer relies on OpenAI’s exclusive model.

At the same time, Microsoft has also launched a new autonomous AI agent that can work independently. They can detect events and participate in our complex workflows with minimal human intervention.

These AI agents can reason autonomously and operate software that performs specific tasks. From now on, the productivity of enterprises will be further improved.

Copilot

Word

In Word, Copilot will help you find Relevant sources and additional content for drafting.

When you start writing a new document, Copilot finds relevant files, emails, and meeting notes and creates a draft based on them. As you write your document, Copilot displays these sources throughout the document and lets you choose whether to keep references in your draft.

This feature will be available in the web version of Word starting in November and will be launched on the desktop in early 2025.

Copilot will also provide improvement suggestions based on tone, structure and storytelling to help you improve the quality of your document. This feature is currently in public preview.

Just enter a prompt in the Copilot chat pane, and it can intelligently retrieve and add relevant images from the SharePoint organization's resource library in the document, or use DALL-E 3 to generate relevant new images. The feature will be rolled out this month.

Excel

Whether you're creating a project budget, inventory tracker, or sales report, creating a spreadsheet from scratch that fits your needs can be daunting and time-consuming.

Now, Copilot can help you solveDecide. Just tell Copilot what you want to create and Copilot will build a template based on your needs. The feature will begin rolling out to Windows and web users with Microsoft 365 Copilot licenses by the end of the year.

For example, you can quickly create a table of your company's sales team. Copilot can add columns, apply conditional formatting, and even change table colors on command. Once you're satisfied with the form, you can insert it into the page and continue using Copilot.

In Excel, Copilot can reference Word, Excel, PowerPoint and PDF files in the company's internal data.

If you ask Copilot to list the announcements in a press release drafted in a Word document in Excel, it will give us a list that we can insert into a new table, or copy and paste into an existing table. .

While focusing on workflow, we can easily obtain corporate information, such as asking Copilot to provide a list of all direct employees of a specific manager and inserting it into a spreadsheet.

In addition, we can also seamlessly conduct network searches in Excel's Copilot. Without disrupting the workflow, we can find dates, statistical data, or find tables of exchange rates in various countries, and copy this information. Paste into your own form.

You can also find and import tables from Excel files using natural language.

For text-intensive Excel tables, such as customer feedback and survey results, Copilot's text analysis function can help you understand the data more deeply. Just ask Copilot to summarize your text data, and it will also attach citation information to help you understand the source of the data.

Copilot can also classify and sort text data by topic, helping you analyze unstructured text data more quickly. For example, if you have thousands of customer reviews, Copilot can divide them into topics such as quality, service, and price, helping you quickly identify topics that require further attention.

You can also insert these topics into new columns in the table, retain the analysis results, and create pivot tables and charts to visualize the number and trends of topics.

PowerPoint

Similar to some functions of Word, Copilot also supports optimizing text and inserting pictures.

In addition, Copilot’s Presentation Translator function can translate presentations into 40 languages. Translatable content includes text boxes, shapes, tables, charts, speaker notes, comments, and text in Smart-Art .

Copilot automatically generates speaker notes for all slides, giving you a reliable first draft, you can modify it further as needed.

With simple prompts and a Word file, Copilot's Narrative Builder can draft a first draft of a PPT based on them.

New features in Copilot Pages

In September, Microsoft released Copilot Pages - a dynamic, persistent canvas designed for multi-person AI collaboration.

In addition to text, Pages now supports more content types, such as code, interactive charts, tables, diagrams, and mathematical formulas.

These components can be created in BizChat based on web pages or work data, and they can be interacted with and modified dynamically in real time, keeping in sync when shared through Microsoft 365 applications such as Outlook and Teams.

Copilot Pages also adds multi-page support, you can create multiple new pages within a single chat session, or add content from multiple chat sessions into a single page.

In addition, you can edit and share Pages on mobile devices to achieve multi-device and multi-person collaboration.

All these features will be officially rolled out to Pages in early 2025.

Other members of Microsoft Family Bucket

In Outlook, through Copilot, users can use natural language to schedule meetings. Copilot can also help you draft a meeting agenda, including a summary and key takeaways.

In OneNote, with Copilot Chat, you can summarize note content, create task lists, or rewrite note content through natural language commands.

In addition, Copilot can help you organize notebook chapters in seconds. Open Copilot in the OneNote app, enter "Organize this chapter", and Copilot will automatically organize your note pages.

The Facilitator agent will support the Teams meeting room function in early 2025, and can record and display notes in real time for all participants inside and outside the meeting room.

Facilitator agent also supports speaker recognition function. This feature will support more meeting scenarios, from Teams meeting rooms to bring-your-own-device (BYOD) meeting rooms, and can identify the voices and people in the room, making each speaker clearly visible. Speaker recognition also enables Copilot, Facilitator, and intelligent meeting review to correctly identify speakers, ensuring accurate meeting minutes and action item attribution.

In addition, Microsoft also launched a new AI translator that can simulate the speaker's voice and provide near-real-time voice translation in nine languages ​​on the Teams platform, including Chinese (Mandarin), English, French, German, Italian, Japanese, Korean, Portuguese (Brazil) and Spanish.

This feature is currently being tested on a small group of users and is expected to be widely available to accounts with Microsoft 365 Copilot licenses in 2025.

Copilot Studio

As mentioned at the beginning, Microsoft has The update is quite significant.

1. Expand knowledge management functions

Developers can use the latest generation models, update and reference third-party data sources in real time, and use the retrieval-augmented generation (RAG) function to improve their intelligence body quality.

2. New analysis function

Developers can filter charts based on specific results to understand key performance indicators (KPIs) and customer satisfaction.

3. New voice and image functions

You can now add voice solutions, including interactive voice response (IVR) systems; or deploy intelligent agents into applications to allow users Interact with the agent through voice. Not only can users communicate with the agent through voice, they can also upload pictures and ask the agent to analyze and answer questions about the picture.

4. The customized autonomous agent function enters the preview stage

Developers can create agents that do not require manual prompts. They can respond at any time after detecting specific events and trigger an event. Series of business operations.

5. Microsoft 365 Agents SDK enters preview stage

With the SDK, developers can now extend the functionality of agents through code and build enterprise-class, scalable multi-channel agents. .

Knowledge Management

Copilot Studio is expanding knowledge management and RAG (Retrieval Augmentation Generation) capabilities. This feature is in preview.

The platform introduces a connector function that allows access to new knowledge sources. Developers can now enable agents to obtain real-time knowledge from multiple third-party data sources such as Salesforce, ServiceNow and Zendesk. Copilot Studio only semantically indexes the metadata of these tables, eliminating the need for data movement, to help answer complex questions from these third-party sources.

You can also mark certain data sources as "authoritative" to ensure that the agent uses these data first and provides reliable responses.

In addition to accessing new knowledge, Copilot Studio also uses the latest GPT model and optimizes the RAG strategy to obtain high-quality answers.

Through advanced knowledge tuning capabilities, developers can better control data sources. For each question, developers can now clearly see which data sources are used and which are stillNot answered yet. Knowledge tuning capabilities also suggest additional knowledge to add from the organization and provide guided methods to filter and prioritize knowledge sources.

Before the agent goes online, developers can also test it to see how accurate its response is.

In addition, the platform also provides detailed analysis data, such as data source usage, etc., to understand whether certain data sources need to be adjusted.

Copilot Studio’s knowledge management function also integrates Azure AI Foundry to support more complex and customized scenarios.

Developers typically store large amounts of data in Azure AI.

This data can now be used directly as a knowledge source in Copilot Studio, allowing search operations to be performed on vectorized indexes built in Azure AI Search to access unstructured data.

Developers can also combine this data with customized RAG strategies into Copilot Studio for more accurate results. By connecting all data sources, organizations can ensure that agents are more relevant to their business data and provide specific, high-quality responses.

That’s not all. More advanced Azure AI capabilities are also introduced in Copilot Studio. Developers will soon be able to access models in the Azure AI Model Catalog, which includes more than 1,800 models. They can also access and call their own custom fine-tuned models directly in Copilot Studio.

New Analysis

With new analysis capabilities, developers can understand how their generative AI systems are performing.

In addition to the analysis of knowledge base data sources mentioned above, the top-level indicators used to measure the overall performance of the system or business have also been comprehensively upgraded. These indicators will be more complete, more efficient, and able to demonstrate long-term trends.

Developers can filter charts based on specific results to understand key performance indicators (KPIs) and customer satisfaction.

Voice and Image

Copilot Studio now includes voice functionality. Enterprises can integrate intelligent agents into interactive voice response systems (IVR), turning originally rigid voice menus into flexible AI assistants.

Smart IVR can easily handle a large number of complex problems on the phone by simply specifying the source of knowledge, significantly reducing the number of problems that need to be escalated to manual processing. These smart voice assistants can also recognize speech, handle user interruptions, detect silence, and re-prompt.

Not only limited to IVR systems, voice-enabled agents can also be embedded in applications, independent terminals, concierge systems and other scenarios.

For example, a hotel application can use a Copilot agent connected to the corresponding knowledge base to act as a dedicated concierge service in the room to provide guests with considerate help.

In addition to the voice functionYes, Copilot Studio also adds new image processing capabilities.

Users can now upload images and ask questions to the agent. For example: "Why is my bill higher this month?"

In addition, image uploads can be used to enter data. The salesperson can say to the agent: "Add these handwritten notes to the CRM system."

Autonomous Agent

The function of customizing autonomous agents was announced at the end of October, and now it is finally open for preview.

Autonomous agents can independently represent users, teams or organizations, automating complex and time-consuming business tasks.

Developers can design agents to detect specific events (such as the arrival of new emails). These agents can respond at any time and use generative AI to trigger a series of actions to automate complex business tasks.

For example, the agent can perform the following operations in one go:

1. Assess the intent of the email 2. Find sender details and account information 3. View past communications 4. Check inventory 5 . Reply to the sender and ask for their preferences 6. Take appropriate action to close the ticket

Developers can either build autonomous agents from scratch or configure pre-built agents with specific functionality in Copilot Studio.

Agent SDK

Microsoft 365 Agents SDK is currently available in preview.

This means that Copilot Studio developers can now extend the functionality of agents through code and build enterprise-class, scalable multi-channel agents.

Developers have complete flexibility and can connect to any AI service, including Azure AI Foundry, Semantic Kernel, Copilot Studio, etc.

Agents built using the SDK can also be deployed to a variety of channels, such as Microsoft 365 Copilot, Microsoft Teams, Web, and more than a dozen third-party messaging platforms.

Two custom chips, specially built for data processing and security

At the conference, Microsoft launched two chips designed specifically for data center basic design: Azure Integrated HSM and Azure Boost DPU.

Azure Boost DPU is Microsoft’s first data processing unit designed for high-efficiency and low-power data-centric workloads.

Microsoft expects that future Azure servers equipped with DPU will run 4 times the performance of existing servers, but will also consume 3 times the power.

The design of this chip may have originated from Microsoft’s announcement in December last year, DPU manufacturer Funcible acquired for US$190 million.

Another chip is Azure Integrated HSM, which is built for security.

It can place the pre-signing key and encryption key in a secure module without affecting performance or increasing latency.

It is Microsoft’s second security chip after Pluton. Pluton is a consumer-focused chip built into Intel, AMD and Qualcomm processors.

Starting next year, Azure Integrated HSM will be equipped on every server in Microsoft data centers to increase Azure hardware protection for confidential and general workloads.

Mini PC only costs 2,500 yuan, but cannot run local applications

In terms of hardware, Microsoft released a PC that it claims is smaller than the Mac mini - Windows 365 Link, priced at US$349 (2,526 yuan).

This mini PC is planned to be launched next year and will be used with Windows 365 cloud services.

That is, it will not run any local content on the device.

Link is designed as a compact, fanless, easy-to-use cloud computer that can connect to displays and external devices.

Through Windows 365 services, enterprises can let employees use virtual machines that exist in the cloud, and these virtual machines can be securely streamed to multiple devices.

It boots up in seconds, wakes instantly from sleep, and uses its own chip to process video from Microsoft Teams or Cisco Webex.

It has two USB-A 3.2 ports and one USB-C 3.2 port, as well as a DisplayPort and HDMI port and Ethernet connection. On the front, there's a USB-A 3.2 port, which also includes Wi-Fi 6 E and Bluetooth 5.3 wireless connectivity.