Microsoft has launched Mu, a brand new synthetic intelligence (AI) mannequin that may run regionally on a tool. Final week, the Redmond-based tech big launched new Home windows 11 options in beta, amongst which was the brand new AI brokers function in Settings. The function permits customers to explain what they wish to do within the Settings menu, and makes use of AI brokers to both navigate to the choice or autonomously carry out the motion. The corporate has now confirmed that the function is powered by the Mu small language mannequin (SLM).
Microsoft’s Mu AI Mannequin Powers Brokers in Home windows Settings
In a weblog put up, the tech big detailed its new AI mannequin. It’s at the moment deployed completely on-device in suitable Copilot+ PCs, and it runs on the gadget’s neural processing unit (NPU). Microsoft has labored on the optimisation and latency of the mannequin and claims that it responds at greater than 100 tokens per second to fulfill the “demanding UX necessities of the agent in Settings state of affairs.”
Mu is constructed on a transformer-based encoder-decoder structure that includes 330 million token parameters, making the SLM match for small-scale deployment. In such an structure, the encoder first converts the enter right into a legible fixed-length illustration, which is then analysed by the decoder, which additionally generates the output.
Microsoft stated this structure was most well-liked as a result of excessive effectivity and optimisation, which is important when functioning with restricted computational bandwidth. To maintain it aligned with the NPU’s restrictions, the corporate additionally opted for layer dimensions and optimised parameter distribution between the encoder and decoder.
Distilled from the corporate’s Phi fashions, Mu was educated utilizing A100 GPUs on Azure Machine Studying. Usually, distilled fashions exhibit increased effectivity in comparison with the dad or mum mannequin. Microsoft additional improved its effectivity by pairing the mannequin with task-specific information and fine-tuning through low-rank adaptation (LoRA) strategies. Apparently, the corporate claims that Mu performs at an identical stage because the Phi-3.5-mini regardless of being one-tenth the dimensions.
Optimising Mu for Home windows Settings
The tech big additionally needed to clear up one other downside earlier than the mannequin might energy AI brokers in Settings — it wanted to have the ability to deal with enter and output tokens to alter tons of of system settings. This required not solely an enormous data community but additionally low latency to finish duties nearly instantaneously.
Therefore, Microsoft massively scaled up its coaching information, going from 50 settings to tons of, and used strategies like artificial labelling and noise injection to show the AI how folks phrase frequent duties. After coaching with greater than 3.6 million examples, the mannequin grew to become quick and correct sufficient to reply in underneath half a second, the corporate claimed.
One essential problem was that Mu carried out higher with multi-word queries over shorter or obscure phrases. As an illustration, typing “decrease display brightness at night time” offers it extra context than simply typing “brightness.” To unravel this, Microsoft continues to point out conventional keyword-based search outcomes when a question is just too obscure.
Microsoft additionally noticed a language-based hole. In situations when a setting might apply to greater than a single performance (for example, “enhance brightness” might discuss with the gadget’s display or an exterior monitor). To handle this hole, the AI mannequin at the moment focuses on probably the most generally used settings. That is one thing the tech big continues to refine.
