CPU, GPU, or NPU?
Testing the pattern Immediate API playground on a Copilot+ PC reveals that, for now at the least, Edge just isn’t utilizing Window’s NPU assist. As an alternative, the Home windows Process Supervisor efficiency indicators present that Edge’s Phi mannequin runs on the machine’s GPU. At this early stage in growth, it is smart to take a GPU-only strategy as extra PCs will assist it—particularly the PCs utilized by the goal developer viewers.
It’s probably that Microsoft will transfer to supporting each GPU and NPU inference as extra PCs add inferencing accelerators and as soon as the Home windows ML APIs are completed. Home windows ML’s frequent ONNX APIs for CPU, GPU, and NPU are a logical goal for Edge’s APIs, particularly if Microsoft prepares its fashions for all of the goal environments, together with Arm, Intel, and AMD NPUs.
Home windows ML supplies instruments for Edge’s builders to first check for acceptable inferencing {hardware} after which obtain optimized fashions. As this course of could be automated, it appears preferrred for web-based AI purposes the place their builders don’t have any visibility into the underlying {hardware}.
