Click To Do lets you feed text to AI models
Click To Do uses optical character recognition (OCR) technology to scan your current screen and make text selectable. Basically, it’s taking a screenshot and letting you interact with elements within it.
For example, if you select an email address, the Click To Do menu presents “Send email” to compose an email. If you select a website URL, you can choose “Open website” to launch it in your default web browser. (Thankfully, this doesn’t always use Microsoft Edge!)
If you select over 10 words of text, things get more interesting. You’ll get a variety of actions that use the Phi Silica language model that runs on your Copilot+ PC’s NPU to perform actions like summarizing text, creating a bulleted list, or rewriting the text in different tones.
Chris Hoffman / Foundry
This is one of Microsoft’s first attempts at integrating NPU-powered text actions on a Copilot+ PC. Because it’s based on a screenshot, you can only send so much text to the language model at once.
That should improve the experience, because let’s be honest: those local language models that run on your Windows PC are nowhere near as powerful as a cloud-based large language models like ChatGPT. (If you’re using a chatbot app in your browser, you’ll wonder why you’re bothering to use a worse app that runs entirely on your PC. That’s a core problem with so many Copilot+ PC local AI features.)
Chris Hoffman / Foundry
There’s also an “Ask Copilot” action that will send your selected text to Microsoft’s Copilot AI chatbot, and a “Draft with Copilot in Word” action to start working on a Word document with Microsoft’s AI.
You can send information to a chatbot in the cloud here, too—but only if you’re using Microsoft’s Copilot (for home users) or Microsoft 365 Copilot (for businesses). Yeah, I know, it can be confusing.
