Microsoft is rolling out a big update to its Copilot Vision for Windows Insiders that lets the AI tool see everything on your screen. This is a pretty significant change, as it could only look at two apps at a time before. Copilot will see your desktop, or a specific browser or app window.
This new “Desktop Share” feature gives you the option of sharing your desktop with Copilot whenever you need. Once you do, it can see what you see and even talk to you about it in real time. It’s a similar idea to how the ChatGPT app works on Windows and Mac, and how Gemini can see your screen if you let it.
I’ve used this feature in AI, and it is pretty accurate for ChatGPT and Gemini. I imagine that Copilot will have the same accuracy that these chatbots have, which could be helpful in the right circumstances. Copilot can analyze content, provide insights, and answer your questions, all while coaching you through it aloud.
The only thing you need to do is click the glasses icon in the composer. Then, select the desktop you want to share, and ask Copilot for help with whatever you’re working on. When you’re done, simply press ‘Stop’ or ‘X’ in the composer to stop sharing your screen with the AI.
It’s important to note that this isn’t like Recall, which automatically takes regular snapshots of your screen in the background. Using Copilot Vision with desktop share is more like screen sharing during a video call. You actively choose to turn it on and off, and screenshots aren’t taken and stored somewhere else.
This update also makes it easier to access Vision from your voice conversations. If you’re already talking with Copilot and realize you need to show it something on your screen for more context, you can now simply click the glasses icon and share your screen right then and there. This is a feature that you may not use all the time, but is incredibly useful when you do find a use for it.
Just keep in mind that it still has the limitations that AI tends to have in a regular conversation. I have used this to check homework, and was given plenty of incorrect suggestions by Gemini despite it seeing exactly what I saw. Even still, there were many times it was better to show the AI the screen to get the information I needed from documents instead of fiddling with the Find search bar.
This new Copilot app update is gradually rolling out to Windows Insiders across all channels through the Microsoft Store. So, you might need to be patient as it makes its way to your computer. Anyone who isn’t part of this program will have to wait until the full release to use this feature, but the time in between will be spent fixing bugs and making sure it works correctly.
