Microsoft’s AI just got eyes on everything you are doing

Microsoft is expanding the capabilities of its AI assistant, Copilot, with a feature called Copilot Vision, allowing the tool to see and interpret everything on a user's screen. 

According to a blog by Microsoft on July 15, the update, currently rolling out to Windows Insiders, marks a significant shift in how generative AI interacts with desktop environments.

Previously, Copilot Vision was limited to analysing content from just two open applications simultaneously. With this update, it can now view the entire desktop or specific application and browser windows, depending on the user's selection. Users can activate the feature by clicking on a glasses icon in the Copilot interface and granting access to the screen they wish to share.

The tool can assist users with a variety of tasks, from offering feedback on creative projects and improving CVs to guiding them through unfamiliar video games. The company began testing Copilot Vision last year as a browser-specific assistant for Edge but is now extending its reach across more platforms.