OpenAI has announced that its ChatGPT can now work with different apps on macOS and Windows desktops. This marks the company’s first direct attempt at computer vision and agent control. The update, released on November 15, is currently in beta for Plus and Team users. It allows ChatGPT to examine coding apps like VS Code, Xcode, Terminal, and iTerm2 to provide better answers for users. Additionally, ChatGPT can also talk to its users through its voice assist feature, take screenshots, upload files, and search the web through SearchGPT. This update follows the recent release of Anthropic’s Claude Artifacts, which allows users to create apps without writing any code. The next step for ChatGPT could be to control and see desktops as an agent, similar to Microsoft’s Copilot Vision and Anthropic’s Claude 3.5 Sonnet. This move by OpenAI shows its focus on developing autonomous agents that can perform tasks and manage business functions on behalf of individuals, teams, and departments. With the release of its agent ‘Operator’ in January 2025, OpenAI is set to make a significant impact in the AI industry.
