SiliconANGLE AI ↗ · 2026-11-20 07:41 · UTC+8
Google Adds Computer Use Capabilities to Gemini's Mac App
Media reports SiliconANGLE AI Source language: English
Google updated its Gemini Mac client to include computer use capabilities, allowing it to interact with local files, desktop applications, and the web. The feature is currently rolling out to Google AI subscribers in the United States.
Full text

Google LLC today released a new version of Gemini’s Mac client that can interact with other apps on the user’s computer.
The feature is available in the U.S. on launch. It requires a Google AI subscription, which equips consumer Google accounts with a broader lineup of artificial intelligence tools. The offering also includes additional storage space and certain other features.
Users can instruct Gemini to launch a local app by including the command “@Computer use” in their prompts. For example, a designer could upload a sketch of a vehicle and ask the AI assistant to turn it into a three-dimensional animation. Gemini finds the local app that is needed for the task, creates a project plan and carries out the work step by step.
The assistant supports tasks that comprise dozens of actions. Furthermore, consumers can use the output of a newly completed Gemini workflow to launch follow-up projects. A user could have the assistant generate a visual asset in a design tool and then turn it into a mini-app with Visual Studio Code.
Gemini can interact with not only local apps but also web services. That means it can load data from webpages into programs that run on the user’s machine, as well as move data in the opposite direction and between cloud services.
The AI carries out its work in a background window. As a result, users don’t have to pause their work while the assistant is running. They can track Gemini workflows via a preview pane built into the Mac client’s chat window. Additionally, Google has included controls that make it possible to give Gemini feedback mid-task.
“Privacy and safety are built in: you choose which apps Gemini can access, watch every step live, pause at any time, and approve key actions before they happen,” Michael Friedman, Google’s group product manager for the Gemini app, wrote in a blog post.
The Mac version of Muse, Meta Platforms Inc.’s recently launched AI agent, can also interact with local apps and data. Furthermore, a cloud-based virtual machine enables it to perform tasks even when the user’s computer is offline. Dots, a competing service from OpenAI Group PBC, offers a similar capability.
Google may seek to match its rivals’ cloud-based workflow hosting features in the future. The company can also extend Gemini in other ways. Notably, it could bring Gemini’s computer use mode to additional operating systems and add features for business users. OpenAI’s Dots already includes capabilities designed to automate business workflows such as procurement.
Image: Google