[Feature]: add ADB operator to let UI‑TARS control physical Android devices via ADB #579
Description
Which destkop app does this feature request relate to?
- Select a project 👇
- Agent-TARS App
- UI-TARS Desktop
- Infra Packages
What problem does this feature solve?
Describe the feature
First of all, thank you for maintaining UI‑TARS Desktop — it has been extremely helpful in my daily automation work! 🙏
I’d like to contribute ADB operator support, so that users can operate physical Android devices directly from UI‑TARS via Android Debug Bridge (ADB).
Motivation
UI‑TARS Desktop can already drive desktop UIs and (via plugins) some emulators, but it cannot operate a physical Android device out of the box.
Many real‑world automation and QA scenarios—mobile UI regression tests, cross‑app RPA flows, hardware‑level validations—must run on an actual phone or tablet that is connected over USB/TCP. Today users have to write external scripts or rely on third‑party wrappers around the Android Debug Bridge (ADB), which leads to fragmented workflows, extra tooling, and brittle hand‑rolled glue code.
Adding native ADB support lets users control a real device from the same UI‑TARS automation graph they already know, keeping all actions observable, reproducible, and CI‑friendly. It broadens UI‑TARS’ reach from “desktop only” to “desktop + mobile” without forcing new infrastructure.
What does the proposed features look like?
The patch introduces first‑class ADB support across the whole stack—operator, IPC, service layer, state store, and renderer UI.
Below is a component‑by‑component tour that maps exactly to the files in the diff so reviewers can follow along quickly.
| Layer / File(s) | Key additions | Purpose |
|---|---|---|
| Operatorapps/ui-tars/src/main/agent/operator.ts | • AdbElectronOperator (extends @ui-tars/operator-adb)• Manual/action‑space help text | Provides the Android‑specific implementation of screenshot, execute, etc., wrapping ADB calls while emitting UI‑TARS‑style logs. |
| Operator package wiringapps/ui-tars/package.json | @ui-tars/operator-adb added to work‑space deps | Exposes the new operator as a workspace package so other layers can import it. |
| Availability check serviceapps/ui-tars/src/main/services/adbCheck.ts | checkAdbAvailability() using getAndroidDeviceId() | Detects whether any device is connected; result cached in global store. |
| IPC routeipcRoutes/adb.ts & ipcRoutes/index.ts | adbRoute.checkAdbAvailability | Makes the availability check callable from the renderer (api.checkAdbAvailability()). |
| Main startupsrc/main/main.ts | Calls checkAdbAvailability() during boot | Ensures the store is primed before the UI mounts. |
| Store & validationstore/types.ts, store/validate.ts | • adbAvailable flag• operator enum extended to 'adb' | Keeps app‑wide reactive state and JSON‑schema in sync with the new mode. |
| Agent runnerservices/runAgent.ts | • Branch to instantiate AdbElectronOperator when settings.operator === 'adb'• Error handling when no device found | Lets GUIAgent switch dynamically between NutJS, Browser, and ADB operators. |
| Renderer UIcomponents/ChatInput/SelectOperator.tsx, const/index.ts | • Adds ADB Operator item with Smartphone icon• Tooltip + inline retry when no device is detected• Auto‑fallback to “Computer Operator” if ADB becomes unavailable | Gives users a one‑click toggle to ADB mode and actionable feedback when a device is missing. |
| IPC consumer (renderer) | api.checkAdbAvailability() call on retry button | Bridges the renderer’s retry button to the main‑process service through the new IPC route. |
Impact:
- No breaking changes: existing NutJS and Browser modes remain the default.
- Optional opt‑in UI keeps the interface uncluttered unless a device is present.
With these pieces in place, UI‑TARS users will be able to record and replay automation flows on real Android hardware exactly the same way they do on the desktop today—one unified toolchain, zero external scripts.