Skip to content

Set up the AI iPhone agent

Tell Manina AI what you want to do, then watch it use your iPhone. This guide takes you from a small board to your first task in the iPhone app. Install the AI iPhone agent firmware on a computer first, or read how the phone-use agent works.

Want to use an agent on your computer instead? See CLI & AI agents.

See how the phone, board, and app work together in this walkthrough.

iPad support is coming soon. Here’s Manina AI finishing a Duolingo Japanese lesson on its own on a real iPad.

The App Store release is in review. In the meantime, the public beta is on TestFlight:

  1. On your iPhone, install Apple’s TestFlight app from the App Store if you don’t have it yet.
  2. Open the TestFlight invite on your iPhone, tap Accept, then Install.
  3. When installation finishes, tap Open to start the app.

The TestFlight app goes with the TestFlight board firmware: install it from the installer’s TestFlight view.

Need to choose a board or cable? The hardware & buying guide has the exact model, phone connections, and the same Amazon links as the homepage.

Item What to prepare
iPhone iPhone XS or newer, running iOS 18 or later, with cellular data. USB-C models connect directly; Lightning models need a Lightning to USB 3 Camera Adapter with a charger (Lightning iPhone).
Board Seeed Studio XIAO ESP32S3, the standard version with 8 MB flash and 8 MB PSRAM, including its Wi-Fi/Bluetooth antenna. The installer supports this version, rather than Sense or Plus.
Data cable A USB cable that carries data, for the first firmware installation. A charge-only cable won’t work.
Computer, for the first setup Desktop Chrome or Edge to prepare the board. The computer isn’t needed while a task runs.
AI service Start with a free AI service available in the app. If you prefer your own service, use an OpenAI-compatible API that accepts images.

During a task, your iPhone joins the board’s Wi-Fi hotspot for Screen Mirroring, and the app gets the picture over the same Wi-Fi. That hotspot has no internet, so model calls use your cellular data. When iOS asks to allow Local Network access, tap Allow; without it the picture falls back to Bluetooth and is slower (details).

Do this once before plugging the board into your iPhone. This video first shows turning on AssistiveTouch on the iPhone, then the browser installation:

  1. Open the firmware installer in Chrome or Edge on a computer. An iPhone or Safari can’t perform this step.
  2. Connect the board with the data cable. Wait for Ready when you are., then click Connect board.
  3. Select the board in the browser’s USB list. If the page says it is restarting into install mode, click Connect board and select it again.
  4. Keep the board connected and the page open while installation runs.
  5. Wait until the page says Installed. Then unplug the board from the computer.

Can’t see the board? Try a different data cable, connect directly to the computer, and close other apps using the serial port. If it still doesn’t appear, hold BOOT while plugging it in, release BOOT, then try Connect board again.

If the firmware list doesn’t load, click Check again. If installation fails, unplug the board, reconnect it, and try Connect board again. See the FAQ if you’re still stuck.

Installation replaces the board’s existing firmware. It doesn’t change your iPhone or computer.

Plug the prepared board into your iPhone and open the app, currently named ESP32 Agent. It walks you through four steps:

Check that the antenna is attached and keep the board close to your phone during setup. If pairing fails or the signal is weak, move it right beside the phone and check the antenna connection.

  1. Plug Me In — pair the board over Bluetooth.
  2. Let Me See the Screen — join the board’s Wi-Fi, then open Control Center → Screen Mirroring → ESP32 Agent.
  3. Let Me Use the Pointer — turn on AssistiveTouch. Open Settings, search AssistiveTouch, turn on the switch at the top. It’s under Settings → Accessibility → Touch → AssistiveTouch. The app calibrates the pointer.
  4. Give Me a Brain — choose a free AI service available in the app to get started. To use your own service instead, enter its address and API key, then choose a vision model. Your key is stored in the iOS Keychain.

A free AI service is available now. We plan to offer a subscription AI service in the future for an easier way to get started. Read the privacy policy before choosing a service.

Keep your phone unlocked and in portrait. The board is powered by the phone.

Enter “Open Settings and go to Wi-Fi”, then press Start.

Manina AI looks at the screen, takes a step, and checks what changed. Follow its progress in the Dynamic Island or return to the app. You can stop the task from either place. If the preview is stuck on an old screen or actions keep repeating, stop and check the mirroring troubleshooting steps before retrying.

If it needs your help, press and hold the notification to respond. You may be asked to Continue, Skip step, Stop, or Take Over.

Start with a task you can watch. Screen Mirroring stops when the phone locks. Use portrait mode. For Chinese text, add the Pinyin keyboard; Manina AI switches between English and Pinyin by itself (ctrl+space). Check what appears on screen.

Save a useful task as an intent to run it again from My intents, Siri, Shortcuts, widgets, or Control Center.

Tap the record button at the top right of the Agent tab (New intent):

  • Record me doing it. After a 5-second countdown, do the task yourself on the mirrored screen, then stop. The steps are worked out from how the screen changed; check the ones marked yellow before saving.
  • Save last run as intent. Keep a run that went well.

Open an intent to edit its instruction, checklist, and steps. Recorded moves replay without the AI while the screen matches; add AI steps, reading from the screen, or a step for you to do by hand. Full guide: Record a demonstration and turn it into an intent.

To review a run, open Runs and select it. If the run ended with an answer, it appears in a Result card: tap to expand it, and tap Copy to copy it. Replay the steps with taps and swipes drawn on the screenshots. To export the record, choose ⋯ → Export trace.

In Settings → Runs → Ask me first, choose when the agent stops and asks you in a notification: Payments only (default), Payments, sending, deleting, or Never.

For connection or pointer problems, see the FAQ. To update the board later, see Firmware updates. Before sending screenshots to an AI service, read the privacy policy.