esp32-iphone-agent
An iPhone automation agent that uses an ESP32-S3 as an AirPlay screen receiver and USB mouse and keyboard, letting an OpenAI-compatible vision model observe and control a real iPhone without a Mac.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
ESP32 iPhone Agent lets an AI vision model operate a real iPhone through an ESP32-S3 board, without a Mac, developer account, or jailbreak. The board creates a Wi-Fi hotspot, receives the iPhone’s AirPlay screen stream, and acts as a USB mouse and keyboard. An app on the iPhone decodes the stream, sends screenshots to an OpenAI-compatible vision model, and performs actions such as tapping, swiping, and typing. Runs can be replayed and exported for review.
Key features
- Creates a Wi-Fi hotspot and receives the iPhone’s AirPlay screen stream, forwarding it to the app over Bluetooth Low Energy.
- Acts as a USB mouse and keyboard to control the iPhone with AssistiveTouch.
- Uses an iPhone app to send screenshots to an OpenAI-compatible vision model and perform actions such as tapping, swiping, and typing.
- Keeps run history for step-by-step replay and export of screenshots, actions, and annotations.
Best for
People who want to automate tasks on a real iPhone with a vision model and an ESP32-S3, without using a Mac. It is also useful for reviewing or exporting agent runs.
- Upstream
- onehumanbeing/esp32-iphone-agent
- Fork on GitHub
- Guo-astro/esp32-iphone-agent
- Upstream stars
- 4.5k
- Category
- AI agents and LLM tools
- Language
- Swift
- License
- MIT
- Forked
- 2026-10-12
- Sync status
- Updated by last syncLast synced 2026-11-09
More in AI agents and LLM tools
/last30days is an AI-agent skill for researching recent discussions across social platforms, video, news, markets, and the web. It brings scattered signals together in a synthesized brief for trend research and preparation.
Forked 2026-10-22Last synced 2026-11-09License: MITAI agents and LLM toolsGitHub
A portable C99 inference engine for running Kimi K3 on a CPU without a GPU or machine-learning framework. It streams the large checkpoint from disk to support inference with limited RAM, for batch text generation or text chat.
Forked 2026-10-21Last synced 2026-11-09License: Apache-2.0AI agents and LLM toolsGitHub
LayerFS gives AI agents writable workspaces over shared history, so they can work in parallel without copying the base. Use it to checkpoint, branch, rewind, and merge agent changes.
Forked 2026-10-21Last synced 2026-11-09License: MITAI agents and LLM toolsGitHub
Magnitude runs open models locally and tunes kernels for your hardware. Use it to connect local models to supported agents or an OpenAI-compatible client.
Forked 2026-10-19Last synced 2026-11-09License: Apache-2.0AI agents and LLM toolsGitHub