RoyalPineapple/TheButtonHeist

<img width="1536" height="1024" alt="Noir-style heist planning board with an iPhone at center labeled The Vault, connected by red string to crew member dossiers: The Inside Job, The Safecracker, The Mastermind, The Fence, and The Bagman. A whiskey glass and desk lamp sit in the foreground." src="https://github.com/user-attachments/assets/ab62f18f-a3bd-480e-906d-3167b90c1d77" />

The Button Heist

ButtonHeist lets agents and tests drive iOS apps by accessibility intent instead of screen coordinates, then returns receipts that say what changed.

It turns the app's accessibility contract into a live world model: ButtonHeist's current settled view of controls, roles, values, actions, and evidence after each move.

The goal is not "I tapped something." The goal is "the app accepted the intent, and here is the evidence."

When an iOS app exposes accessibility well, it already explains itself. Labels name things. Traits describe controls. Values and state say what changed. Actions and rotors say what can happen next. VoiceOver renders that contract through speech and gesture; ButtonHeist renders it as structured text, typed intent, and receipts.

The core trick is small:

capture settled accessibility state
-> perform typed intent
-> capture settled accessibility state
-> compute evidence
-> return receipt

Everything else in ButtonHeist is built from this core unit. A direct command is one loop. A heist is a reusable set of ButtonHeist instructions: do these actions, wait for these facts, and keep the receipt. The DSL gives heists