so i’ve been messing around with this idea for a while, and it’s probably gonna sound wild, but i think we’re all looking at mobile automation wrong. everyone’s focused on ADB commands or rooting/jailbreaking their devices to get deep control, right? and yeah, that works, but it’s always OS-specific, fragile, and a pain in the ass to maintain. what if… what if you just simulated a human touching the screen? like, literally. we’ve been building this thing, Aiden, which uses a super cheap RV1106 dev board—seriously, like ten bucks—to capture HDMI output from a phone, run some local LLM stuff on it, and then act as a USB HID to ‘touch’ the screen. no ADB, no root, just pure hardware-level interaction. it’s OS-agnostic because it doesn’t care what’s running, just what it sees and where it needs to tap. right now, the biggest bottleneck is the ReAct loop, there’s definitely a noticeable lag, but i’m convinced this is the way forward for truly general-purpose agents. like, if you want an AI to actually use your phone or tablet like a human, without being tied to specific APIs or software versions, this feels like the only real path. part of me thinks ‘is this just massive overengineering for a problem that already has solutions?’ but then i think about the flexibility. no more ‘oh, this app updated, now my script is broken.’ it just sees the new UI and adapts. i’m kinda stoked about it, but also ready to be told i’m an idiot. anyway, we put some of the demo stuff up on github, tell me why this is a terrible idea, or why it’s brilliant. what am i missing? submitted by /u/Alookaparatha9144
Originally posted by u/Alookaparatha9144 on r/ArtificialInteligence