让编码 Agent 无需模拟器渲染 Android UI：截图并不够

When a coding agent changes Android XML, it is fairly confident about what it can read: it finds the layout, understands the constraints, and produces a diff. But there was almost always one more manual step afterwards. I would open the screen and decide whether the button, the text, or the card had actually ended up in the right place. I wanted to remove that repeated switch. I was not trying to hand over responsibility for the product or to declare the emulator unnecessary. I wanted the result of the agent's changes to be available inside the same workflow in which it edited the code. My first idea was a screenshot: the agent changes XML, gets an image, and looks at it. In practice, that turned out to be only half of the solution. The setup The loop I wanted for XML/View screens: change XML → render → check the result → fix Enter fullscreen mode Exit fullscreen mode I put the rendering into a local MCP server built with Kotlin/JVM and Robolectric. The agent supplies a layout and explicit data for it, and gets the rendered result back. MCP is not the interesting part here. It is just a way to take me out of a repetitive intermediate step: launching the render by hand and passing the result back to the agent. The scope is deliberately narrow. This is for XML/View projects, not Compose. It is not a replacement for a device, it does not promise a pixel-for-pixel match with a physical phone, and it does not try to emulate hardware surfaces, video, or a camera. The PNG worked, and the tool was still not finished The screenshot made the result visible immediately. The agent could notice that text was clipped, that a block had unexpectedly disappeared, or that two elements were clearly overlapping. That was already better than editing XML blind. At that point the project looked almost done. Screenshots were generated, the MCP server responded, and an inspect_view tool existed. A closer review broke that impression. The screenshot was real, but the sidecar returned an artificial root node as the View tree. There was no real child View to query, so a tool called inspect_view was barely useful. That showed how much an image alone leaves unresolved. It is poor at answering the questions that often decide whether a UI change is correct: What is the actual distance between two Views? Has an element gone beyond the screen boundary? Where exactly is the View with a particular id? A person can estimate these by eye. The agent needs verifiable data, not just an illustration. A second wrong assumption: the warm JVM Around the same time the coding agent pointed out another oversimplified idea in my design. You cannot just start the Robolectric runtime once and keep reusing it after changes to XML or Kotlin: you risk inspecting stale resources and an outdated classpath. That observation changed the architecture. It removed the temptation to treat a "warm JVM" as a problem we had already solved. The default sidecar still is not the persistently warm JVM from my early architectural plan. The fix: measure what you render The turning point was simple to state: the PNG and the data must come from the very same layout, after layout has been performed. After inflate → fixture → measure → layout → draw, the renderer saves both the image and the real View tree. Each node includes its identifier, type, text and state, padding, margins, and absolute bounds. Three small operations make up the loop: render_layout → creates a render session get_view_tree → returns the tree of that session inspect_view → returns a single node by id Enter fullscreen mode Exit fullscreen mode The API names matter less than what they allow. The agent can move from "these look a little too close together" to a measurable fact, such as comparing the edge of a button with the edge of its container without estimating anything from a PNG. That gave us a geometry feedback loop: change → render → inspect → evidence → fix Enter fullscreen mode Exit fullscreen mode The image helps reveal a problem, the tree provides the geometry, and the next change goes through the same render again. Underneath there are also a source fingerprint and an allowlisted Gradle pipeline. They exist so the loop cannot silently render stale state or run arbitrary shell commands. Testing it on a real layout After these changes I connected the MCP server to an actual Android project I was working on. The coding agent used it to fix a real product-card layout with a required-items section. Before the fix. The UI is in Russian. After the fix. The UI is in Russian. The two screenshots show different items, so they are not a pixel-level comparison. For me this was the key acceptance test. The tool had moved beyond its RFC and its own unit tests into a real UI task, and a coding agent used the loop in an actual UI change. What the experiment did and did not prove What I wanted to check Result Can a real Android XML/View layout be rendered through MCP + Robolectric? Yes Can a real, machine-readable View tree be returned alongside the PNG? Yes Did a coding agent use this loop in a real UI change? Yes Can the agent's actions be fully reproduced from the saved data? Not yet I do not consider this a replacement for an emulator or a device. Robolectric does not guarantee a pixel-perfect match with hardware. I have no compatibility matrix covering custom Views and complex themes. For the next experiment of this kind I want to keep the complete record: the prompt, the tool calls, render IDs, the XML diff, the measurements, and the final verdict. These are reasonable limits for a first version. What matters more is that they can now be described as facts rather than as the impression left by a good screenshot. Who did what I did not give the agent the authority to "decide the UI." I defined a bounded problem, set the limits of support, refused to accept a PNG as sufficient evidence, and kept bringing the work back to verifiable criteria. The coding agent was more than a code generator. It helped implement the tool, flagged the risk of a stale Robolectric runtime, and helped build and use the loop itself. For me the main result is not that the agent now has an image. A screenshot makes an interface visible. A PNG together with a View tree makes it verifiable. That is how manual visual review can leave the routine loop: not because I decided to trust the agent, but because it gets artifacts that can be measured and checked again. The renderer is open source: android-ui-renderer-mcp on GitHub. The full case study is on my site: hram.github.io/en/articles/android-ui-renderer-mcp.