最新资讯
dev.to 为 CI/CD 构建 AI 质量门禁 dev.to Kakao 发布面向 AI Agent 的决策模型 Kanana-2-v-decide computerworld.com Microsoft 对 Copilot Chat 企业用户扩大未公开的使用限制 pub.towardsai.net Google 发布 Gemini 4 Carbon:主打网络安全能力的前沿模型 gradientflow.com METR 研究:AI 正在加速构建 AI 的团队本身 www.infoq.com GitLab 19.5 将 Agentic 开发纳入 DevOps 平台的治理体系 aiacceleratorinstitute.com 研究:AI Agent 的复杂流程编排可能反而是问题所在 pub.towardsai.net MongoDB 推出 Vercel AI SDK 集成:为 AI 应用提供持久记忆 thenewstack.io Google 捐出被 Anthropic 和 OpenAI 使用的代码沙箱 gVisor,押注 AI Agent 基础设施开源 dev.to 在信任 AI 生成的测试用例前,如何逐条复核 unite.ai Franklin Templeton 构建 AI 研究工作流以揭示风险 techxplore.com AI 助力寻找更优的钛合金 3D 打印工艺参数
多元动态
登录 注册
智能体 dev.to · @orange

我让 AI Agent 直接抓取 Google Maps 而不是写爬虫,成本是多少

原文 我的收藏

作者用仅配备浏览器工具(打开页面、页内搜索、提取文本、执行页面 JS)的 Claude Sonnet Agent,抓取 3 个美国城市 60 条 Google Maps 咖啡店列表,并与同一批页面上的普通 Playwright 脚本对比。Agent 在抽查行上字段全部正确,但每条消耗 19 万–54 万处理 token、耗时 23–35 秒;脚本为 6–14 秒且不消耗模型 token。作者强调这是小样本对照,附有完整数据与注意事项。

AI推荐理由 作者做了对照实验:用仅带浏览器工具的 Claude Sonnet Agent 抓取 3 个美国城市 60 条 Google Maps 咖啡店信息,并与 Playwright 脚本对比。给出具体成本数据——Agent 每条 19 万至 54 万处理 token、23–35 秒,脚本 6–14 秒且零模型 token,字段准确率方面 Agent 在抽查行上全部正确。属小样本、单一场景,作者也自述有注意事项,不能外推为通用结论,但对 Agent 与确定性脚本的取舍提供了可参考的量化证据。

62

正文

Summary: I ran a Claude agent with only browser tools (no pre-written scraper code) against 60 coffee-shop listings on Google Maps across 3 US cities, and compared it to a plain Playwright script on the same pages. The agent got every field right on every checked row, but cost 190k-540k processed tokens and 23-35 seconds per place, against 6-14 seconds and zero model tokens for the script. Full numbers and caveats below. A thread in r/automation asked whether an LLM agent could just replace a scraper for a simple site instead of someone writing selectors. I didn't answer with an opinion. I ran it. The setup The agent was Claude Sonnet with browser tools only: open pages, search inside them, pull page text, run JavaScript in the page. No scraper was written for it in advance; the only code involved was short in-page JavaScript the agent itself wrote and ran during the runs. It searched Google Maps for "coffee near " and collected name, rating, review count, address, and phone for each result. I ran it four times: two passes over the same 10 places in Times Square, NYC (to check repeatability), one pass over 10 places in Union Square, SF, and one pass over 30 places in Millennium Park, Chicago. 60 place records total, all in the US, all on one site. An earlier Shibuya pass is excluded entirely; none of its numbers appear here. Token and context numbers came from a small script (measure.py) that reads the agent's own transcript (JSONL) and sums usage fields per model call. "Processed" is the raw sum of every token count the API reported across all calls, cache reads included. "Weighted" is the same usage with plain input counted at 1x, cache reads at 0.1x, cache writes at 2x, and output at 5x, an internal cost proxy I use to compare runs, not a real price. I did not compute a dollar cost for this test. Before the agent even opens a page, its context already sits around 31k tokens (system prompt, tool definitions, instructions). Every number below is on top of that baseline. The agent runs Run Target Rows Tool uses Model calls Wall Context end Tokens processed Weighted 1 Times Square NYC, 10 10/10 47 54 292 s 87k 2.93M 492k 2 Times Square NYC, 10 (repeat) 10/10 63 70 347 s 108k 5.38M 781k 3 Union Square SF, 10 10/10 58 67 290 s 77k 3.64M 507k 4 Millennium Park Chicago, 30 30/30 74 86 680 s 123k 5.67M 804k Per-place cost isn't flat. Run 1: about 29 s wall, 293k processed, 49k weighted per place. Run 4, which had 3x the places: about 23 s wall, 189k processed, 27k weighted per place, the cheapest per-record run of the four. That drop wasn't from the agent getting more efficient at reading pages generally. In run 4 it switched strategy: instead of using its general-purpose "find" and "get page text" tools on each place page, it ran one JavaScript call per page that pulled the values straight out of aria-label attributes. That one change is most of why the per-place number fell. Runs 1 and 2 did the exact same task (same 10 places) back to back, and the numbers weren't identical: wall time was 19% higher and weighted tokens were 59% higher on the second run. That run also hit a stalled results feed (a spinner stuck at 6 loaded items) and needed a page reload plus a scripted scroll to recover, which explains most of the gap. Cost also isn't flat within a single run. For run 4, splitting its calls into thirds and summing context per third gives 1.15M, then 1.64M, then 2.88M: the last third cost about 2.5x the first, because each new call has to re-read a context that keeps growing across the whole run. The script baseline For comparison, a Sonnet worker (a different, non-browsing agent) wrote a plain Playwright script from a one-paragraph spec, with one iteration to fix the reviews-count wait condition. After that, the script ran with zero model tokens per record; it's ordinary code. To be precise about scope: the script only re-fetched place pages whose URLs the agent had already found. It never did its own search or list discovery on Google Maps, so this is "script vs. agent on fetching a known page," not the whole search-and-collect task. Set URLs Wall Per URL Reviews empty Failures NYC 10 10 61 s 6.1 s 6/10 0 SF 10 10 100 s 10.0 s 7/10 0 Chicago 30 30 420 s 14.0 s 15/30 1 (h1 never appeared) The script's weak spot was the review count, which loads via an async widget: it came back empty on 28 of 50 rows across these three sets, and one Chicago row failed outright (the page's h1 never became visible within the wait, a Playwright timeout). The agent had 0 empty reviews across all 60 rows, because it also read the search results feed, not just the place page, and could fall back to that. Per-URL time also rose over the 30-URL Chicago batch. Seven of its 30 rows took 32-36 seconds each, and those were exactly the rows that came back with an empty review count; the other rows took 2-7 seconds, apart from the one failed fetch. The 10-URL SF batch had one such 32-second row out of ten, and the NYC batch none. That's consistent with some kind of throttling or soft bot-detection building up over a long single-browser run, but I didn't isolate the cause; I'm flagging it, not claiming it. Accuracy I checked the agent's 60 rows against the script's output field by field: name, rating, address, phone, reviews. Zero true value mismatches on name, rating, address, or phone across all 60 rows. Reviews couldn't be checked on 28 of those rows because the script itself came back empty there (not a disagreement, just missing data on the script's side), plus the one hard failure already mentioned. For the rows the script couldn't confirm, a separate browser-driven check re-read the live page directly (scoped JavaScript on the rating widget, address button, and phone button aria-labels). That covered every row the script had left unverified, 29 in total including the failed row, and all 29 matched the agent's original values, including one case where the review count had genuinely ticked up between the agent's run and the recheck (3563 to 3564 on one shop), which is live drift, not an error. Put together: 60 rows times 5 fields is 300 field checks, 0 confirmed errors. To be exact about "confirmed": every value was checked by a script and, where the script came up empty, by a second automated browser read of the live page. No human visited any of these 60 shops or looked them up independently outside those two automated checks. Two things worth naming. One shop's address is shown on the page as "10036 326 W 47th St, New York, NY 10036" (an odd leading zip code, which the script also captured verbatim). Agent run 1 copied it as-is; agent run 2 wrote "326 W 47th St, New York, NY 10036", dropping the leading token without saying so. Arguably a sensible cleanup, but it is one silent edit of a source value in 300 fields, and the kind of thing you'd want an agent to flag rather than fix quietly. And the first attempt at the browser recheck itself grabbed a value from a "related places" carousel card once instead of the page's own header, before I fixed it by anchoring to the specific rating widget, a trap that applies to any extractor, agent or script. What I'd conclude The agent produced correct data with no scraper code written in advance, using only generic browser tools (search, page-text reads, in-page JavaScript). By the last run it had converged on a script-like approach anyway: one targeted JavaScript extraction per page. The "no-code" agent's cheapest, most reliable mode still looked like a hand-written extractor; it just wrote and ran that extractor itself, per page, inside the same conversation. The honest trade-off: the plain script is roughly 1.6x to 5.7x faster per URL (6.1-14 s versus 23-35 s) and burns zero model tokens once written, but is brittle on anything that loads asynchronously (reviews here) and can hard-fail on an unexpected page structure. The agent is slower and far more expensive per record (tens to hundreds of thousands of tokens versus none), but degrades more gracefully: when one source was empty, it used another. For a recurring job, the agent's token cost scales with time per record and with how much context accumulates in a single run, which grew noticeably over a 30-item batch. Can the cost be cut? The obvious lever is the number of model calls, since every call re-reads the whole context. So I tried the split that the agent itself drifted toward: let the plain script do the bulk, and use the agent only for the rows the script could not fill (the 28 empty review counts and the 1 failed fetch, plus 2 rows I had missed in the first pass, 29 rows in all). The agent re-read those 29 rows in the browser with a handful of batched calls. Approach (same 50 places) Model calls Wall Tokens processed Weighted Agent only (runs 1, 3, 4) 207 1,262 s 12.24M 1,803k Script for all 50 + agent for the 29 gap rows 34 1,088 s 2.88M 519k One-time: writing the script (a Sonnet worker, 1-paragraph spec, 1 iteration, incl. a first 10-URL run) 66 744 s - 516k Weighted tokens fell by 71%, wall time by 14%. Even with the one-time cost of writing the script, the first batch of 50 came out cheaper than running the agent on everything (1,035k versus 1,803k weighted). Two things inflate that number in the hybrid's favor. The script never did list discovery; it re-fetched place URLs the agent had already found, and I did not measure what discovery would cost a script. And the gap-filling agent read its 29 rows in just 4 batched browser calls (23 model calls for the first 27 rows), so part of the saving is plain batching, not the split itself. Run 4 spent 86 calls on 30 rows; the same batching applied to a full agent run would cut its cost too. So I ran the Chicago 30 again the next day with the agent following a recipe I wrote from its own earlier transcript: scroll the list with real wheel events, extract all cards in one JavaScript call, then for each place one batched call of navigate, wait, and a JavaScript read of the aria-labels. Chicago 30 Model calls Wall Context end Tokens processed Weighted Run 4: agent chose its own method 86 680 s 123k 5.67M 804k Run 5b: agent following a written recipe 41 512 s 92k 2.26M 623k Calls fell by 52% and processed tokens by 60%, but weighted tokens only by 23%: what remains is output and cache writes, which batching does not remove. Two scroll batches were wasted because my scroll coordinate missed the list panel, and one small screenshot was needed to see why. The honest framing is that run 5b measures an agent following a human-written recipe, which is most of the way to a script. Accuracy held: the same no-model script confirmed name, rating, address and phone on the 28 rows it could fetch (it timed out on 2, both branches of the same chain), and the review counts it left empty match the previous day's verified values within live drift. A browser re-read confirmed the two rows the script failed on (name, rating, review count, address and phone all matched). What I did not test Anything beyond these 60 places across 3 US cities on one site. No other verticals, countries, or the earlier Shibuya pass. List discovery cost for the script: it only re-fetched place-detail URLs the agent had already found, never its own search or scroll-and-collect. Whether the per-URL slowdown in the 30-item batch was really throttling; I noticed the pattern but didn't isolate the cause. Dollar cost. Weighted numbers are an internal proxy, not a price. Thousands of records, or any site besides Google Maps. This was a personal measurement, not a client project or anything running in production. I also publish Google Maps related actors on Apify Store (https://apify.com/reverenced_garnet).

来源报道 · 1 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容