构建 FoxyInvoice 第 11 章：触达——SEO、AI 爬虫与成为机器可读的答案

This series is written in the open, from a real production system. This chapter is the reach stack: prerendering, robots.txt, llms.txt, IndexNow, and measuring it all with access logs. [All chapters and diagrams live in the public repo.] Somewhere between "launched" and "learned people exist," every product discovers distribution. Ours arrived with a twist that defines this chapter: half your future traffic now arrives via machines that never render your app — search crawlers that mostly cope, and AI answer engines (GPTBot, ClaudeBot, PerplexityBot) that flat-out don't run JavaScript. If your site is a client-side SPA, those engines see a blank page with a title. This chapter is everything we built so the crawlable half of the internet can actually read the product — and how we measure it. The problem, precisely The FoxyInvoice app is an Angular SPA: the server ships an empty shell and JavaScript builds the page. Human browsers: fine. Googlebot: tolerant, eventually. AI crawlers: blind. Fetch the marketing page with curl — which is exactly what a crawler does — and you got: Invoicing Enter fullscreen mode Exit fullscreen mode One word and an empty div. To every machine reader, our entire free invoice-template business was the word "Invoicing." The fix stack, bottom to top 1. The allowlist robots philosophy Our robots.txt doesn't list what's forbidden — it lists what's allowed, then disallows everything else: User-agent: * Allow: /$ /pricing /privacy /terms /templates /templates/* Disallow: / Enter fullscreen mode Exit fullscreen mode Every future route — admin consoles, /upgrade, QA harnesses — is non-indexable by default until deliberately made public. The same policy is enforced server-side with X-Robots-Tag: noindex, nofollow headers at the edge proxy, per URL: private pages carry it; public pages don't. Belt and suspenders, because robots.txt disallow alone never guarantees de-indexing. 2. Prerendering at build time (the pragmatic middle) Full server-side rendering means an always-on SSR server — violating our boringness constraint — for pages that are 95% app shell. Instead, a post-build script generates static, crawler-ready copies of every public route when the SPA compiles: Real and meta description per page ("Free Plumbing Invoice Template — FoxyInvoice") Canonical URL, Open Graph tags, JSON-LD structured data Full article body copy inside the shell — line items, how-to steps, FAQ — because meta tags describe content; answer engines quote content The build fails if the sitemap and the template data drift apart (a one-line count check that caught a real bug on its first run) The edge proxy's try_files serves these static files to anything that fetches the URL; real browsers still get the app (Angular replaces the static body on boot). No new server, no SSR framework, crawler-complete pages. 3. llms.txt — a menu for answer engines The emerging convention: a markdown file at the root telling AI systems what the product is and linking its key pages, in their vocabulary. Ours lists the product, every template page with a one-line description, and contact points. Cheap, human-readable, and exactly the artifact a "recommend an invoicing tool" query wants. 4. IndexNow: push instead of wait Crawlers traditionally rediscover content on their own schedule. IndexNow flips it: on every deploy, the pipeline POSTs all sitemap URLs to the alliance endpoint (Bing-powered — which feeds several AI answer engines) with a key file proving domain ownership. Our first ping returned HTTP 202 — accepted — and the pages were in Bing's queue the same evening. 5. Measurement: access logs are the truth "You can't improve what you can't see," so the edge now logs every request as JSON (rotated, bounded). The analysis is one script: user-agents, paths, crawler classes. Within days we could see search crawlers reading robots.txt and template pages — and, just as valuable, probe-noise (stray /wp-admin scans) being correctly absorbed. Pair this with Google Search Console + Bing Webmaster registration (a five-minute runbook lives in the repo) for the indexing-and-impressions view logs can't give you. The funnel the surface feeds Reach is only worth building if it lands somewhere. The template gallery — now crawlable — is the top of a three-slice funnel we shipped in order: Slice 1 — no-signup value + persistence: the generator works anonymously, autosaves locally (30 days), and offers resume on return. Slice 2 — the signup handoff: "Save & finish online" carries the exact filled invoice through signup into the new account as a real draft. Slice 3 — the soft capture: "Email me this invoice" stores the draft server-side against their address (with a nurture email); if they later sign up with that address, the dashboard converts it into a real invoice automatically. Each slice was verified in a real browser before shipping — the funnel is the one place where "should work" is banned. Recap. Serve machines real HTML (prerendered at build), allowlist what's public (default-closed), publish llms.txt, push updates via IndexNow, and measure with access logs + Search Console. The era of "GEO" — being the machine-readable answer — is just SEO where the reader never renders your JavaScript. Reading this and want to be found the way this series was? Create a free workspace at foxyinvoice.com, then redeem founding code U8B4Z8S87X on the Upgrade page — 6 months of Pro, free, no card. If anything breaks, there's a feedback button in the app. I read every one. Next: Chapter 12 — Engagement & automation: UI first, then automate.