Open Source // MIT
See what AI sees
when it looks at your page
Paste a URL to see how GPTBot, ClaudeBot, Googlebot, PerplexityBot, and Gemini process your content. No scores — just the ground truth.
Robots & Access
See which AI bots your robots.txt allows or blocks — GPTBot, ClaudeBot, PerplexityBot, Googlebot, and more.
Per-Bot Extraction
Each bot processes your page differently. See the exact content each one extracts — chunks, Markdown, rendered DOM.
Side-by-Side Compare
Compare word counts, truncation, and content visibility across all bots in a single summary view.
Why Does This Matter?
Every AI system processes your page differently. GPTBot doesn't render JavaScript. Googlebot does. ClaudeBot converts HTML to Markdown and truncates at 100KB. ChatGPT reads in 200-word sliding window chunks. Google silently drops everything after 2MB.
A React SPA that ranks #1 on Google may be completely invisible to ChatGPT, Claude, and Perplexity — because those bots see an empty <div id="root"></div>. Headlights makes the invisible visible.
How Each Bot Processes Your Page
ChatGPT
No JS
Plain text, ~200-word sliding window chunks
Claude
No JS
Turndown HTML → Markdown
Full JS
Rendered DOM, passage-level blocks
Perplexity
No JS
Plain text extraction
Gemini
Full JS (via Google)
Search snippets, passage-level
Claude Code
No JS
Turndown → Markdown → LLM summary
The Visibility Gap
Site owners optimize for Google and assume the rest follows. It doesn't. Blocking Google-Extended in robots.txt prevents Gemini from grounding on your page — but has no effect on AI Overviews. Blocking PerplexityBot doesn't stop Perplexity-User from fetching on demand.
The rules are fragmented, underdocumented, and changing fast. Headlights doesn't interpret or score — it shows you the raw truth of what each bot actually ingests.
Quick Reference: Key Constraints
Stop Guessing. Start Seeing.
Enter any URL above to see exactly what each AI bot extracts from your page. No signup. No tracking. Open source.