Open Source // MIT
See what AI sees
when it looks at your page
Paste a URL. See what GPTBot, ClaudeBot, Googlebot, PerplexityBot, and Gemini extract from your page.
Robots & Access
Check which AI bots your robots.txt allows. GPTBot, ClaudeBot, PerplexityBot, Googlebot, and more.
Per-Bot Extraction
Each bot processes your page differently. See what each one extracts: chunks, Markdown, rendered DOM.
Side-by-Side Compare
Compare word counts, truncation, and visibility across every bot in one view.
Why Does This Matter?
Every AI system processes your page differently. GPTBot doesn't render JavaScript. Googlebot does. ClaudeBot converts HTML to Markdown and truncates at 100KB. ChatGPT reads in 200-word sliding window chunks. Google silently drops everything after 2MB.
A React SPA that ranks #1 on Google can be invisible to ChatGPT, Claude, and Perplexity. Those bots see an empty <div id="root"></div>. Headlights shows you the difference.
How Each Bot Processes Your Page
ChatGPT
No JS
Plain text, ~200-word sliding window chunks
Claude
No JS
Turndown HTML → Markdown
Full JS
Rendered DOM, passage-level blocks
Perplexity
No JS
Plain text extraction
Gemini
Full JS (via Google)
Search snippets, passage-level
Claude Code
No JS
Turndown → Markdown → LLM summary
The Visibility Gap
Site owners optimize for Google and assume the rest follows. It doesn't. Blocking Google-Extended in robots.txt prevents Gemini from grounding on your page — but has no effect on AI Overviews. Blocking PerplexityBot doesn't stop Perplexity-User from fetching on demand.
The rules are fragmented, underdocumented, and changing fast. Headlights shows you what each bot actually ingests. No interpretation, no score.
Quick Reference: Key Constraints
Try Your URL
Enter your URL above. No signup, no tracking. Open source.