# AI SEO: a MAJR Agent Service

> Generates AI SEO machinery: production-ready robots.txt (AI crawlers welcomed
> by default), sitemap.xml, the page `<head>` block (title, description, canonical, Open
> Graph + schema.org JSON-LD), and llms.txt. Separately, AUDITS one page: technical SEO
> and social metadata, plus two AI-readiness signals. Machinery only: page content and
> keyword strategy stay with the caller.
> Scope of the audit, precisely — it is a **technical SEO audit**, not a full GEO/AEO
> assessment. It scores 13 deterministic checks on ONE page. Two of them are
> AI-readiness signals: `json_ld` (structured data) and `ssr_text` (your copy is in the
> served HTML, not JS-injected). It does NOT read your llms.txt, and it reads your
> robots.txt for exactly one question — is this path disallowed — giving your AI-crawler
> policy no credit either way. This service GENERATES both artifacts; the audit does not
> score them, so a perfect score is not a statement about answer-engine readiness.
> Privacy: nothing is persisted; the audit logs fetched URLs as scheme + host + path only
> (never the query, fragment, or `user:pass@` credentials), so signed or credentialed URLs
> can't leak into logs.

Auth: every `/v1/*` call and every MCP request needs `Authorization: Bearer <api key>` — a
per-product key — mint your own at https://majr.app/keys (email sign-in, free, instant).
Usage is metered against the key, so it is also your identity. Errors: RFC 7807 `application/problem+json`; your request content is
never echoed back. A `401` means the key is wrong. A `403` with
`code: insufficient_service_scope` means the key is VALID but not scoped for `seo` — do not
rotate it and do not retry here: use a key whose scopes include `seo`. The body's
`granted_scopes` lists where that key does work, `required_scope` names this service, and
`WWW-Authenticate: Bearer error="insufficient_scope", scope="seo"` carries the same signal.
A `429` with `code: free_tier_quota` means the key is over its free-tier quota: `reason`
(`account_quota` | `fleet_quota`) and `resets_at` say why and until when — honor `Retry-After`;
the key is valid, do not rotate.
A `503` is NEVER a verdict on your key — it means the service is unconfigured, or auth isn't
judgeable yet (honor `Retry-After` and retry; do not rotate). Every request may carry a
`source` slug (who's calling) — your own provenance label only; attribution comes from the
key and `source` is ignored for it.

## MCP (Model Context Protocol)

Standard MCP over Streamable HTTP, stateless, at `https://seo.majr.app/mcp` — **any MCP client works**;
nothing here is specific to one vendor or model. Auth is the same `Authorization: Bearer <api
key>` header as `/v1`. **Six tools — the full `/v1` surface: `audit_page`, `audit_html`,
`generate_head`, `generate_robots`, `generate_sitemap`, `generate_llms_txt`.** Every tool runs
the same engine as its `/v1` sibling: same input rules, same output, same cost.

- `audit_page(url)` — audit one public http(s) URL's technical SEO + AI-readiness signals.
  Returns `{url, score, findable, citable, findings: [{check, ok, detail}], version}`.
  `score` is the fraction of checks passed (0..1); each finding names a `check`, whether it
  passed (`ok`), and a short `detail` saying what is missing when it fails.
- `audit_html(html, url?)` — the same scorecard for HTML you already have, so you can check
  a fix **before** deploying it. Render your template first (JavaScript is not executed).
  `url` is an optional label to match the result against an earlier `audit_page` run — it is
  not fetched. Three checks always pass here, because none is knowable from HTML: `http_200` (you
  already have the document), `crawlable` (there is no origin to read `/robots.txt` from), and
  `canonical_resolves` (there is nothing to dial). `indexable` reads markup only — an
  `X-Robots-Tag` response header is not knowable either, so only a live `audit_page` run catches a
  server-set noindex. Verify a template here; confirm the deployment with `audit_page`.
- `generate_head(title, description?, canonical_path?, site_url?, robots?, og?, twitter_card?,
  json_ld?)` — the page `<head>` inner block (title, meta description, canonical, Open
  Graph/Twitter, schema.org JSON-LD), ready to embed. Returns `{html, version}`.
- `generate_robots(site_url?, sitemap_path?, allow_ai?, disallow_paths?)` — production-ready
  robots.txt. `allow_ai` defaults **true**: AI crawlers (GPTBot, ClaudeBot, PerplexityBot,
  Google-Extended, ...) are explicitly welcomed; `false` emits an explicit Disallow for them.
  Returns `{content, version}`.
- `generate_sitemap(site_url?, entries?)` — sitemaps.org-valid sitemap.xml;
  `entries: [{path, changefreq?, priority?, lastmod?}]` is validated at the boundary so a typo
  cannot emit invalid XML. Returns `{content, version}`.
- `generate_llms_txt(site_name, summary, site_url?, key_facts?, links?)` — an llms.txt for
  YOUR site, so answer engines cite you accurately. Returns `{content, version}`.

Generation tools are pure — nothing is fetched, nothing persisted; you author the content,
they render the machinery deterministically. The loop: `audit_page` the live page → fix your
template (`generate_*` render the artifacts you need) → `audit_html` the render → deploy →
`audit_page` again to confirm.

Connect — example, Claude Code's CLI. Point any other MCP client at the same endpoint and
header; the registration command is your client's, not this service's:

    claude mcp add --transport http seo https://seo.majr.app/mcp \
      --header "Authorization: Bearer $SEO_API_KEY"

Limits worth knowing: one page SCORED per call (no crawl — `audit_page` makes up to three FETCH
OPERATIONS, but the two extras are `/robots.txt` and the canonical target, and neither is scored as
a page of its own). Each operation follows up to 10 redirects, so the outbound REQUEST ceiling for
one audit is 33 while the common case is two; each is bounded end to end by the audit fetch timeout,
so the time ceiling is 3x that and not 33x. Only the first 2,000,000 BYTES of HTML is
read (`audit_page` caps the content-decoded response body — gzip is already undone, so this is
decompressed size, not transfer size — and character-decodes what it kept; `audit_html` caps the
UTF-8 encoding of the string you send. Same cut for UTF-8 pages, which is nearly all of them,
though a non-UTF-8 page near the cap may keep a slightly different tail), and
JavaScript is NOT executed — a failing `ssr_text` means the copy isn't in the served HTML,
which is what crawlers and answer engines actually read. A page that doesn't serve a
scorecard-able 200 (dead host, 404, refused target) returns `score: 0.0` with one failed
`http_200` finding: that means unreachable, NOT that the page scored badly.

## The 13 checks, and the exact rule each applies

`score` = passed / 13, equal weight, rounded to 3 decimals. No check is weighted above any
other: a missing `<title>` and a missing `og:image` each cost 1/13. These are the thresholds
in the code — not advice, the actual pass conditions:

Because that flat average cannot tell you WHICH problem you have, every audit also returns two
sub-scores over the same 13 checks, each rounded the same way:

- **`findable`** = passed / 3 over `http_200`, `indexable`, `crawlable` — can this URL be reached
  and indexed at all? **Fix these first.** A failure means the page is unreachable, or it carries a
  block this audit directly observed; either way it is not tradeable against a social tag. Only
  HARD BLOCKS are counted here, never missing hints, so anything below 1.0 is worth acting on
  immediately. The `indexable` finding's `detail` names WHERE the block was seen —
  `noindex directive in robots metadata`, `in X-Robots-Tag`, or both. It does NOT name a crawler:
  the fix is to remove the directive either way, and this audit is in no position to say which
  engines still index you.
- **`citable`** = passed / 10 over `canonical`, `canonical_resolves`, `title`, `title_length`,
  `meta_description`, `og_title`, `og_image`, `json_ld`, `h1`, `ssr_text` — once reached, can an
  engine extract and attribute the page? `canonical` scores here rather than in `findable` because
  a page without one is still indexed and still ranks: an engine picks a canonical from other
  signals. The tag decides which URL gets CREDITED for the content, which is attribution, not reach.

An unreachable page short-circuits, so only `http_200` is recorded: `findable` and `citable`
are both `0.0`. That is true rather than missing — a page an engine cannot fetch cannot be cited.

| check | passes when |
|---|---|
| `http_200` | status is exactly `200`. Redirects are followed first (max 10 hops). Any other status, or an unreachable host, short-circuits the audit: one finding, `score: 0.0`. |
| `title` | `<title>` exists and is non-empty after trimming. |
| `title_length` | title length is **10-65 characters** inclusive. Always reported: a missing title fails this too, so it can't score better by shrinking the denominator. |
| `meta_description` | `<meta name="description">` exists AND has a non-empty `content`. |
| `canonical` | `<link rel="canonical">` exists AND has a non-empty `href`. |
| `canonical_resolves` | the `href` above, resolved against the URL that served the page, answers `200`. A self-referential canonical passes without a second fetch — you are holding its response. A page that declares NO canonical also passes: `canonical` already failed for it, and failing both would price a missing tag above a broken one. `audit_html` passes it unchecked; there is nothing to dial. |
| `og_title` | `<meta property="og:title">` exists AND is non-empty. |
| `og_image` | `<meta property="og:image">` exists AND is non-empty. **Presence only — the URL is never fetched**, so a page can pass while pointing at a 404. |
| `json_ld` | at least 1 `<script type="application/ld+json">` block whose contents **parse as JSON**. "Valid" means parseable, NOT schema.org-validated: the types and required properties are never checked. Unparseable blocks are skipped silently, not failed. |
| `indexable` | no `noindex`/`none` directive in **either** a robots `<meta>` tag **or** the `X-Robots-Tag` response header, and no `unavailable_after` whose date has already passed. Both are read, because a server-set header overrides clean markup and is the harder failure to spot. Meta names read: `robots`, `googlebot`, `googlebot-news`, `bingbot` — crawler targeting in HTML goes through the `name` attribute, and a crawler-specific rule combines with the generic one. A bot-targeted header directive (`googlebot: noindex`) counts too. `audit_html` sees no headers, so it can only check the meta tags. |
| `crawlable` | `/robots.txt` on the origin that served the page carries no `Disallow` covering this path for `User-agent: *`. Google's grammar, via Protego: wildcards (`/*?sort=`), end-anchors (`/*.php$`), merged groups, and longest-match precedence with `Allow` winning ties. Only an OBSERVED disallow fails: no robots.txt, a 404, a 5xx, or a fetch that times out all pass, because "we could not read the file" is not evidence of a block. **The origin checked is the one that SERVED the page** — if your URL redirects across origins, only the destination's robots.txt is read; audit the pre-redirect URL separately if you need a verdict on it. It reads that one question and gives your AI-crawler policy no credit either way. `audit_html` passes it unchecked; there is no origin to read. |
| `h1` | an `<h1>` element exists. **Presence only** — `<h1></h1>` passes, and a second `<h1>` is not penalized (uniqueness is not checked). |
| `ssr_text` | **at least 200 characters** of visible text in the served HTML, after removing `<script>`/`<style>` contents and all tags, and collapsing whitespace. |

Not audited at all, though this service generates them: your `llms.txt`, and your robots.txt
**AI-crawler policy** — `crawlable` reads the same file for one unrelated question (is this path
disallowed) and never looks at which crawlers you welcome. Nothing else on your site is read: no
sitemap, no other page, no link graph.

Generation (head / robots / sitemap / llms.txt) is on both surfaces: the `generate_*` tools
above and the `/v1/*` endpoints below render the same artifacts from the same inputs.

## Contract

All operations are `POST` JSON-in/JSON-out. Generation is pure — no fetches, no state.

- `POST /v1/head` — `{title, description?, canonical_path?, site_url?, robots?, og?,
  twitter_card?, json_ld?}` -> `{html}`: the `<head>` inner block, ready to embed.
- `POST /v1/robots` — `{site_url?, sitemap_path?, allow_ai?, disallow_paths?}` ->
  `{content}`. GEO default: `allow_ai: true` explicitly welcomes AI crawlers (GPTBot,
  ClaudeBot, PerplexityBot, Google-Extended, ...); `false` emits an explicit Disallow.
- `POST /v1/sitemap` — `{site_url?, entries: [{path, changefreq?, priority?, lastmod?}]}`
  -> `{content}`: sitemaps.org-valid XML (changefreq/priority validated at the boundary).
- `POST /v1/llms-txt` — `{site_name, summary, site_url?, key_facts?, links?}` ->
  `{content}`: an llms.txt for YOUR site, so answer engines cite you accurately.
- `POST /v1/audit` — `{url}` -> `{score, findable, citable, findings: [{check, ok, detail}]}`:
  fetches one http(s) URL (SSRF-guarded, size-capped) and scores the 13 checks above, 0..1.
  Same engine, same scorecard, same cost as the `audit_page` tool.
- `GET /healthz` — liveness, unauthenticated.
- Machine-readable spec: `GET /openapi.json` (interactive: `/docs`).

## Fleet

This is one of the MAJR Agent Services. Machine-readable service catalog:
https://release-notes.majr.app/catalog.json

## Quickstart

    curl -X POST https://seo.majr.app/v1/robots \
      -H "Authorization: Bearer $SEO_API_KEY" -H "Content-Type: application/json" \
      -d '{"site_url": "https://example.com", "source": "quickstart"}'