OPEN SPECIFICATION · RFC DRAFT 1.0-RC1

An Open Standard for Agent-Readable Web Content

Websites were built for human eyes. index-ai gives AI agents a parallel, structured, and token-efficient interface — from AI Manifests to Navigable Agent Graphs.

The gap index-ai closes: only 28% of 50,665 declared links deliver text directly — see AUTOPSY V4

“llms.txt is a note for LLMs. index-ai is an interface for agents.”

Give us a star on GitHub Try Validator CLI

Agent View — The Machine-Readable Layer

The Core Paradigm

The Virtual DOM Analogy for the AI Web

Two consumers · Two interfaces · One source of truth

In 2013, React introduced the Virtual DOM because the browser's visual DOM was too expensive to manipulate directly. index-ai applies the exact same principle to websites: HTML/CSS/JS is the Human Interface; Agent View is the Agent Interface.

Human Interface

HTML / CSS / JavaScript

Built for visual hierarchy, layout rendering, styling, and user interaction.

80% of character count spent on markup & scripts
Client-side rendering causes empty DOM retrieval failures
Unknown token cost before download

Agent Interface (index-ai)

Structured Agent View

Built exclusively for machine consumption, semantic graph traversal, and token budgeting.

0% markup bloat — clean Markdown/text endpoints
Phase 1 pre-fetch decision via llm_summary (200 chars)
content_chars pre-measured code point budget before fetch

The Agent Web: Why This Matters — Hard Machine Labs architectural breakdown

Progressive Adoption

The 4 Conformance Tiers

Adopt incrementally in 15 minutes to 2 hours

LEVEL 1

AI Manifest

/.well-known/index-ai.json

Single HTTP call declaring identity, publisher role, freshness timestamps, usage policies, and discovery links.

⚡ ~15 min implementation

LEVEL 2a

Agent Index

/agent-index.json

Flat array of content nodes with clean text endpoints (llm_url) and measured code point bounds (content_chars).

⚡ ~2 hour implementation

LEVEL 2b

Agent Graph

/agent-index.json + relations

Extends Level 2a with typed relationships (parent, children, related) forming a valid Directed Acyclic Graph (DAG).

👑 Full Graph Navigation

LEVEL 3

Query Interface

MCP Server

Exposes a parameterized Model Context Protocol (MCP) server allowing agents to execute live typed queries over site data.

🚀 Dynamic Tool Execution

Normative Schema & Examples

Interactive Specification Explorer

Click tabs to inspect valid Level 1 & Level 2b payloads

"$schema": "https://raw.githubusercontent.com/index-ai/standard/main/schema/v1/index-ai.schema.json",
"spec_version": "1.0",
"manifest_version": 1,

"identity": {
  "name": "Atlas Hotels",
  "description": "Real-time hotel booking covering 12 cities in Morocco. 247 hotels.",
  "domain": "atlashotels.ma",
  "category": ["hotels", "booking", "travel"]
},

"publisher": {
  "name": "Atlas Hotels SAS",
  "role": "official-platform",
  "contact": "[email protected]",
  "verification_hint": "dns-txt"
},

"freshness": {
  "content_updated_at": "2026-07-29T14:30:00Z",
  "refresh_frequency": "continuous",
  "valid_until": "2026-12-31T23:59:59Z"
},

"policy": {
  "usage_preferences": { "search": true, "summarization": true, "citation": true },
  "citation": { "required": true }
},

"access": {
  "agent_index": "/agent-index.json",
  "llms_txt": "/llms.txt"
}

"spec_version": "1.0",
"nodes": [
  {
    "id": "root-about",
    "type": "company-profile",
    "label": "About Hard Machine Labs",
    "content": {
      "llm_summary": "Systems engineering lab specializing in deterministic AI agent orchestration.",
      "llm_url": "/clean/about.md",
      "content_chars": 4820,
      "content_chars_mode": "exact"
    },
    "relations": {
      "parent": null,
      "children": ["node-shokunin", "node-index-ai"]
    }
  },
  {
    "id": "node-index-ai",
    "type": "specification",
    "label": "index-ai Open Standard",
    "content": {
      "llm_summary": "Verifiable specification for agent-readable web representation.",
      "llm_url": "/clean/spec.md",
      "content_chars": 14250,
      "content_chars_mode": "exact"
    },
    "relations": {
      "parent": "root-about",
      "children": []
    }
  }
]

<!-- 1. HTML <head> Tag (RECOMMENDED for Web Pages) -->
<link rel="agent-manifest" href="/.well-known/index-ai.json" type="application/json">

/* 2. HTTP Link Header (RECOMMENDED for API & Headless Agents) */
Link: </.well-known/index-ai.json>; rel="agent-manifest"

/* 3. robots.txt Directive */
Agent-Manifest: /.well-known/index-ai.json

# Response served at llm_url (/clean/about.md)
# Content-Type: text/markdown; charset=utf-8

# Hard Machine Labs — Systems Architecture

Hard Machine Labs is a specialized systems engineering laboratory founded by **Jordach Makaya**.
It develops deterministic tools for the agentic web, including the **index-ai** standard and **Shokunin Harness**.

## Core Verification Rules
1. Verified ≠ Declared
2. Pre-measured token budgets before HTTP fetch
3. Mechanical release gates for all multi-agent code releases

Released Reference Implementation

index-ai-validator CLI & Audit Suite

Package published on npm: @hardmachinelabs/index-ai-validator

The official reference validator CLI tests any domain against all 4 levels of the specification — verifying JSON schemas, DAG acyclicity, content_chars code point precision, secret leaks, and publisher verification hints.

index-ai/validator v0.2.0 Audit ReportTarget: https://jordach.dev

PASSED

Conformance Level 2b — Agent Graph

Verification date: 2026-07-29 · Readiness Score: 100% · 0 failures, 0 critical errors

reached

AI Manifest

reached

L2a

Agent Index

reached

L2b

Agent Graph

Query MCP

index-ai conformantLevel 2b · verified 2026-07

Embed snippet ready: <script src="https://agent-view.com/badge.js" data-domain="jordach.dev"></script>

DAG & Content Graph Verification

Total Graph Nodes24

Root Nodes3

Parent / Child Pairs21 (Consistent)

Cycles Detected0 (Valid DAG)

Orphan References0

Token Bounds & Leaks Audit

llm_url Endpoint Coverage100%

content_chars PrecisionExact NFC match

Secret Value Scanning0 secrets exposed

robots.txt Precedence CheckAligned

Landscape Alignment

Comparison with Existing Standards

Why existing standards are necessary, but insufficient alone

Standard / Feature	Question Answered	Format	Agent Interface Capability	index-ai Alignment RECOMMENDED
robots.txt	"Where may crawlers go?"	Text File	Access permissions only. No content, no structure.	Additiveindex-ai respects robots.txt restrictions as primary authority.
sitemap.xml	"What URLs exist?"	XML	URL list for search engines. No clean text, no token bounds.	AdditiveSitemap discovers URLs; index-ai structures content & pre-measures token cost.
llms.txt	"Here is context for LLMs"	Markdown	Human-readable markdown note. Not a queryable JSON schema.	Bridgellms.txt is a note; index-ai is a queryable interface.
Schema.org	"What entities are in this page?"	JSON-LD	Page-scoped entity metadata. No site-wide graph or clean endpoints.	ComplementarySchema.org describes in-page entities; index-ai structures the site graph.
★ index-ai (L1–L3)	"What is this site, what content exists, and how much will it cost to fetch?"	JSON + MCP	Full Agent View: Manifest + DAG Nodes + Pre-fetch summaries + Exact content_chars.	Complete StandardThe Complete Open Interface for Agents.

llms.txt vs index-ai: architectural comparison infographic by Hard Machine Labs

Developer Adoption Guide

Implement it yourself now

Prompts written for your LLM Coding Agent (Cursor, Claude, AGY, Copilot)

Select your level of conformance

The conformance ladder. Selected Level: Level 1 — AI Manifest (/.well-known/index-ai.json). Copy the coding agent prompt below.

Coding Agent Prompt for Level 1 — AI Manifest (/.well-known/index-ai.json)(Copy & paste into your coding agent)

You are a senior web engineer. Implement Level 1 of the index-ai v1.0-rc1 specification on this website.

Requirements:
1. Create a JSON manifest served at `/.well-known/index-ai.json` (RFC 8615), with a fallback alias at `/index-ai.json`. Content-Type `application/json`, UTF-8, hard max 50 KB (10 KB RECOMMENDED).
2. Schema:
   - spec_version: "1.0", manifest_version: 1 (integer, bump whenever content changes)
   - identity: name, description (factual, written for an LLM consumer -- MUST NOT contain marketing superlatives), domain, category [] (lowercase slugs, `[a-z0-9-]+`), language [], geo (optional)
   - publisher: name, role ("official-platform" | "publisher" | "aggregator" | ...), contact, verification_hint ("dns-txt" | "meta")
   - freshness: content_updated_at (ISO-8601 -- MUST reflect the real last content update, NEVER the manifest/deploy/build timestamp), manifest_generated_at, refresh_frequency ("static" | "daily" | "weekly" | "monthly" | "continuous"), valid_until, cache_max_age_seconds
   - policy: usage_preferences (search, summarization, citation, agent_navigation, training, commercial_reuse -- these are real content-licensing decisions; `training` in particular MUST be an explicit choice confirmed with the site owner, never silently defaulted), citation (required, preferred_format), rate_limits (advisory only -- MUST still be enforced at the infrastructure level, this field does not enforce anything itself)
   - entrypoints []: at least one { topic, description, url, params }, pointing agents at your main content area(s)
   - access: agent_index ("/agent-index.json"), llms_txt ("/llms.txt")
   - llm_instructions: free text, factual and operational, MUST NOT exceed 500 characters
3. Discovery: add `<link rel="agent-manifest" href="/.well-known/index-ai.json" type="application/json">` to every page's `<head>`, add an HTTP `Link: </.well-known/index-ai.json>; rel="agent-manifest"` response header, and append `Agent-Manifest: /.well-known/index-ai.json` to `robots.txt`.
4. Generate this file at build/deploy time for static or infrequently-changing sites -- do not compute it per-request unless your content genuinely changes continuously.

Verify your implementation

Once your coding agent has committed the implementation files, run the official reference validator to verify schema compliance, DAG graph integrity, content_chars NFC code point accuracy, and generate a standalone HTML Audit Report.

$ npx @hardmachinelabs/index-ai-validator validate https://your-site.com --html=report.html

Audit AI Retrieval & Signals (Agent View)

Optional Step

Want to see what signals AI models (ChatGPT, Claude, Perplexity, DeepSeek) actually use when extracting facts from your site? Run a free local delivery scan or launch a full retrieval evidence trace on Agent View.

Free CLI Delivery Scan

FREE · LOCAL

Runs the validator's scan command to evaluate site reachability, extractability, visual gap, and signal-to-markup ratio in 60 seconds.

$ npx @hardmachinelabs/index-ai-validator scan https://your-site.com

Premium Agent View Audit

PREMIUM PLATFORM

Test your site live on agent-view.com. See the split-screen Visual Gap, trace search candidate selection, inspect rejected sources, and verify model retrieval evidence.

Launch Agent View Audit (agent-view.com) ↗