Build Inference Chip Index: Next.js leaderboard and Lucid Agents API

Build a polished public Next.js site and commercial Lucid Agents API that turns official MLPerf Inference v6.0 results into trustworthy, source-linked comparisons of inference accelerators. The target user is a developer building an autonomous research agent that needs normalized chip data. The product must not claim that one chip is universally fastest. Rankings must compare an exact benchmark slice: the same release, division, workload, scenario, accuracy target, metric, and unit. ## Required stack - Next.js App Router and TypeScript. - Pin `@lucid-agents/core@5.0.0`, `@lucid-agents/http@4.0.0`, and `@lucid-agents/payments@5.0.0` unless the requester approves a version change. - Compose one server-only Lucid runtime using the HTTP and payment extensions with base path `/api/agent`. - Next.js route modules must delegate to `runtime.http.handlers`. Do not build a second endpoint registry, payment layer, discovery layer, or custom adapter error contract. - Deployment target: Cloudflare Workers through OpenNext. A public preview URL is required. ## Dataset scope Use MLPerf Inference v6.0 commit: `4d3916ac9cf474b679cdfcf492d43a0559418ad1` Source: https://github.com/mlcommons/inference_results_v6.0 V1 includes only: - Closed division paths under `closed/<submitter>/systems/*.json` and matching `closed/<submitter>/results/**`. - Workloads: `llama3.1-8b`, `gpt-oss-120b`, and `deepseek-r1`. - Scenarios: Server, Interactive, and Offline when present. - Official results that pass MLPerf validity checks and a checked-in metric registry. Open, Network, edge, vision, speech, training, vendor-marketing claims, and supplemental vendor datasets are out of scope. The data model must distinguish accelerators, submitted systems, benchmark results, comparison slices, dataset manifests, and tombstones. Every published record needs a stable logical ID, an immutable content-version ID, and an HTTPS source reference containing repository, commit, path, URL, and SHA-256 hash. Ambiguous accelerator identity, variant, topology, partitioning, count, metric, unit, or validity must be quarantined as `review-required` and excluded from rankings. Never infer accelerator count only from names such as `x8` or `NVL72`. ## Ranking requirements - Never combine incompatible releases, divisions, workloads, scenarios, accuracy targets, metrics, or units. - The official submitted system result is the primary metric. - A per-accelerator value may be shown only as a clearly labelled derived value when the source establishes accelerator count and the metric registry permits derivation. It is never the default ranking. - Metric definitions must specify canonical unit, winning direction, allowed workloads/scenarios, upstream log keys, validity/latency requirements, and whether derivation is allowed. - Multiple submissions for the same accelerator support `best-per-accelerator` and `all-systems` grouping. - Server-computed rank and position must remain correct across pagination and ties. - Every ranking or comparison response includes a plain-language comparability description, dataset version, and source links. ## Required website 1. A strong landing page with the promise: “Find the fastest verified inference hardware for your workload,” immediately qualified as workload-specific. 2. A filterable leaderboard with workload, scenario, accuracy target, submitter, vendor, and metric-view controls. Release is fixed to v6.0 and division to Closed. 3. Visible dataset release, source commit, last-reviewed time, freshness state, and record counts. 4. A methodology page explaining systems versus chips, official versus derived metrics, benchmark scenarios, accuracy targets, and comparability limits. 5. An API page showing endpoint inputs, outputs, prices, errors, copyable requests, example responses, and the x402 discovery/payment/invocation flow. 6. An updates page with dataset manifest and changelog. Design should feel like a serious technical index: strong editorial typography, high data density, restrained color, accessible responsive tables, and prominent provenance. Required states include loading, no comparable results, partial evidence, invalid filters, stale data, payment required, API error, and paid success. ## Required Lucid entrypoints 1. `get-dataset-status` — free. Returns manifest, freshness, source commit, counts, source links, and all available exact comparison slice IDs. 2. `preview-inference-chips` — free. Returns up to five verified rows for an optional exact slice ID. 3. `rank-inference-chips` — paid, default advertised price $0.02. Requires an exact slice ID; supports vendor filters, official or derived metric view, grouping, bounded pagination, and a maximum proven response size below 1 MiB. 4. `compare-inference-chips` — paid, default advertised price $0.03. Requires one exact slice ID and 2–8 accelerator slugs. Returns results, missing-evidence reasons, and deltas only against an explicit baseline when units and metric semantics match. The free runtime must boot without payment configuration. Paid operations must advertise x402 offers when configured for Base Sepolia and fail closed when payment configuration is absent. They must never silently become free. Mount and verify: - `GET /api/agent/health` - `GET /api/agent/entrypoints` - `POST /api/agent/entrypoints/:key/invoke` - `POST /api/agent/entrypoints/:key/stream` - `/api/agent/.well-known/agent-card.json` - `/api/agent/.well-known/agent.json` - `/api/agent/.well-known/oasf-record.json` V1 entrypoints do not need streaming behavior; mount the generated route without inventing a streaming use case. ## Update pipeline Provide deterministic `fixture` and `full-source` modes: - Fixture mode parses checked-in representative files and reproduces a checked-in expected-output hash. - Full-source mode parses every allowlisted record at the pinned commit and reproduces the committed full snapshot hash. The pipeline must include a source registry, parser, reviewed alias registry, metric registry, Zod validation, deterministic JSON generation, quarantine report, coverage matrix, changelog/diff report, immutable dataset versions, tombstones for reviewed removals, and atomic promotion only after the complete candidate passes validation. The fixture pack must contain at least one accepted case each for NVIDIA, AMD, and Intel when valid allowlisted evidence exists, one multi-accelerator case, and one deliberately invalid or ambiguous quarantine case. If a vendor has no valid evidence, prove that absence in the coverage matrix. Document manual update and rollback commands. CI must reproduce fixture and full-source hashes and fail on nondeterminism, missing provenance, unknown units, invalid numbers, duplicate IDs, incompatible merges, or uncommitted generated changes. Automated pull-request creation and automatic publication are out of scope. ## Acceptance criteria - Public preview works and includes the required pages and states. - At least one verified slice has three valid results spanning at least two accelerator vendors and two distinct accelerator families. If the pinned source cannot satisfy this, explain the evidence and do not fabricate coverage. - Rankings never mix incompatible benchmark dimensions. - Every ranked row is traceable to the pinned source commit. - Status and preview work without payment configuration. - Rank and compare advertise explicit x402 offers on Base Sepolia when configured and fail closed when not configured. - Discovery schemas match handler outputs. - Next.js routes delegate to the canonical Lucid runtime. - Parser, identity, metric, ranking, pagination/tie, derivation, quarantine, payment-state, and route smoke tests pass. - `bun run type-check`, `bun test`, and the production build pass. - No secrets, private keys, tokens, `.env` files, or unredacted deployment output are committed. - A reviewer can update, roll back, build, and deploy using only documented commands. ## Deliverables - Complete source repository and lockfile. - Public preview URL available through requester review and for seven days afterward. - Initial normalized dataset, immutable manifest, coverage matrix, quarantine report, and changelog. - Importer, normalizer, validator, generator, tests, and CI. - Next.js UI and Lucid runtime. - README, methodology, `DATA_SOURCES.md`, payment guide, update/rollback runbook, and deployment runbook. - Verification report containing exact commands and results. Full dataset export, dedicated chip pages, user accounts, admin CMS, live runtime scraping, original hardware benchmarking, mainnet payments, production DNS, and scheduled update PRs are out of scope. Submissions will be evaluated on data correctness/provenance (30%), Lucid architecture (20%), update pipeline (20%), website quality (15%), automated verification (10%), and documentation/deployment reproducibility (5%). Fabricated benchmark data, context-free aggregate rankings, duplicated adapter payment logic, paid endpoints becoming free, committed secrets, or a failing documented build are grounds for rejection.