How AI scoring works

Every score the platform produces is grounded in your evidence — nothing comes from the model's general world knowledge. This page walks through the five-step pipeline.

1 — Chunking + embedding

When you upload a document, it's chunked into ~800-character passages with a 120-character overlap so context isn't lost at boundaries. Each chunk is embedded with Cohere's embed-multilingual-v3.0 (1024 dimensions, EN + AR aware) and stored in Postgres with a pgvector HNSW index for sub-50ms similarity search.

2 — Retrieval

When scoring a question, the platform retrieves the top-12 most relevant chunks via cosine similarity. The query is the question text plus the rubric. RLS filters chunks to your tenant only — no cross-tenant leakage even if the index were corrupted.

3 — Scoring

Our scoring model receives the question, the rubric, and the 12 chunks. It returns:

• A 0–4 score
• A 0.0–1.0 confidence value
• A 2-3 sentence rationale that quotes the chunks by ID
• A list of "unmet evidence gaps" — what evidence would push the score higher

The system prompt explicitly bans the model from speculating beyond the provided chunks. If no chunk supports a level, it scores low and says so.

4 — Independent verification

A separate verifier model reads the scoring output and checks: does every cited chunk actually exist? Does every claim in the rationale appear in the chunks it cites? If verification fails, the score is rejected and surfaced as "insufficient evidence" — better to say don't know than to ship a fabricated number.

5 — Surface

The score lands on the question page with three artefacts: the model's rationale (collapsible), the chunks it cited (each linkable back to the source document and page), and the confidence value. You can dispute any score — disputing triggers a rescore once you've uploaded new evidence.

Cost transparency: every AI call is logged with token counts + dollar cost in Admin → LLM usage. Tenants have monthly caps to prevent runaway bills.