Consultant Council API

What it's for

Two frontier models answering independently catch each other's factual errors. That is the entire value proposition; everything else is plumbing.

A single model returns a fluent, plausible, well-formatted answer whether or not it is correct. Nothing in the output distinguishes the two cases. You find out downstream, after you have built on it.

Use this service when being wrong is expensive and a second independent opinion costs less than the mistake. For general-purpose generation, one model is cheaper and faster — call the vendor directly.

Quickstart

# Consultations are asynchronous. POST returns a job id immediately.
export COUNCIL_URL=https://council.smartledger.app
export COUNCIL_TOKEN=<your token>

curl -s -X POST "$COUNCIL_URL/v1/consult" \
  -H "Authorization: Bearer $COUNCIL_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"prompt":"Is the Collatz conjecture proved? One sentence."}'

# → {"job_id":"04961d7b-…","state":"queued",
#    "poll":"/v1/jobs/04961d7b-…","stream":"/v1/jobs/04961d7b-…/stream"}

curl -s "$COUNCIL_URL/v1/jobs/04961d7b-…" -H "Authorization: Bearer $COUNCIL_TOKEN"

Authentication

Every /v1/* route requires a bearer token. x-council-token is accepted as an alternative header. /health is deliberately open so monitoring needs no credential.

Authorization: Bearer $COUNCIL_TOKEN

WebSocket handshakes also accept ?token=, because browsers cannot set headers on an upgrade. Over wss:// that query string sits inside the TLS session, but it can still reach reverse-proxy access logs — use the header from server-side clients and treat the query parameter as the browser affordance it is.

Projects & isolation

One deployment serves many projects. Each token maps to a project, and every read, delete, and subscription is scoped to it.

Consultations were never at risk of colliding — they are UUID-keyed, one file each, written atomically. But a single shared token meant no isolation: any caller could list, read, and delete any other caller's records, the firehose broadcast all of them, and a busy project could evict a quiet one under global retention. Tokens now carry identity.

Configuration

# preferred: JSON map of project -> token
COUNCIL_PROJECTS='{"hdlab":"tok_a…","acme":"tok_b…"}'

# compact equivalent
COUNCIL_PROJECTS='hdlab:tok_a…,acme:tok_b…'

# sees every project — for operators, not for applications
COUNCIL_ADMIN_TOKEN='tok_admin…'

# legacy single token still works; its records land in project "default"
COUNCIL_TOKEN='tok…'

What isolation guarantees

Fair share

Each project has its own concurrency budget, so one caller firing fifty consultations cannot occupy every upstream slot or push the shared vendor rate limits into 429 for everyone else.

• MAX_CONCURRENT_PER_PROJECT — default 4
• MAX_CONCURRENT_TOTAL — default 12
• MAX_JOBS_PER_PROJECT — retention budget per project, default 5000

Retention is applied per project. A global cap would let a busy project silently evict a quiet one's records, which for a service whose product is the record is data loss, not a capacity policy.

Upgrading an existing deployment

Jobs written before projects existed carry no stamp and are treated as belonging to default. Keep COUNCIL_TOKEN and nothing changes; add COUNCIL_PROJECTS and new work is scoped while the old records stay reachable through the legacy token.

Choosing a mode

The one decision that matters per request. Pick by how expensive it would be to be wrong.

What debate returns

Each model is shown both answers, labelled by vendor, and asked to adjudicate in five parts:

A · self-correction

Where its own prior answer was wrong or overconfident.

B · cross-examination

Where the other answer is wrong, with corrections.

C · genuine disagreements

Points where they actually differ, with a ruling on each.

D · merged recommendation

One answer, not two.

E · what both missed

The highest-value point absent from either answer.

Section A is what earns the extra calls: models retract and revise when shown an alternative, and the revised confidence is better calibrated than the first pass. Section E reliably surfaces what neither volunteered unprompted.

Verify mode

The other three modes generate judgement about one question. Triaging a findings list is a different shape: N independent claims, most of them false positives, where the useful output is the set the two models disagree about.

{
  "mode": "verify",
  "claims": [
    {"id":"f1", "claim":"BMV was disproved in 2013 by a 3x3 counterexample."},
    {"id":"t1", "claim":"BMV was proved true by Herbert Stahl.",
     "evidence":"optional supporting text"}
  ],
  "prompt": "optional shared context for all claims"
}

Both models see the same list and are told to refute, defaulting to "refuted" when they cannot substantiate a claim. For triage that asymmetry is right: a false negative costs a second look, a false positive costs someone acting on nonsense.

{
  "primary": "verdicts",
  "summary": { "total":6, "upheld":2, "refuted":4, "disputed":0 },
  "upheld": ["t1","t2"], "refuted": ["f1","f2","f3","c1"], "disputed": [],
  "verdicts": [
    { "id":"f1", "claim":"…", "agreed":true, "consensus":"refuted",
      "verdicts":[{"vendor":"openai","verdict":"refuted","confidence":0.95,"reason":"…"}] }
  ]
}

Start a consultation

{
  "prompt":      "required, non-empty string",

  "system":      "optional — matters more than you think, see Writing the prompt",
  "mode":        "fanout",              // fanout | synthesis | debate
  "vendors":     ["openai", "gemini"], // default: both
  "models":      { "openai": "gpt-5.4" }, // per-vendor override
  "effort":      "high",               // OpenAI reasoning: low | medium | high
  "temperature": 0.9,                  // Gemini only
  "maxTokens":   120000,               // per call — reasoning counts against this
  "synthesizer": "openai",             // synthesis mode: who merges
  "wait":        false,                // true = block for the result

  "name":        "the sharding debate",  // human label, shown in listings
  "tags":        ["arch", "q3"],       // filter with ?tag=
  "store_prompt": true,                 // false = do not retain the prompt
  "claims":      ["…"],                // verify mode only
  "schema":      { "…" },               // JSON Schema — see Structured output
  "schema_name": "claim"               // [A-Za-z0-9_-]{1,64}
}

The 202 response echoes the effective parameters actually used, after defaulting, plus a pre-flight cost estimate:

{
  "job_id": "04961d7b-…", "state": "queued",
  "effective": {
    "mode":"debate", "vendors":["openai","gemini"], "effort":"high",
    "maxTokens":120000, "models":{"…"}, "system_provided":true },
  "estimated_cost": { "usd_low":0.06, "usd_high":0.51, "calls":4,
                       "rate_card":"2026-08-29" },
  "poll": "/v1/jobs/…", "stream": "/v1/jobs/…/stream"
}

Without the echo you cannot tell from a stored record whether effort: high was applied, ignored, or clamped — which makes a result non-reproducible, and reproducibility is half of what an audit record is for.

Structured output

Pass a JSON Schema as schema and the answer is constrained at decode time rather than requested in prose. This is not “please reply in JSON”: the sampler will not produce a token sequence the schema forbids.

{
  "mode": "fanout", "vendors": ["openai"],
  "prompt": "Extract the hypotheses of this theorem.",
  "schema_name": "claim",
  "schema": {
    "type": "object",
    "properties": { "hypotheses": { "type": "array", "items": { "type": "string" } } },
    "required": ["hypotheses"],
    "additionalProperties": false
  }
}

What the schema must satisfy

Which calls it applies to

A schema describes an answer to your prompt, so it is applied only where the model is answering you — never to a leg the council writes in its own words.

A model may still decline. That returns as an error on that vendor’s entry reading model refused: …, rather than as text.

Read a consultation

{
  "id": "04961d7b-…", "state": "done", "mode": "synthesis",
  "created_at": "2026-08-29T15:19:36.538Z",
  "started_at": "2026-08-29T15:19:36.545Z",
  "finished_at": "2026-08-29T15:19:42.370Z",
  "duration_ms": 5825, "error": null,
  "prompt": "…the exact submitted prompt…",
  "system": "…the system prompt, if one was given…",
  "prompt_stored": true,
  "name": "the sharding debate", "tags": ["arch"],
  "effective": { /* the parameters actually used */ },
  "usable_answers": 2, "failed_vendors": [],
  "cost": { "usd":0.0015, "calls":2, "rate_card":"2026-08-29" },

  "result": {
    "answers": [
      { "vendor": "openai", "model": "gpt-5.4", "text": "…", "ms": 1962,
        "tokens": { "input":61, "output":8, "reasoning":10, "total":79, "cached":0 },
        "usage":  { /* raw vendor block, unnormalised */ } },
      { "vendor": "gemini", "…": "…" }
    ],
    "synthesis": { "vendor": "openai", "role": "synthesis", "text": "…" },
    "tokens": { "input":112, "output":10, "reasoning":253, "total":375, "calls":2 }
  }
}

Job states

interrupted exists so a poller is never stranded on a job that died with the process. Treat it as retryable.

Result shape by mode

Listing & status

Health is the first thing to check when something looks wrong:

{
  "ok": true, "auth": "bearer", "websocket": true,
  "storage": { "durable": true, "dir": "/data", "stored_jobs": 4 },
  "boot":    { "writer": true, "instance": "990d270d", "interrupted": 0 }
}

• auth: "open" — no token configured. Incident.
• storage.writable: false — the volume is not mounted; running memory-only.
• storage.volume.persisted — see below. Writable is not persistent.
• boot.writer: false — another instance holds the writer lock, or a stale lock is present.

Usage & cost

Token spend per project, so cost is attributable rather than a single shared bill. Admin sees every project; a project sees only itself.

{
  "project": "hdlab",
  "usage": {
    "hdlab": { "consultations":1, "calls":2, "input":113,
                "output":12, "reasoning":270, "total":395, "ms":4571 }
  }
}

Counts come from the normalised tokens block, so output means visible output across both vendors and reasoning is separate. Reasoning usually dominates.

Export

Every visible job as newline-delimited JSON, full records including prompt, result and cost. Retention is 30 days or 5,000 jobs per project — for anyone treating consultations as business documentation that is a deletion deadline, and polling every job by id to beat it is absurd.

curl "$COUNCIL_URL/v1/export?since=2026-08-01" \
  -H "Authorization: Bearer $COUNCIL_TOKEN" > council-archive.ndjson

WebSocket

Sends a snapshot of current state immediately, pushes updates, then closes when terminal. Subscribing after a job finished still returns a snapshot containing the full result, so a late client is never stranded.

const ws = new WebSocket(
  `wss://council.smartledger.app/v1/jobs/${id}/stream?token=${TOKEN}`
);

ws.onmessage = (e) => {
  const { event, job, result, error } = JSON.parse(e.data);
  if (event === 'snapshot') console.log('state:', job.state);
  if (event === 'done')     console.log(result.answers);
  if (event === 'error')    console.error(error);
};

Events: snapshot, started, done, error.

Opens with a snapshot of the 25 most recent jobs, then emits service-wide events. Built for a dashboard; not the right tool for waiting on one job.

The server pings every 30 seconds so idle sockets survive proxy idle timeouts.

Token budget

maxTokens caps output tokens, and reasoning counts against it. A high-effort model can spend its whole budget thinking and emit no visible text at all.

empty completion (status=incomplete, reason=max_output_tokens,
output_tokens=32000, reasoning_tokens=32000). Raise maxTokens or lower effort.

The default is 120,000, which is ample. You only need to think about this if you lower it.

Reading the counts

The tokens block is normalised across vendors; the raw usage block is not. OpenAI folds reasoning into output_tokens; Gemini reports it separately and excludes it from candidatesTokenCount. Summing the obvious field across both would undercount one and double-count the other.

In tokens, output always means visible output and reasoning is always separate. Use result.tokens.total for the cost of the whole consultation — it counts every upstream call, which matters because debate makes four. Reasoning usually dominates.

Writing the prompt

The system prompt does more work than the user prompt

Without a system you get a competent generalist. A domain-specific system prompt changes answer quality more than any other parameter. State the persona, the standard of evidence, and what to refuse:

You are a senior consultant on distributed systems. You are blunt and specific.
Name actual technologies, failure modes, and numbers. If a proposal will not work
at the stated scale, say so plainly and give the version that will. Call out
approaches that are already commoditised. No motivational preamble.

Three habits that pay

Ask for structure

Numbered sections come back cleanly and make debate far more useful, because round two can address each section. Vague prompts produce vague disagreement.

Give the constraints

Hardware, budget, headcount, deadline. Otherwise both models propose things that assume a cluster.

Ask what is missing

“What is the most important point missing from this analysis?” reliably surfaces what neither model raised unprompted.

Cost discipline

Every call spends real credits, and debate spends four times what a single model would.

• Default to fanout. Escalate when the decision warrants it.
• Develop at effort: "low". Iterate on prompt shape cheaply, then raise effort for the real run.
• Do not poll faster than every 10 seconds. Polling costs no API credits but it does hammer the service. Use WebSocket if you need to know immediately.
• Restrict vendors with vendors: ["openai"] when you do not need the cross-check — though at that point, consider calling the vendor directly.

Storage & the record

Consultations are written to a persistent volume and survive restarts and redeploys. Writes are atomic — temp file then rename — so a crash mid-write cannot corrupt a job. Retention defaults to 30 days or 5,000 jobs, whichever comes first.

Writable is not persistent

storage.writable only tells you the path can be written to. Whether it survives a deploy is a different question, and the two were conflated: three consecutive production deploys silently lost every stored consultation while health reported the storage healthy, because the orchestrator handed the container a fresh volume each time.

Each boot now stamps a marker and reads back what the previous boot left:

"storage": {
  "writable": true,
  "volume": { "persisted": true, "boots": 4,
              "first_seen": "2026-08-29T18:00:00.000Z" }
}

Single-writer model

The service is single-writer by design: flat files plus an in-memory index, with a boot sweep that rewrites running jobs to interrupted. Only one instance may run that sweep, so instances coordinate through a heartbeat lock on /data.

• An instance serves traffic immediately on boot and acquires the lock in the background. The destructive sweep runs only once it is provably the sole writer.
• During a rolling deploy the new container waits for the old to exit, and does not reconcile meanwhile — so it cannot mark a still-running job as interrupted out from under the draining instance.
• A container killed with SIGKILL leaves the lock held; the next instance takes over automatically once the heartbeat goes stale, in about twenty seconds.

Upstream resilience

• Every vendor call has an abort timeout, default fifteen minutes. Without it a hung provider would keep a job running and its prompt resident in memory indefinitely.
• 429, 408 and 5xx are retried with exponential backoff and jitter, honouring Retry-After. Rate limits are transient — fanning one prompt to two vendors crosses tokens-per-minute limits easily under load.
• Other 4xx responses fail immediately; they will not succeed on retry.

Runbook