Agents don't throw errors when they degrade.
They keep confidently producing worse outputs.¶

dedrift catches silent behavioral drift in AI agents — with calibrated false-alarm rates, FDR-controlled alerting, and config-change attribution. Every statistical claim is enforced by simulation tests that run in CI.

Get started The statistics GitHub

$ pip install dedrift

Your observability catches crashes.
It doesn't catch character changes.

Model updates, prompt edits, tool-schema changes, RAG refreshes, provider-side silent updates — all shift agent behavior without a single error in your logs.

Calibrated, not vibes-based

Every detector's false-alarm rate is validated by simulation tests that run in CI on every commit — and inside the release pipeline. A release cannot ship if the statistics fail their own audit.
Two gates before any alert

Benjamini–Hochberg FDR across every test in the check, then a materiality gate on effect size. Statistically significant but practically trivial? You don't get paged.
Frozen canaries, N repetitions

LLM outputs are random, so single-run comparisons are meaningless. Canaries run N times per cycle; dedrift compares distributions — against a rolling window and a frozen golden baseline.
Attribution, honestly framed

Every record carries a config fingerprint. When behavior shifts, the report correlates onset with the nearest config event — "consistent with the model change 5h earlier", never "caused by".
Runs on a laptop

JSONL logs + SQLite. No servers, no docker-compose, no SaaS account. Core installs with zero ML dependencies; embeddings are an optional extra with a pinned-forever model.
Honest about power

Small canary suites have limited detection power. The docs show the math — including which shifts you won't detect at default scale — instead of hiding it.

Try it in two minutes¶

No API keys: the built-in simulator plays an agent whose model version is swapped mid-history.

pip install dedrift
mkdir drift-demo && cd drift-demo
dedrift init
dedrift sim --cycles 8 --change-cycle 7
dedrift baseline set cycle-0000 cycle-0001 cycle-0002
dedrift check

$ dedrift check
Current cycle: cycle-0007
Sudden (vs rolling 4 cycles): DRIFT DETECTED
Cumulative (vs golden 3 cycles): DRIFT DETECTED
Alerts: 168 (q=0.05, materiality-gated)
  [golden] adversarial/refusal two_proportion_z: effect=+21.0 pp, p_adj=0.0009
  [golden] adversarial/format_valid two_proportion_z: effect=-21.9 pp, p_adj=0.0023
  [golden] edge_case/output_words ks: d=+1.89, p_adj=1.4e-20
# dedrift report --out report.md → attribution: nearest config event,
# model fingerprint change, 0.0 h before onset — consistent with the swap.

With your own agent¶

One function and a YAML file of frozen canary inputs:

# myagent.py
def agent_fn(input: dict) -> dict:
    response = my_agent.run(input["text"])
    return {"text": response.text, "structured": response.json}

dedrift canary run --suite canaries.yaml --agent myagent:agent_fn \
    --model 'anthropic/claude-sonnet-5@2026-05-01'
dedrift check && dedrift report

Run it on a schedule. When behavior shifts, you'll know what moved, by how much, since when — and what changed in your stack at the same time.

dedrift Pro¶

Anytime-valid sequential inference (e-processes), conditional production-traffic drift, and importance weighting are part of a separate commercial tier — contact support@dedrift.ai.

dedrift¶

Agents don't throw errors when they degrade.They keep confidently producing worse outputs.¶

Try it in two minutes¶

With your own agent¶

dedrift Pro¶

Agents don't throw errors when they degrade.
They keep confidently producing worse outputs.¶