{
  "title": "AI judges for data hygiene, Nov 2026",
  "date": "2026-11-06",
  "run_date": "2026-11-06",
  "wording_frozen": "2026-11-06",
  "wording_sha256": "172f2ba6b39449c0",
  "data_source": "a B2B SaaS company's outbound and CRM records",
  "jobs": [
    "dup",
    "addr",
    "same",
    "employer"
  ],
  "models": {
    "decision_models": 25,
    "llms": 8,
    "web_agents": 1,
    "total": 34
  },
  "cases": {
    "dup": 200,
    "addr": 200,
    "same": 200,
    "employer": 102,
    "total": 702
  },
  "repeats": 3,
  "spend": {
    "judge_calls_usd": 8.53,
    "openrouter_usd": 6.44,
    "parallel_usd": 2.09,
    "answer_key_profiles_usd": 1.73,
    "clay_credits": 1399,
    "source": "Judge call invoices: OpenRouter and Parallel."
  },
  "provisional": false,
  "findings": [
    "AI can't clean your CRM on its own yet. If you allow at most 1 mistake in 100, the best model handles 17% of employer cases, 14% of duplicate pairs, 7% of same-person cases and 4% of address cases by itself. A person checks the rest.",
    "Cheap models match expensive ones on duplicates, same-person matches and employers. On duplicates, drex-max is right on 79% of pairs at $0.0123 per 1,000 cases and Gemini 4 Argon on 78% at $1.1463, and our test finds no real difference.",
    "Address picking is the exception. Gemini 4 Argon (77%) and Gemini 3.8 Flash (76%) are clearly ahead of GPT-6.1 Terra (64%) and mammoth-1 (59%), the best decision model. Parallel (64%) is behind them too, but our test cannot say that for sure.",
    "A free rule built from the answer key scores at least as high as the best model on two jobs. It is right on 85% of duplicate pairs against 79%, and on 90% of same-person cases against 88%. These test records give away more than real records may.",
    "OpenAI's GPT-6 Luna Decisions was refused by its provider on 120 of the 200 address questions, the same ones on every try. It got 53% of the 80 it answered."
  ]
}