Data freshness

You see how fresh the data is — before you use it

One call shows the last run, the backlog and open background jobs for every data pool.

curl "$API/health/ingest"

{
  "lanes": {
    "us":  { "lastRun": "2026-09-01T02:14:00Z", "stale": false },
    "eu":  { "lastRun": "2026-09-02T03:40:00Z", "stale": false },
    "de":  { "lastRun": null, "stale": true, "reason": "never ran" }
  }
}

What you can do with it

Check before an analysis whether the data is currentPer pool, not globally.
Trace where the figures come fromSource and run per pool.
Track throughputPer source over a window.

Why we built it this way

A data source that fails silently is more dangerous than one that fails loudly — the numbers keep looking right, they just get old.

So every lane reports when it last ran. And a lane that has never run says so explicitly instead of staying quiet. That exact case was invisible here for months: no run, no report, no alarm.

Improvements

Sources that never ran are reported as such instead of silently passed over
The backlog per lane is visible
New coverage overview across all data pools

Endpoints

GET	`/status`	Overview
GET	`/health/ingest`	Freshness per lane
GET	`/coverage`	Coverage per data pool
GET	`/stats/throughput`	Throughput per source

Give feedback or ask a question

Something missing, something off, or an answer you don't understand? Write it here — it reaches us directly.