<!-- Source: https://issued.live/developers/feeds -->
issued.live
 /
 Data feeds and downloads

# Data feeds and downloads

How to fetch the daily files with your API key, and the cursor-paged live
 feed for incremental delivery. What is inside each file, column by column, is on the
 [daily file reference](/developers/files).

**Files are published daily from midnight US Eastern** (04:00 UTC, or 05:00 during standard time).
 Datasets land one at a time through the morning, so a day's listing grows during the run.
 [Downloads](/downloads) shows what is available right now.

- [What gets published each day](#published)
- [Formats](#formats)
- [Fetching with your API key](#fetch)
- [Automating the download](#automating)
- [File structures](#structures)
- [Live NRD feed](#live)
- [Where to go next](#next)

## What gets published each day

Each day publishes a delta for each of five datasets: domains and DNS on Basic, plus DNS
 by vantage point, certificates and address history on Pro. A full snapshot for seeding a new
 system is published alongside them.

A delta holds every record that changed since that dataset's previous file. Consecutive
 deltas overlap by a few hours so nothing falls between them; apply rows as upserts on each
 dataset's key, in day order. Files stay on the download page for 14 days, and every file we
 publish is archived permanently.

The [daily file reference](/developers/files#deltas) covers the exact rules,
 the manifest, and every column.

## Formats: csv.zst and json.zst

Each file ships in both formats. Pick one and ignore the other.

`csv.zst` opens with a header row naming the columns, in the order the
 [file reference](/developers/files) gives. `json.zst` holds one JSON object per line, keyed by those same
 column names. Stream it a line at a time and the file never has to fit in memory.

Both arrive Zstandard-compressed. Decompress as you read: `zstd -dc`, or
 natively in DuckDB, polars, pandas and ClickHouse, all of which read `.zst`
 without a separate step. We moved off gzip because on this data Zstandard is both smaller
 and markedly faster to decompress.

## Fetching a file with your API key

Your API key works as a bearer token. There is no browser step and no session, so a
 download drops straight into cron.

```
# list what is available
curl -sS -H "Authorization: Bearer $ISSUED_KEY" \
  https://issued.live/api/v1/files

# fetch one (-f keeps an error response out of the file)
curl -sSf -H "Authorization: Bearer $ISSUED_KEY" -O \
  https://issued.live/api/v1/files/YYYY-MM-DD/delta.domains.csv.zst
```

Each day's files are named `{kind}.{dataset}.{format}.zst`
 and described by that day's `manifest.json`. The
 [file reference](/developers/files#names) lists them.

Send the key in the `Authorization` header. Query strings land in access logs,
 proxy logs and `Referer` headers, and a key that reaches a log has to be
 rotated.

The same files are listed on [Downloads](/downloads), and a
 download there and one with your key draw on the same allowance.

### One download per file per day

Each file is generated once a day. Keep what you fetched and read it locally.

Re-downloading is the main way bulk access gets expensive, and this cap keeps the price
 flat for everyone. [Pricing](/pricing) lists it beside the per-minute API
 limits.

## Automating the download

The allowance is one download per file per day, shared between this API and the download
 buttons on [Downloads](/downloads). A script has to cope with three things: a file that is not
 generated yet, an allowance already spent, and a transfer that dies part way through.

### What the API answers

| Status | Meaning | What your script should do |
|---|---|---|
| `200` | The file is streaming. | Write it, then verify the length and the Zstandard checksum. |
| `206` | A partial transfer is resuming. | Append. A resumed transfer does not spend a second allowance. |
| `302` | The file has been moved to cold storage; the
 `Location` is a signed, short-lived URL for it. | **Follow it** (`curl -L`). Every full snapshot and older
 days answer this way. The allowance is spent when the redirect is issued, so a client
 that does not follow it loses that file until 00:00 UTC. Do not send your key to
 the signed URL; it does not need one, and curl drops the header across hosts anyway. |
| `416` | Your partial file already holds every byte. | Verify it and keep it. |
| `404` | That file is not published for that day. | Stop and try later. Do not retry in a tight loop. A dataset missing from a day's manifest can still be added to that day later; otherwise its rows are in its next file. The [gap check](/developers/files#applying) tells the two apart. |
| `429` | Today's allowance for this file is spent. | Skip this file and carry on with the others. Do not sleep through `Retry-After`: it counts down to the 00:00 UTC reset. The first scheduled run after the reset takes the file. |
| `401` | The key is missing, wrong or rotated. | Stop. Retrying cannot fix it. |

A `429` here is not a rate limit you can wait out in seconds. It means something
 already took this file today, which may have been you clicking the button on
 [Downloads](/downloads).

### A complete example

This lists what is available, takes only the files you choose, skips anything it already
 holds, and resumes a transfer that dropped part way instead of starting over. A spent
 allowance is a normal outcome, not an error. A file reaches its final name only after its
 size, Zstandard checksum and SHA-256 all check out. Run it from cron once an hour: a run
 that finds another still going exits at once, a transfer that stalls for five minutes is
 abandoned and resumed by the next run, and a run with nothing new does nothing.

```
#!/usr/bin/env bash
# issued.live daily file sync, for Linux. Needs bash, curl, jq, zstd, flock (util-linux)
# and GNU coreutils, and ISSUED_KEY in the environment.
set -euo pipefail

DEST="${DEST:-/var/lib/issued-files}"
API="https://issued.live/api/v1/files"
AUTH="Authorization: Bearer ${ISSUED_KEY:?set ISSUED_KEY}"
# Which files to take. The CSV and JSON of a dataset hold the same rows, so take one
# format. This takes the Basic deltas as CSV; widen it to the datasets your plan includes.
WANT='^delta\.(domains|dns)\.csv\.zst$'

mkdir -p "$DEST"
# One run at a time. Two runs resuming the same .part interleave their bytes.
exec 9>"$DEST/.lock"
flock -n 9 || { rc=$?; [ "$rc" = 1 ] && exit 0; echo "flock failed (exit $rc)" >&2; exit 1; }

# 1. What does the server have? One cheap call, and it does not spend an allowance.
listing=$(curl -sSf --connect-timeout 30 --max-time 120 -H "$AUTH" "$API") || {
  echo "listing failed; curl's error is above" >&2; exit 1; }

# 2. Walk the files we want. Each listing entry carries day, name, bytes and sha256.
echo "$listing" | jq -r --arg want "$WANT" \
  '.files[] | select(.name | test($want)) | [.day, .name, .bytes, .sha256] | @tsv' |
while IFS=$'\t' read -r day name bytes sha; do
  out="$DEST/$day/$name"
  part="$out.part"
  mkdir -p "$DEST/$day"

  # Already hold exactly this file? Compare the hash, not the size: a file can be
  # regenerated with different bytes.
  if [ -f "$out" ] && [ "$(cat "$out.sha256" 2>/dev/null)" = "$sha" ]; then
    continue
  fi

  have=$(stat -c%s "$part" 2>/dev/null || echo 0)
  if [ "$have" -lt "$bytes" ]; then
    # -C - resumes from the end of the .part; a resumed transfer continues the download
    # that already spent the allowance. -f keeps an error response out of the file.
    # No --retry: curl would sleep through a 429's Retry-After for hours. A transfer slower
    # than 1 KB/s for five minutes is abandoned, and the next run resumes it.
    #
    # -L IS REQUIRED, NOT OPTIONAL. Older days and every full snapshot are served as a 302 to
    # object storage; without -L curl reports 302, writes nothing, and the allowance for that
    # file is already spent. curl does not forward the Authorization header across hosts, so
    # the key is not sent to the storage provider.
    rc=0
    code=$(curl -sS -f -L -w '%{http_code}' -o "$part" -C - \
             --connect-timeout 30 --speed-limit 1024 --speed-time 300 \
             -H "$AUTH" "$API/$day/$name") || rc=$?
    case "$code" in
      200|206|416) ;;   # data arrived, or the .part already holds everything
      404) echo "$day/$name not published (yet)"; continue ;;
      429) echo "$day/$name: allowance already spent today, resuming after 00:00 UTC"; continue ;;
      401) echo "key rejected, stopping" >&2; exit 1 ;;
      *)   echo "$day/$name: HTTP $code (curl exit $rc), trying again next run" >&2; continue ;;
    esac
    have=$(stat -c%s "$part" 2>/dev/null || echo 0)
    if [ "$have" -lt "$bytes" ]; then
      # The connection dropped part way. Keep the .part: the next run resumes it.
      echo "$day/$name: $have of $bytes bytes so far, the next run resumes"
      continue
    fi
  fi

  # All bytes are here. Verify before trusting the file; a truncated or mismatched file is
  # removed so the next run fetches it again instead of skipping it.
  if [ "$have" -gt "$bytes" ] || ! zstd -tq "$part" || ! echo "$sha  $part" | sha256sum -c --status; then
    echo "$day/$name failed verification, removing" >&2
    rm -f "$part"
    continue
  fi
  mv -f "$part" "$out"
  echo "$sha" > "$out.sha256"
  echo "fetched $out"
done
```

### Points that matter in practice

- **Verify before you trust the file.** A truncated `.zst` is the worst
 failure mode, because the next run finds a file on disk, assumes it is done, and skips it
 forever. Check the Zstandard checksum and the SHA-256 the listing gives you, and keep a file
 under its final name only once both pass.

- **Apply deltas in day order, as upserts.** Consecutive deltas overlap, and
 address history needs one extra rule. The
 [file reference](/developers/files#applying) has the details and a tested
 example.

- **Do not let curl retry on its own.** `--retry` treats a
 `429` as retryable and sleeps for its `Retry-After`, which can be
 hours. Let the next scheduled run try again.

- **Resume rather than restart.** `curl -C -` continues from the byte
 it reached. Restarting a 30 GB transfer because the connection dropped at 28 GB wastes your
 bandwidth and ours, and on a file whose allowance is already spent a restart is not
 available to you.

- **Do not schedule against 00:00 UTC.** The allowance resets then and every
 script in the world fires at midnight. Pick a minute that suits you; the files do not move.

- **Decide who downloads.** The allowance is per file, not per key and not per
 person. If a script and a human both need the file, have the script fetch it and let the
 human read it from wherever the script put it. Clicking the button on
 [Downloads](/downloads) consumes
 the copy your cron job was going to take.

## File structures

The column layout of every file, the encoding rules, sample rows in both
 formats, and the manifest format are on the [daily file
 reference](/developers/files):

- [Basic: domains](/developers/files#domains)
- [Basic: DNS](/developers/files#dns)
- [Pro: DNS by vantage point](/developers/files#dns-vantage)
- [Pro: certificates](/developers/files#certificates)
- [Pro: address history](/developers/files#address-history)

## Live feed: newly registered domains, cursor-paged

GET https://issued.live/api/v1/nrd

This endpoint is live today and needs a **Pro** key -- a Basic key is
 refused here, as it is on every endpoint under `/api/v1/` except the files. It
 returns newly registered domains in
 ascending time order from a cursor, so a consumer resumes exactly where it stopped.

```
curl -sS "https://issued.live/api/v1/nrd?certified=1&limit=500" \
  -H "Authorization: Bearer $ISSUED_KEY"
```

```
{
  "certified": true,
  "count": 500,
  "next_cursor": "1789340111.click.joffulm",
  "caught_up": false,
  "window": { "since": "2026-09-12T04:00:00Z", "until": "2026-09-14T04:00:00Z" },
  "coverage": { "from": "2026-08-14T00:00:00Z", "to": "2026-09-14T03:58:02Z",
                "max_span_months": 13 },
  "domains": [
    { "seen_at": "2026-09-13T22:55:11Z", "domain": "joffulm.click", "tld": "click" }
  ]
}
```

The `domains` array is shown truncated to its last row, which is
 the row `next_cursor` points at. Every `seen_at` falls inside
 `window`.

### How the cursor works

Every response carries `next_cursor`. Pass it back as `?after=` on
 the next call, and persist it between runs.

Store the cursor on every page, including pages where `caught_up` is true. It
 is what lets a consumer that was away for a week resume without a gap.

`caught_up` reports whether anything is waiting right now. The server reads one
 row past your `limit` to decide, so the flag is an observation of the data.

A caught-up feed is still running. Back off, then poll again with the same cursor.

| Parameter | Default | Meaning |
|---|---|---|
| `after` | 48 hours back | Cursor. Your position within the window, and the thing that guarantees you skip and repeat nothing. |
| `since` | the cursor | Lower bound. RFC 3339, `YYYY-MM-DD`, or a unix timestamp; UTC when no zone is given. Whichever of `since` and your cursor is further along wins. |
| `until` | now | Upper bound, always clamped to now. |
| `tld` | - | Restrict to one TLD. A leading dot and any case are accepted. |
| `limit` | 50 | Rows per page, up to 500. An unparseable value falls back to the default. |
| `certified` | - | `1` or `true` restricts the feed to newly registered domains that have already had a certificate issued. |
| `max_hours_to_cert` | - | Restrict to domains certificated within N hours of registration. Requires `certified=1`; sending it alone returns a 400. |

A window may span at most 13 calendar months per request. A wider one comes back as
 `400 window_too_wide` naming the limit. Walk a long range in slices, moving both
 bounds forward.

Each response also carries `coverage`, holding the `from` and
 `to` of the history we hold plus `max_span_months`, all read from the
 data rather than hardcoded. It is what separates an empty page inside coverage from a
 request that reached past our history.

`page`, `offset`, `skip` and `cursor` each
 return a 400 naming `after` as the replacement. `date` returns a 400
 pointing at `since` and `until`, where a single day is
 `since=2026-09-01&until=2026-09-02`.

A missing or unrecognized key returns `401 unauthorized` with a
 `WWW-Authenticate` challenge.

### When to use the feed and when to use the files

Reach for the feed when you want events within minutes, and you want every one of them.
 Registration and certification signals decay fast.

Reach for the files when you want the whole corpus, or a day's changes across every field.
 The feed carries registrations; the files carry the full record: domains, DNS, DNS by
 vantage point, certificates and address history, as the
 [file reference](/developers/files) lays out.

Most subscribers run both. The files seed and refresh the base load overnight, and the
 feed keeps the same-day arrivals flowing in between.

## Where to go next

- [Developer hub](/developers): what each API covers and which plan reaches it.
- [Public API](/developers/api): the free and Basic domain lookup.
- [Pro API](/developers/pro): reverse IP, pattern search, certificate and SPKI pivots.
- [Daily file reference](/developers/files): datasets, columns, the manifest,
 and how to apply the daily deltas.

- [Contact](/contact): questions about the files or a column you need.

↑ Top

---

issued.live, operated by Tuxxin LLC. Source: https://issued.live/developers/feeds