Sign in

Data feeds and downloads

How to fetch the daily files with your API key, and the cursor-paged live feed for incremental delivery. What is inside each file, column by column, is on the daily file reference.

Files are published daily from midnight US Eastern (04:00 UTC, or 05:00 during standard time). Datasets land one at a time through the morning, so a day's listing grows during the run. Downloads shows what is available right now.

What gets published each day

Each day publishes a delta for each of five datasets: domains and DNS on Basic, plus DNS by vantage point, certificates and address history on Pro. A full snapshot for seeding a new system is published alongside them.

A delta holds every record that changed since that dataset's previous file. Consecutive deltas overlap by a few hours so nothing falls between them; apply rows as upserts on each dataset's key, in day order. Files stay on the download page for 14 days, and every file we publish is archived permanently.

The daily file reference covers the exact rules, the manifest, and every column.

Formats: csv.zst and json.zst

Each file ships in both formats. Pick one and ignore the other.

csv.zst opens with a header row naming the columns, in the order the file reference gives. json.zst holds one JSON object per line, keyed by those same column names. Stream it a line at a time and the file never has to fit in memory.

Both arrive Zstandard-compressed. Decompress as you read: zstd -dc, or natively in DuckDB, polars, pandas and ClickHouse, all of which read .zst without a separate step. We moved off gzip because on this data Zstandard is both smaller and markedly faster to decompress.

Fetching a file with your API key

Your API key works as a bearer token. There is no browser step and no session, so a download drops straight into cron.

# list what is available
curl -sS -H "Authorization: Bearer $ISSUED_KEY" \
  https://issued.live/api/v1/files

# fetch one (-f keeps an error response out of the file)
curl -sSf -H "Authorization: Bearer $ISSUED_KEY" -O \
  https://issued.live/api/v1/files/YYYY-MM-DD/delta.domains.csv.zst

Each day's files are named {kind}.{dataset}.{format}.zst and described by that day's manifest.json. The file reference lists them.

Send the key in the Authorization header. Query strings land in access logs, proxy logs and Referer headers, and a key that reaches a log has to be rotated.

The same files are listed on Downloads, and a download there and one with your key draw on the same allowance.

One download per file per day

Each file is generated once a day. Keep what you fetched and read it locally.

Re-downloading is the main way bulk access gets expensive, and this cap keeps the price flat for everyone. Pricing lists it beside the per-minute API limits.

Automating the download

The allowance is one download per file per day, shared between this API and the download buttons on Downloads. A script has to cope with three things: a file that is not generated yet, an allowance already spent, and a transfer that dies part way through.

What the API answers

StatusMeaningWhat your script should do
200The file is streaming. Write it, then verify the length and the Zstandard checksum.
206A partial transfer is resuming. Append. A resumed transfer does not spend a second allowance.
302The file has been moved to cold storage; the Location is a signed, short-lived URL for it. Follow it (curl -L). Every full snapshot and older days answer this way. The allowance is spent when the redirect is issued, so a client that does not follow it loses that file until 00:00 UTC. Do not send your key to the signed URL; it does not need one, and curl drops the header across hosts anyway.
416Your partial file already holds every byte. Verify it and keep it.
404That file is not published for that day. Stop and try later. Do not retry in a tight loop. A dataset missing from a day's manifest can still be added to that day later; otherwise its rows are in its next file. The gap check tells the two apart.
429Today's allowance for this file is spent. Skip this file and carry on with the others. Do not sleep through Retry-After: it counts down to the 00:00 UTC reset. The first scheduled run after the reset takes the file.
401The key is missing, wrong or rotated. Stop. Retrying cannot fix it.

A 429 here is not a rate limit you can wait out in seconds. It means something already took this file today, which may have been you clicking the button on Downloads.

A complete example

This lists what is available, takes only the files you choose, skips anything it already holds, and resumes a transfer that dropped part way instead of starting over. A spent allowance is a normal outcome, not an error. A file reaches its final name only after its size, Zstandard checksum and SHA-256 all check out. Run it from cron once an hour: a run that finds another still going exits at once, a transfer that stalls for five minutes is abandoned and resumed by the next run, and a run with nothing new does nothing.

#!/usr/bin/env bash
# issued.live daily file sync, for Linux. Needs bash, curl, jq, zstd, flock (util-linux)
# and GNU coreutils, and ISSUED_KEY in the environment.
set -euo pipefail

DEST="${DEST:-/var/lib/issued-files}"
API="https://issued.live/api/v1/files"
AUTH="Authorization: Bearer ${ISSUED_KEY:?set ISSUED_KEY}"
# Which files to take. The CSV and JSON of a dataset hold the same rows, so take one
# format. This takes the Basic deltas as CSV; widen it to the datasets your plan includes.
WANT='^delta\.(domains|dns)\.csv\.zst$'

mkdir -p "$DEST"
# One run at a time. Two runs resuming the same .part interleave their bytes.
exec 9>"$DEST/.lock"
flock -n 9 || { rc=$?; [ "$rc" = 1 ] && exit 0; echo "flock failed (exit $rc)" >&2; exit 1; }

# 1. What does the server have? One cheap call, and it does not spend an allowance.
listing=$(curl -sSf --connect-timeout 30 --max-time 120 -H "$AUTH" "$API") || {
  echo "listing failed; curl's error is above" >&2; exit 1; }

# 2. Walk the files we want. Each listing entry carries day, name, bytes and sha256.
echo "$listing" | jq -r --arg want "$WANT" \
  '.files[] | select(.name | test($want)) | [.day, .name, .bytes, .sha256] | @tsv' |
while IFS=$'\t' read -r day name bytes sha; do
  out="$DEST/$day/$name"
  part="$out.part"
  mkdir -p "$DEST/$day"

  # Already hold exactly this file? Compare the hash, not the size: a file can be
  # regenerated with different bytes.
  if [ -f "$out" ] && [ "$(cat "$out.sha256" 2>/dev/null)" = "$sha" ]; then
    continue
  fi

  have=$(stat -c%s "$part" 2>/dev/null || echo 0)
  if [ "$have" -lt "$bytes" ]; then
    # -C - resumes from the end of the .part; a resumed transfer continues the download
    # that already spent the allowance. -f keeps an error response out of the file.
    # No --retry: curl would sleep through a 429's Retry-After for hours. A transfer slower
    # than 1 KB/s for five minutes is abandoned, and the next run resumes it.
    #
    # -L IS REQUIRED, NOT OPTIONAL. Older days and every full snapshot are served as a 302 to
    # object storage; without -L curl reports 302, writes nothing, and the allowance for that
    # file is already spent. curl does not forward the Authorization header across hosts, so
    # the key is not sent to the storage provider.
    rc=0
    code=$(curl -sS -f -L -w '%{http_code}' -o "$part" -C - \
             --connect-timeout 30 --speed-limit 1024 --speed-time 300 \
             -H "$AUTH" "$API/$day/$name") || rc=$?
    case "$code" in
      200|206|416) ;;   # data arrived, or the .part already holds everything
      404) echo "$day/$name not published (yet)"; continue ;;
      429) echo "$day/$name: allowance already spent today, resuming after 00:00 UTC"; continue ;;
      401) echo "key rejected, stopping" >&2; exit 1 ;;
      *)   echo "$day/$name: HTTP $code (curl exit $rc), trying again next run" >&2; continue ;;
    esac
    have=$(stat -c%s "$part" 2>/dev/null || echo 0)
    if [ "$have" -lt "$bytes" ]; then
      # The connection dropped part way. Keep the .part: the next run resumes it.
      echo "$day/$name: $have of $bytes bytes so far, the next run resumes"
      continue
    fi
  fi

  # All bytes are here. Verify before trusting the file; a truncated or mismatched file is
  # removed so the next run fetches it again instead of skipping it.
  if [ "$have" -gt "$bytes" ] || ! zstd -tq "$part" || ! echo "$sha  $part" | sha256sum -c --status; then
    echo "$day/$name failed verification, removing" >&2
    rm -f "$part"
    continue
  fi
  mv -f "$part" "$out"
  echo "$sha" > "$out.sha256"
  echo "fetched $out"
done

Points that matter in practice

File structures

The column layout of every file, the encoding rules, sample rows in both formats, and the manifest format are on the daily file reference:

Live feed: newly registered domains, cursor-paged

GET https://issued.live/api/v1/nrd

This endpoint is live today and needs a Pro key -- a Basic key is refused here, as it is on every endpoint under /api/v1/ except the files. It returns newly registered domains in ascending time order from a cursor, so a consumer resumes exactly where it stopped.

curl -sS "https://issued.live/api/v1/nrd?certified=1&limit=500" \
  -H "Authorization: Bearer $ISSUED_KEY"
{
  "certified": true,
  "count": 500,
  "next_cursor": "1789340111.click.joffulm",
  "caught_up": false,
  "window": { "since": "2026-09-12T04:00:00Z", "until": "2026-09-14T04:00:00Z" },
  "coverage": { "from": "2026-08-14T00:00:00Z", "to": "2026-09-14T03:58:02Z",
                "max_span_months": 13 },
  "domains": [
    { "seen_at": "2026-09-13T22:55:11Z", "domain": "joffulm.click", "tld": "click" }
  ]
}

The domains array is shown truncated to its last row, which is the row next_cursor points at. Every seen_at falls inside window.

How the cursor works

Every response carries next_cursor. Pass it back as ?after= on the next call, and persist it between runs.

Store the cursor on every page, including pages where caught_up is true. It is what lets a consumer that was away for a week resume without a gap.

caught_up reports whether anything is waiting right now. The server reads one row past your limit to decide, so the flag is an observation of the data.

A caught-up feed is still running. Back off, then poll again with the same cursor.

ParameterDefaultMeaning
after48 hours backCursor. Your position within the window, and the thing that guarantees you skip and repeat nothing.
sincethe cursorLower bound. RFC 3339, YYYY-MM-DD, or a unix timestamp; UTC when no zone is given. Whichever of since and your cursor is further along wins.
untilnowUpper bound, always clamped to now.
tld-Restrict to one TLD. A leading dot and any case are accepted.
limit50Rows per page, up to 500. An unparseable value falls back to the default.
certified-1 or true restricts the feed to newly registered domains that have already had a certificate issued.
max_hours_to_cert-Restrict to domains certificated within N hours of registration. Requires certified=1; sending it alone returns a 400.

A window may span at most 13 calendar months per request. A wider one comes back as 400 window_too_wide naming the limit. Walk a long range in slices, moving both bounds forward.

Each response also carries coverage, holding the from and to of the history we hold plus max_span_months, all read from the data rather than hardcoded. It is what separates an empty page inside coverage from a request that reached past our history.

page, offset, skip and cursor each return a 400 naming after as the replacement. date returns a 400 pointing at since and until, where a single day is since=2026-09-01&until=2026-09-02.

A missing or unrecognized key returns 401 unauthorized with a WWW-Authenticate challenge.

When to use the feed and when to use the files

Reach for the feed when you want events within minutes, and you want every one of them. Registration and certification signals decay fast.

Reach for the files when you want the whole corpus, or a day's changes across every field. The feed carries registrations; the files carry the full record: domains, DNS, DNS by vantage point, certificates and address history, as the file reference lays out.

Most subscribers run both. The files seed and refresh the base load overnight, and the feed keeps the same-day arrivals flowing in between.

Where to go next

↑ Top