<!-- Source: https://issued.live/developers/files -->
issued.live
 /
 Daily file reference

# Daily file reference

What is inside each daily file, how the daily deltas work, and how to apply
 them. How to fetch the files with your API key is on
 [Data feeds and downloads](/developers/feeds).

This page describes what is inside each file.
 [Downloads](/downloads) lists what is published right now, with sizes and row
 counts, and `GET /api/v1/files` returns the same listing to a script.

- [What each day holds](#overview)
- [File names](#names)
- [The manifest](#manifest)
- [Deltas](#deltas)
- [Applying a delta](#applying)
- [Full snapshot](#snapshots)
- [Retention and allowance](#retention)
- [Encoding rules](#encoding)
- [Domains](#domains)
- [DNS](#dns)
- [DNS by vantage point](#dns-vantage)
- [Certificates](#certificates)
- [Address history](#address-history)
- [Sample rows](#samples)
- [Where to go next](#next)

## What each day holds

Files are published into one directory per day, named for the UTC date the run started.
 Generation starts at midnight US Eastern (04:00 UTC, 05:00 during standard time) and the datasets publish one at a time over the
 following hours.

A day holds up to five datasets, each in two formats, plus a `manifest.json`
 describing them. Basic covers the first two datasets; Pro covers all five.
 [Compare the plans](/pricing).

| Dataset | One row per | Key to upsert on | Plan |
|---|---|---|---|
| [`domains`](#domains) | registrable domain | `domain` | Basic |
| [`dns`](#dns) | name and record type, merged across vantage points | `name`, `record_type` | Basic |
| [`dns_vantage`](#dns-vantage) | name, record type and vantage point | `name`, `record_type`, `vantage` | Pro |
| [`certificates`](#certificates) | certificate | `cert_id` | Pro |
| [`address_history`](#address-history) | domain and address | `domain`, `ip` | Pro |

A key appears at most once in a file. Every row carries the record's complete current
 state, not just the fields that changed.

## File names

Each file is named `{kind}.{dataset}.{format}.zst`. The kind
 is `delta` for a daily file and `snapshot` for a full one, and the
 format is `csv` or `json`.

```
2026-09-17/manifest.json
2026-09-17/delta.domains.csv.zst
2026-09-17/delta.domains.json.zst
2026-09-17/delta.dns.csv.zst
2026-09-17/delta.dns.json.zst
2026-09-17/delta.dns_vantage.csv.zst
...
```

The CSV and JSON file of a dataset for a day are cut from the same data. They hold the
 same rows, with the same values, in the same order, so pick one format and ignore the
 other.

## The manifest: what a day contains and what each file covers

`manifest.json` lists every file published for the day, sorted by name. It is
 the source of truth for what a day contains.

```
{
  "day": "2026-09-17",
  "generated": "2026-09-17T05:41:09Z",
  "files": [
    {
      "day": "2026-09-17",
      "name": "delta.domains.csv.zst",
      "dataset": "domains",
      "kind": "delta",
      "format": "csv",
      "since": "2026-09-16T00:00:00Z",
      "as_of": "2026-09-17T04:00:02Z",
      "rows": 61204877,
      "bytes": 1597224401,
      "sha256": "0f1c...e9a4"
    }
  ]
}
```

| Field | Type | Meaning |
|---|---|---|
| `day` | string | The day directory, `YYYY-MM-DD`. |
| `name` | string | The file name within the day. |
| `dataset` | string | One of the five datasets. |
| `kind` | string | `delta` or `snapshot`. |
| `format` | string | `csv` or `json`. |
| `since` | timestamp \| null | The lower bound this delta covers; see [Deltas](#deltas). null for a snapshot. |
| `as_of` | timestamp | When the data for this file was read. Everything recorded before this moment is in the file. |
| `rows` | integer | Data rows in the file, the same for both formats. The CSV header is not counted. |
| `bytes` | integer | Size of the compressed file. |
| `sha256` | string | SHA-256 of the compressed file, 64 lowercase hex characters. |

**A file listed in the manifest is complete.** Files are written elsewhere
 and moved into the day only after their row counts and compression have been verified.

**The manifest grows during the run.** Each dataset is added when it
 publishes. A day where nothing has been published yet has no manifest at all.

**A missing dataset is not an empty one.** If a dataset could not be
 generated, it is left out of that day's manifest and its next file covers the gap: its
 `since` reaches back to cover the missed days. A dataset can also be added to a
 day's manifest later, including after that UTC day has ended, when it is generated
 again.

**Trust `sha256`, not the size.** A file can be regenerated,
 which changes its bytes and its hash. While that happens its entries briefly leave the
 manifest, so a manifest can for a moment list fewer files, or none at all.

A CSV file can have more lines than `rows` plus one, because a quoted value can
 contain a line break. Count records with a CSV parser, not with line counts.

## Deltas: a lower bound and no upper bound

A delta holds every record that changed on or after its `since` up to the
 moment the file was generated.

`since` is 00:00 UTC of the day this dataset's previous file was
 published for. On a normal day that is yesterday. If a day was missed, it stays at the last
 published day, so the next file is larger and nothing falls between two files.

Because there is no upper bound, **consecutive files overlap**. The
 2026-09-17 file covers from 2026-09-16 00:00, and the 2026-09-16 file already carried
 changes up to its own `as_of` that morning. Records that changed in that overlap
 appear in both files. Applying rows as upserts makes this harmless.

| Dataset | A row is in the delta when |
|---|---|
| `domains` | Anything about the domain was updated on or after `since`: registration data, address, certificate or zone sighting. |
| `dns` | The answer for the name and record type changed at any vantage point on or after `since`. |
| `dns_vantage` | The answer at that vantage point changed on or after `since`. |
| `certificates` | We recorded the certificate on or after `since`, for the first time or again. |
| `address_history` | The address appeared in a new or changed answer for the domain on or after `since`. |

**Late arrivals still ship.** Some records reach us after the fact and carry
 an older timestamp, such as a zone file that is a day old when we process it. A delta also
 includes every record we received on or after `since`, so these arrive in the
 next file even though their dates are earlier than `since`.

**Historical backfill does not.** When we import older data in bulk, it
 reaches you through the [full snapshot](#snapshots) rather than as a sudden
 oversized daily file.

## Applying a delta

Upsert every row on its dataset's key, replacing the whole row. Apply each dataset's
 files in day order. Replaying an older file after a newer one would move rows backwards.

- **Check for a gap before applying.** Keep, for each dataset, the day of
 the last file you applied. If a file's `since` is later than 00:00 UTC of
 that day, a file between the two is missing: apply the missing one first.

- **Nothing is ever deleted by a delta.** A record that stops appearing keeps
 its last values. A DNS record type that stops answering, for example, remains with the
 values it last had.

- **Address history: `current` moves between rows.** A delta carries
 the address a domain moved to, with `current` set to true. It does not re-send
 the address the domain moved away from. When you apply a row with `current`
 true, set `current` to false on every other address you hold for that
 domain.

- **Confirmation times refresh through the snapshot.** A delta is driven by
 changes. When a name is re-resolved and its answer has not changed, its
 `last_confirmed` or `last_seen` moves forward in our data but does not
 by itself put the row in a delta for domains, DNS or certificates; address history is keyed
 on the sighting itself, so it does move. The [full snapshot](#snapshots) carries
 the current confirmation time for every record.

An example for the DuckDB command-line client, applying one day's domains delta to a
 local table keyed on `domain`. Verify the file first, against the manifest's
 `sha256` and with `zstd -t`: DuckDB reads a truncated
 `.zst` without reporting an error.

```
.bail on
BEGIN;
DROP TABLE IF EXISTS d;
CREATE TEMP TABLE d AS
  SELECT * FROM read_csv('delta.domains.csv.zst', header = true, all_varchar = true,
                         allow_quoted_nulls = false);
CREATE TABLE IF NOT EXISTS domains AS SELECT * FROM d LIMIT 0;
DELETE FROM domains WHERE domain IN (SELECT d.domain FROM d);
INSERT INTO domains SELECT * FROM d;
COMMIT;
```

Each line guards against a failure we reproduced. Without
 `.bail on` the client carries on past an error, and `COMMIT` keeps
 the `DELETE` of a file that did not load. With `domain` unqualified
 in the subquery, a file that loads without a `domain` column makes the
 `DELETE` remove every row. Without `allow_quoted_nulls = false`,
 empty values load as NULL instead of empty strings. `DROP TABLE IF EXISTS d`
 lets one session apply several days in turn.

## The full snapshot

The whole corpus for each dataset is published as `snapshot.{dataset}` files,
 with the same columns and encoding as the deltas. Only the latest is kept on the download
 page. A snapshot's manifest entries carry `kind` `snapshot` and a null
 `since`. Seed a new system from it, then apply the daily deltas published after
 its `as_of`.

## Retention and the download allowance

**Files stay downloadable here for 14 days, counting today. Every file we publish is
 archived permanently.** Those are two different numbers and it is worth keeping them
 apart: 14 days is how long a file stays on the download page, not how long the data lives.
 A consumer that falls further behind than that has missed deltas and needs to re-seed: take
 the latest [full snapshot](#snapshots) from the
 [download page](/downloads) and apply the deltas published after its
 `as_of`.

The archive is what makes the 14-day certificate window in the database a working window
 rather than a limit. Certificate detail ages out of the API after 14 days; the daily file
 that recorded it does not age out at all.

Each file can be downloaded once per day, shared between the API and the buttons in your
 account. How to automate the download within that allowance is covered in
 [Automating the download](/developers/feeds#automating).

## Encoding rules shared by every file

| Topic | Rule |
|---|---|
| Compression | Zstandard. DuckDB, polars, pandas and ClickHouse read `.zst` directly; otherwise stream through `zstd -dc`. |
| CSV | A header row naming the columns in the order given below. Every string value and every header name is double-quoted, with a literal `"` written as `""`. Integers and booleans are not quoted. Lines end with LF. |
| JSON | One object per line, keys in column order. Standard JSON escapes apply, and `/` is written as `\/`. |
| Arrays | A real JSON array in the JSON file. In the CSV file the cell holds the same JSON text, quoted as a CSV string. |
| Timestamps | `YYYY-MM-DDTHH:MM:SSZ` in UTC, whole seconds. |
| Unknown values | An empty string, `""`, in both formats. Never `null`, never a placeholder date, never `::` or a hash of zeros. An empty value is an answer about what we hold: we hold no value for that field. |
| Integers and booleans | Never empty. Booleans are `true` or `false`. |
| Addresses | IPv4 as a dotted quad, never as `::ffff:`-mapped IPv6. IPv6 in its standard compressed text form. |
| Hex | Lowercase. |
| Row order | Identical in the CSV and JSON file of the same dataset and day. Otherwise unspecified. |
| Empty files | A delta with no rows is still published: a CSV file with only the header row and an empty JSON file. |

## Basic: domains

One row per registrable domain, keyed on `domain`. Where the
 [domain lookup API](/developers/api) publishes the same fact, the column carries
 the API field's name.

| Column | Type | Meaning |
|---|---|---|
| `domain` | string | The registrable domain, resolved through the Public Suffix List and lowercased. |
| `tld` | string | The final label, with no leading dot. A `.co.uk` domain carries `uk`. |
| `registered` | timestamp | The earliest creation date in the registry (WHOIS and RDAP) data we hold for the domain. Empty where we hold none, which can happen while `registrar` and `expires` are filled. Never inferred from certificate dates or zone files. A domain that was deleted and registered again keeps the earliest creation date we hold. |
| `expires` | timestamp | The latest expiry date in the registry data we hold. |
| `registrar` | string | A sponsoring registrar named in the registry data we hold. After a transfer it may not be the current one. |
| `ip` | string | An address from the most recent new or changed A or AAAA answer our resolvers recorded for the domain or any name under it. One address, even where the answer holds several; which one is arbitrary, and it can change without the answer changing. Empty where our resolvers have recorded no address for the domain. |
| `first_cert_seen` | timestamp | The earliest time our Certificate Transparency collection recorded a certificate for the domain. It is not when the domain's first certificate was issued, so it can be later than the certificate's own date, and it is empty where every certificate we hold for the domain predates our record of it. |
| `first_zone_seen` | timestamp | When the domain first appeared as a new name in a registry zone file we hold. Our zone history begins in March 2025, so a domain already listed then and never deleted has none. It is an upper bound on the domain's first registration. For a domain that was deleted and registered again it can predate the current registration by months. |
| `last_update` | timestamp | The newest timestamp we hold for the domain from any source: a DNS answer, a certificate, a zone file or a registry record. For registry data it is when the record was fetched, which can predate our other collection. |

## Basic: DNS

One row per name and record type, merged across our vantage points, keyed on
 `name` and `record_type`.

**DNS does not expire on a clock.** We hold the current answer for as long as
 the name still resolves, however many years that runs, and we keep the record of what
 changed for 18 months. A name that stops answering entirely leaves the live data; the daily
 files that recorded it keep it permanently. There is no fixed window to design around.

| Column | Type | Meaning |
|---|---|---|
| `name` | string | The fully qualified name. |
| `record_type` | string | `A`, `AAAA`, `NS`, `MX`, `TXT` and the rest. |
| `values` | array of strings | Every value in the current answer at any vantage point, de-duplicated. The order carries no meaning. |
| `ttl` | integer | TTL in seconds, the largest across vantage points, as the resolver returned it when the answer last changed. It is not refreshed by re-resolutions that return the same answer, and a resolver's cached TTL can be lower than the authoritative one. |
| `last_confirmed` | timestamp | The most recent time any vantage point resolved this name and type and either got its current answer back unchanged or saw it change. |

Unchanged re-resolutions have been recorded since 2026-09-16. A name not
 resolved again since then carries the time its answer last changed; each name is picked up
 as the sweep reaches it.

## Pro: DNS by vantage point

One row per name, record type and vantage point, keyed on all three. A name can answer
 differently by geography, and the merged Basic file hides that.

| Column | Type | Meaning |
|---|---|---|
| `name` | string | The fully qualified name. |
| `record_type` | string | Record type. |
| `vantage` | string | An identifier for the resolver node that saw this answer, such as `fra`, `lax`, `nyc` or `dal`. Several nodes can share a region, and `nrd` is the lane that resolves newly registered domains. Treat it as an opaque label: nodes are added over time. |
| `values` | array of strings | The answer this vantage point holds, unmerged. |
| `ttl` | integer | TTL in seconds as this vantage point's resolver returned it when the answer last changed. Re-resolutions that return the same answer do not refresh it. |
| `last_seen` | timestamp | The most recent time this vantage point resolved the name and type and either got the same answer back or saw it change. |
| `last_changed` | timestamp | When the answer at this vantage point last changed. |
| `changes` | integer | How many times the answer has changed at this vantage point. A high count on an A record marks rotation; on NS, a migration. |

Before 2026-09-17, most heavily on 2026-08-27 and 2026-08-28, a fault on our
 side reset some rows without a real change: `last_changed` moved to the time of
 the check and `changes` went back to 0. The next real change corrects
 `last_changed`. It does not correct `changes`, which stays lower than
 the true count.

## Pro: certificates

One row per certificate, keyed on `cert_id`. [The Pro
 API](/developers/pro) returns the same fields for a single certificate.

**Certificates cover a rolling 14 days in the database, and every daily file is kept
 permanently.** A certificate we have not recorded for 14 days leaves the database,
 and `first_observed` and `ct_logs_seen` count only the records still
 held there. The daily certificates file for the day we recorded it keeps the full row
 regardless, so a date outside the window is a file to read rather than a gap in what we
 observed.

| Column | Type | Meaning |
|---|---|---|
| `cert_id` | string | Our certificate key, 32 lowercase hex characters. It is the same value as a domain record's `ssl_cert`, which the API writes in uppercase, so join on it case-insensitively. |
| `cert_sha256` | string | SHA-256 of the DER-encoded certificate. Empty where we have seen only the precertificate. |
| `serial` | string | Serial number, hex. |
| `spki_sha256` | string | SHA-256 of the SubjectPublicKeyInfo. Certificates sharing it were issued for the same key pair. Empty where we have not derived it. |
| `issuer` | string | Full issuer distinguished name, for example `CN=YE2,O=Let's Encrypt,C=US`. |
| `issuer_org` | string | Issuing CA organization on its own, for grouping. |
| `subject_cn` | string | Subject common name. Empty on certificates that carry names only in the SAN extension, and where we did not derive it, which happens only on certificates first recorded before 03:00 UTC on 2026-09-07. |
| `key_alg` | string | Public key algorithm, such as `RSA` or `ECDSA`. |
| `key_bits` | integer | Key size in bits. `0` where we did not derive it, which happens only on certificates first recorded before 03:00 UTC on 2026-09-07. |
| `signature_alg` | string | Signature algorithm, such as `SHA256-RSA`. |
| `sans` | array of strings | Every name the certificate covers. This is the column that links co-tenants behind one CDN certificate. |
| `san_count` | integer | Length of `sans`. |
| `wildcard` | boolean | True where any SAN begins with `*.`. |
| `ja4x` | string | JA4X fingerprint of the certificate's structure. Certificates minted by the same tooling share it. |
| `first_observed` | timestamp | When we first recorded the certificate from a CT log, within the 14 days we keep. |
| `last_observed` | timestamp | When we last recorded it. |
| `ct_logs_seen` | integer | How many CT logs we have seen it in, within the 14 days we keep. |

## Pro: address history

One row per domain and address, keyed on `domain` and `ip`. It
 covers addresses our resolvers saw for the domain and for any name under it, including the
 ones it has moved away from. An address known to us only from a third-party dataset is not
 included.

| Column | Type | Meaning |
|---|---|---|
| `domain` | string | The registrable domain. |
| `ip` | string | An address the domain, or a name under it, resolved to. |
| `first_seen` | timestamp | The first time this address appeared in an answer for the domain. For some addresses that also appear in a third-party hosting dataset we loaded, it is 2026-08-05, the date of that dataset, rather than our own first answer. |
| `last_seen` | timestamp | The last time this address appeared in a new or changed answer. Re-resolutions that return the same answer do not move it; the DNS files carry confirmation times. |
| `observed` | integer | How many new or changed answers included this address. For an address that also appears in the third-party hosting dataset, the count includes that sighting once. |
| `current` | boolean | True for the address in the domains file's `ip` column; which one is arbitrary when several were recorded together. When you apply a row with `current` true, clear it on the domain's other addresses; see [Applying a delta](#applying). |

## Sample rows

The same two domains rows in each format, as the generator writes them. The second domain
 has no registration data, so those cells are empty.

```
"domain","tld","registered","expires","registrar","ip","first_cert_seen","first_zone_seen","last_update"
"example-shop.com","com","2019-04-02T11:15:00Z","2027-04-02T11:15:00Z","GoDaddy.com, LLC","203.0.113.42","2026-09-06T06:22:11Z","","2026-09-16T04:11:50Z"
"newly-parked.xyz","xyz","","","","","2026-09-16T01:19:30Z","2026-09-15T22:07:48Z","2026-09-16T04:12:44Z"
```

```
{"domain":"example-shop.com","tld":"com","registered":"2019-04-02T11:15:00Z","expires":"2027-04-02T11:15:00Z","registrar":"GoDaddy.com, LLC","ip":"203.0.113.42","first_cert_seen":"2026-09-06T06:22:11Z","first_zone_seen":"","last_update":"2026-09-16T04:11:50Z"}
{"domain":"newly-parked.xyz","tld":"xyz","registered":"","expires":"","registrar":"","ip":"","first_cert_seen":"2026-09-16T01:19:30Z","first_zone_seen":"2026-09-15T22:07:48Z","last_update":"2026-09-16T04:12:44Z"}
```

A DNS-by-vantage row with an array value. In the CSV file the array is JSON text inside
 a quoted cell, so its quotes are doubled.

```
"name","record_type","vantage","values","ttl","last_seen","last_changed","changes"
"example-shop.com","TXT","fra","[""v=spf1 ip4:203.0.113.0\/24 -all""]",300,"2026-09-16T05:02:11Z","2026-08-30T17:40:02Z",1
```

```
{"name":"example-shop.com","record_type":"TXT","vantage":"fra","values":["v=spf1 ip4:203.0.113.0\/24 -all"],"ttl":300,"last_seen":"2026-09-16T05:02:11Z","last_changed":"2026-08-30T17:40:02Z","changes":1}
```

## Where to go next

- [Data feeds and downloads](/developers/feeds): fetching files with your API
 key, the download allowance, a sync script, and the live newly registered domains
 feed.

- [Public API](/developers/api): the domain lookup whose field names the files
 reuse.

- [Pro API](/developers/pro): certificate, SPKI and reverse IP pivots.
- [Pricing](/pricing): which datasets each plan includes.

Last updated September 17, 2026. Questions the docs do not answer:
 [get in touch](/contact).

↑ Top

---

issued.live, operated by Tuxxin LLC. Source: https://issued.live/developers/files