API documentation

How the API works

The API serves whole files, not a live query endpoint. You download a full dataset or a single slice, then slice and filter it on your own machine. That keeps downloads fast and puts no cap on how you query.

1. Create a key

Generate a Bearer API key in your account, optionally locked to an IP allowlist. Send it as an Authorization header.

2. Download files

Fetch the full export, or one slice by TLD, TLD suffix, country or technology. Resume interrupted downloads with a Range request.

3. Filter locally

Decompress with zstd and query with jq, a warehouse or your own code. One dimension per slice, intersect the rest yourself.

Authentication

Most endpoints require a Bearer API key. The public catalogue, preview and sample endpoints need no auth.

Create and manage keys in your account. The full key is shown once at creation, so store it safely.
Send it as Authorization: Bearer wbtl_… on every authenticated request.
Optionally restrict a key to an IP allowlist. Requests from other addresses get 401.
Keys are rate limited per key. Rotate or revoke a key at any time.

GEThttps://webatla.com/api/v1/me Authorization: Bearer wbtl_•••••••f3a9 # 200 OK { "user": { "id": "…", "email": "you@example.com", "role": "customer" }, "apiKeyId": "…", "scopes": ["me:read"] }

Endpoints

The full, machine-readable contract lives at /api/v1/openapi.json (OpenAPI 3.1).

Method	Endpoint	Auth	Rate limit	Purpose
GET	`/api/v1/datasets`	Public	60/min	List published datasets with price and row counts
GET	`/api/v1/preview/{scope}/{key}`	Public	60/min	Free preview rows for a scope
GET	`/api/v1/sample/{scope}/{key}`	Public	60/min	Free sample export for a scope
GET	`/api/v1/me`	Bearer	200/min	The current key's user and scopes
GET	`/api/v1/me/orders`	Bearer	200/min	Your orders and their status
GET	`/api/v1/me/downloads`	Bearer	200/min	Your download history (`?limit=` up to 500)
GET	`/api/download/{slug}`	Bearer	240/min	Full dataset export as `.jsonl.zst`
GET	`/api/download/{slug}/{scope}/{key}`	Bearer	240/min	One per-scope slice of a dataset

scope is one of tld, tld_suffix, country or technology. Download endpoints require paid, unexpired access to that dataset.

Quickstart

Every download is one file. Pull the whole dataset, or narrow to a single dimension first.

Download the full export

GEThttps://webatla.com/api/download/all-data Authorization: Bearer wbtl_•••••••f3a9 # Streams the whole dataset as one .jsonl.zst file

Download a single slice

GEThttps://webatla.com/api/download/all-data/tld/com https://webatla.com/api/download/all-data/tld_suffix/com.ua https://webatla.com/api/download/websites-ranking/country/DE https://webatla.com/api/download/technologies/technology/wordpress Authorization: Bearer wbtl_•••••••f3a9 # Exactly one dimension per file. Intersect the rest locally.

curl, resume and local filter

curl -OJ -H "Authorization: Bearer wbtl_•••••••f3a9" \ https://webatla.com/api/download/technologies/technology/wordpress # Resume an interrupted download with -C - curl -C - -OJ -H "Authorization: Bearer wbtl_•••••••f3a9" \ https://webatla.com/api/download/all-data # Decompress and filter on your machine zstdcat webatla-technologies-WordPress-*.jsonl.zst \ | jq -c 'select(.["Country by IP"] == "DE")' > wordpress-de.jsonl

List past downloads with GET https://webatla.com/api/v1/me/downloads. Try a free preview with no key: GET https://webatla.com/api/v1/preview/tld/com?dataset=all-data.

Data format

Files are JSON Lines (one JSON object per line), zstd-compressed. The schema is shared across datasets, so signals line up on the same domain key.

Each line is one domain. Decompress with zstd -d or stream with zstdcat.
Column names can contain spaces, so read them with bracket keys in jq, for example .["Country by IP"].
Fields may be missing or null. Default them, for example .Registrar // "".
Dates are ISO 8601 strings, for example "2026-11-18T04:47:13.573" or "2026-04-22".

Column reference

Column	Type	Example	Datasets
Domain	string	"arriva.com.hr"	d1–d7
TLD suffix	string	"com.hr"	d1–d7
TLD	string	"hr"	d1–d7
Domain type	string: "ICANN" \| "PRIVATE"	"ICANN"	d1–d4, d6, d7
HTTP status	integer \| null	200	d3, d6, d7
IP address	string \| null	"31.13.236.27"	d3, d6, d7
Country by IP	string (ISO-2) \| null	"DE"	d2, d3, d6, d7
Social networks	object \| null	{"facebook": ["https://…"]}	d3, d6, d7
Technologies last data checked	string (date) \| null	"2026-04-22"	d3, d6, d7
Technologies	array \| object \| null	["WordPress"] / {"PHP": "4.4.9"}	d3, d6, d7
DNS last data checked	string (date)	"2026-06-03"	d4, d6, d7
DNS Status	string: "TRUE" \| "FALSE"	"TRUE"	d4, d6, d7
DNS records	object \| null	{"A": ["31.13.236.27"], "MX": [{"host": "…", "priority": 0}]}	d4, d6, d7
PR value	number	4.808574e-9 / 0	d2, d3, d6, d7
Harmonic value	number	10305265 / 0	d2, d3, d6, d7
RDAP/WHOIS last date checked	string (date)	"2026-01-04"	d5, d7, d6*
RDAP/WHOIS method	string: "rdap" \| "whois"	"whois"	d5, d7, d6*
Domain creation date	string (ISO 8601) \| null	"2024-12-12T14:04:04"	d2, d3, d5, d6, d7
Domain expiration date	string (ISO 8601) \| null	"2026-12-12T14:04:04"	d2, d3, d5, d6, d7
Domain last changed	string (ISO 8601) \| null	"2026-01-04T04:43:01"	d5, d7, d6*
Registrar	string \| null	"REGRU-RU"	d5, d6, d7
RDAP/WHOIS Record	object \| null	{"raw_response": "% TCI Whois…"}	d5, d6*
Response Status RDAP/WHOIS	string	"Domain is active"	d5, d6*

Datasets: d1 All active domains · d2 Websites + Ranking · d3 Technologies · d4 DNS · d5 RDAP & WHOIS · d6 All data · d7 Domain Investor.
* In All data (d6) the RDAP/WHOIS fields are present only for the domains that carry a registration record.

Nested field shapes

Technologies is either an array of names or an object of name to version.
DNS Status is the string "TRUE" or "FALSE", not a boolean.
DNS records maps a record type to its values: A, AAAA, NS, SOA, TXT, MX, CNAME, CAA, HTTPS, DNSKEY, DS, SRV, PTR…. MX is [{"host", "priority"}].
Social networks maps a platform to profile links: facebook, instagram, x-twitter, linkedin, youtube, whatsapp, telegram, tiktok, github…. These are the site's own public links, not a person's contact.
RDAP/WHOIS Record holds the raw registration record, redacted of personal fields per ICANN rules.

Recipes

Verified against real export files. Everything runs locally after you download a file.

Count, extract and export to CSV

# Count rows in a slice zstdcat webatla-all-data-com-*.jsonl.zst | wc -l # One domain per line zstdcat webatla-all-data-com-*.jsonl.zst | jq -r '.Domain' > domains.txt # Selected columns to CSV zstdcat webatla-all-data-com-*.jsonl.zst \ | jq -r '[.Domain, .["Country by IP"] // "", .Registrar // ""] | @csv' > out.csv

Filter by technology

# Technologies is an array OR an object, handle both zstdcat webatla-all-data-com-*.jsonl.zst \ | jq -c 'select(.Technologies | if type=="array" then index("WordPress") elif type=="object" then has("WordPress") else false end)' > wordpress.jsonl

DNS and expiry

# Parked or dead domains zstdcat webatla-dns-com-*.jsonl.zst \ | jq -r 'select(.["DNS Status"] == "FALSE") | .Domain' > parked.txt # Unique A records zstdcat webatla-dns-com-*.jsonl.zst \ | jq -r '.["DNS records"].A[]?' | sort -u > ips.txt # Domains expiring in June 2026 zstdcat webatla-all-data-com-*.jsonl.zst \ | jq -r 'select((.["Domain expiration date"] // "") | startswith("2026-06")) | .Domain'

Python

# Decompress first zstd -d webatla-all-data-com-*.jsonl.zst # Then stream line by line import json with open("webatla-all-data-com-2026-06-09.jsonl", encoding="utf-8") as f: for line in f: row = json.loads(line) t = row.get("Technologies") names = t if isinstance(t, list) else list(t) if isinstance(t, dict) else [] if "WordPress" in names and row.get("Country by IP") == "DE": print(row["Domain"])

Response codes and limits

Errors return a JSON body of the shape {"error": "…"}. Rate-limit headers are returned on every request.

Code	Meaning
200 / 206	Success. 206 is a partial (resumed) download.
302	A browser session is missing on a download route. API keys get 401 instead.
401	Missing, invalid or IP-blocked API key.
403	No active paid access to this dataset.
404	Unknown dataset or entity, or the file is not uploaded yet.
416	Malformed or multipart Range request.
429	Rate limit exceeded. Back off and retry.

Public endpoints are limited per IP. Authenticated endpoints are limited per key or per user, as listed above.

Start building

Create a key in your account, or browse the datasets and pull a free sample first. Questions about a custom slice? Email support@webatla.com.

Get an API key View datasets

The webatla API