---
title: "API documentation"
description: "webatla API reference: Bearer authentication, dataset and download endpoints, JSONL and zstd file format, rate limits and worked examples."
url: https://webatla.com/api-docs
---

# The webatla API

Pull the domain database straight into your own stack. Authenticate with a Bearer key, download a whole dataset or one slice as compressed JSONL, and filter it locally. No per-row metering.

[Get an API key](https://webatla.com/account/api)
[OpenAPI spec](https://webatla.com/api/v1/openapi.json)

REST, over HTTPS, JSON responses

Bearer, token auth, IP allowlist

JSONL, zstd-compressed files

OpenAPI, 3.1 machine spec

## How the API works

The API serves whole files, not a live query endpoint. You download a full dataset or a single slice, then slice and filter it on your own machine. That keeps downloads fast and puts no cap on how you query.

#### 1. Create a key

Generate a Bearer API key in your account, optionally locked to an IP allowlist. Send it as an Authorization header.

#### 2. Download files

Fetch the full export, or one slice by TLD, TLD suffix, country or technology. Resume interrupted downloads with a Range request.

#### 3. Filter locally

Decompress with zstd and query with jq, a warehouse or your own code. One dimension per slice, intersect the rest yourself.

## Authentication

Most endpoints require a Bearer API key. The dataset catalogue is open to everyone. Preview and sample downloads need a free account or a key.

- Create and manage keys in [your account](https://webatla.com/account/api). The full key is shown once at creation, so store it safely.

- Send it as `Authorization: Bearer wbtl_…` on every authenticated request.

- Optionally restrict a key to an IP allowlist. Requests from other addresses get `401`.

- Keys are rate limited per key. Rotate or revoke a key at any time.

```
GET https://webatla.com/api/v1/me
  Authorization: Bearer wbtl_•••••••f3a9

# 200 OK
{
  "user": { "id": "…", "email": "you@example.com", "role": "customer" },
  "apiKeyId": "…",
  "scopes": ["me:read"]
}
```

## Endpoints

The full, machine-readable contract lives at [`/api/v1/openapi.json`](https://webatla.com/api/v1/openapi.json) (OpenAPI 3.1).

| Method | Endpoint | Auth | Rate limit | Purpose |
| --- | --- | --- | --- | --- |
| GET | `/api/v1/datasets` | Public | 60/min | List published datasets with price and row counts |
| GET | `/api/v1/preview?dataset={slug}` | Account or API key | 30/min | Free 10,000-row sample of a whole dataset |
| GET | `/api/v1/sample/{scope}/{key}?dataset={slug}` | Account or API key | 30/min | Free 20-row sample for one entity (TLD, suffix, country, technology) |
| GET | `/api/v1/me` | Bearer | 200/min | The current key's user and scopes |
| GET | `/api/v1/me/orders` | Bearer | 200/min | Your orders and their status |
| GET | `/api/v1/me/downloads` | Bearer | 200/min | Your download history (`?limit=` up to 500) |
| GET | `/api/download/{slug}` | Bearer | 240/min | Full dataset export as `.jsonl.zst` |
| GET | `/api/download/{slug}/{scope}/{key}` | Bearer | 240/min | One per-scope slice of a dataset |

`scope` is one of `tld`, `tld_suffix`, `country` or `technology`. Download endpoints require paid, unexpired access to that dataset.

## Quickstart

Every download is one file. Pull the whole dataset, or narrow to a single dimension first.

#### Download the full export

```
GET https://webatla.com/api/download/all-data
  Authorization: Bearer wbtl_•••••••f3a9

# Streams the whole dataset as one .jsonl.zst file
```

#### Download a single slice

```
GET https://webatla.com/api/download/all-data/tld/com
https://webatla.com/api/download/all-data/tld_suffix/com.ua
https://webatla.com/api/download/websites-ranking/country/DE
https://webatla.com/api/download/technologies/technology/wordpress
  Authorization: Bearer wbtl_•••••••f3a9

# Exactly one dimension per file. Intersect the rest locally.
```

#### curl, resume and local filter

```
curl -f -OJ -H "Authorization: Bearer wbtl_•••••••f3a9" \
  https://webatla.com/api/download/technologies/technology/wordpress

# Resume an interrupted download with -C -
curl -f -C - -o all-data.jsonl.zst -H "Authorization: Bearer wbtl_•••••••f3a9" \
  https://webatla.com/api/download/all-data

# Large files: aria2c with 3 parallel connections is faster and resumes on its own
aria2c -x 3 -s 3 -c -o all-data.jsonl.zst \
  --header="Authorization: Bearer wbtl_•••••••f3a9" \
  https://webatla.com/api/download/all-data

# Decompress and filter on your machine
zstdcat webatla-technologies-WordPress-*.jsonl.zst \
  | jq -c 'select(.["Country by IP"] == "DE")' > wordpress-de.jsonl
```

List past downloads with `GET https://webatla.com/api/v1/me/downloads`. Preview any dataset with your key: `GET https://webatla.com/api/v1/preview?dataset=all-data`.

## Data format

Files are JSON Lines (one JSON object per line), zstd-compressed. The schema is shared across datasets, so signals line up on the same domain key.

- Each line is one domain. Decompress with `zstd -d` or stream with `zstdcat`.

- Column names can contain spaces, so read them with bracket keys in jq, for example `.["Country by IP"]`.

- Fields may be missing or null. Default them, for example `.Registrar // ""`.

- Dates are ISO 8601 strings, for example `"2026-11-18T04:47:13.573"` or `"2026-04-22"`.

#### Column reference

| Column | Type | Example | Datasets |
| --- | --- | --- | --- |
| Domain | string | "arriva.com.hr" | d1–d7 |
| TLD suffix | string | "com.hr" | d1–d7 |
| TLD | string | "hr" | d1–d7 |
| Domain type | string: "ICANN" \| "PRIVATE" | "ICANN" | d1–d4, d6, d7 |
| HTTP status | integer \| null | 200 | d3, d6, d7 |
| IP address | string \| null | "31.13.236.27" | d3, d6, d7 |
| Country by IP | string (ISO-2) \| null | "DE" | d2, d3, d6, d7 |
| Social networks | object \| null | {"facebook": ["https://…"]} | d3, d6, d7 |
| Technologies last data checked | string (date) \| null | "2026-04-22" | d3, d6, d7 |
| Technologies | array \| object \| null | ["WordPress"] / {"PHP": "4.4.9"} | d3, d6, d7 |
| DNS last data checked | string (date) | "2026-06-03" | d4, d6, d7 |
| DNS Status | string: "TRUE" \| "FALSE" | "TRUE" | d4, d6, d7 |
| DNS records | object \| null | {"A": ["31.13.236.27"], "MX": [{"host": "…", "priority": 0}]} | d4, d6, d7 |
| PR value | number | 4.808574e-9 / 0 | d2, d3, d6, d7 |
| Harmonic value | number | 10305265 / 0 | d2, d3, d6, d7 |
| RDAP/WHOIS last date checked | string (date) | "2026-01-04" | d5, d7, d6* |
| RDAP/WHOIS method | string: "rdap" \| "whois" | "whois" | d5, d7, d6* |
| Domain creation date | string (ISO 8601) \| null | "2024-12-12T14:04:04" | d2, d3, d5, d6, d7 |
| Domain expiration date | string (ISO 8601) \| null | "2026-12-12T14:04:04" | d2, d3, d5, d6, d7 |
| Domain last changed | string (ISO 8601) \| null | "2026-01-04T04:43:01" | d5, d7, d6* |
| Registrar | string \| null | "REGRU-RU" | d5, d6, d7 |
| RDAP/WHOIS Record | object \| null | {"raw_response": "% TCI Whois…"} | d5, d6* |
| Response Status RDAP/WHOIS | string | "Domain is active" | d5, d6* |

**Datasets:** d1 All active domains · d2 Websites + Ranking · d3 Technologies · d4 DNS · d5 RDAP & WHOIS · d6 All data · d7 Domain Investor.

* In All data (d6) the RDAP/WHOIS fields are present only for the domains that carry a registration record.

#### Nested field shapes

- `Technologies` is either an array of names or an object of name to version.

- `DNS Status` is the string `"TRUE"` or `"FALSE"`, not a boolean. Select live domains by "TRUE", not by a non-empty DNS records object, because a FALSE row can carry its last known records.

- `DNS records` is an object keyed by record type. Each type holds a list, except SOA, which is one object. SRV entries name the service. As domains are rechecked, rows also gain the derived fields DKIM, DNSSEC, CDN, ASN, WWW (the records of the www host), MX_TLSA and NULL_MX, which can be strings, booleans or objects. PARTIAL lists the record types that did not answer at the last check. Their values, if any, come from an earlier check. The record types are `A, AAAA, CAA, CNAME, DNSKEY, DS, HTTPS, MX, NS, SOA, SRV, SSHFP, TLSA, TXT`. `MX` is `[{"host", "priority"}]`.

- `Social networks` maps a platform to profile links: `facebook, instagram, x-twitter, linkedin, youtube, whatsapp, telegram, tiktok, github…`. These are the site's own public links, not a person's contact.

- `RDAP/WHOIS Record` holds the raw registration record, redacted of personal fields per ICANN rules.

## Recipes

Verified against real export files. Everything runs locally after you download a file.

#### Count, extract and export to CSV

```
# Count rows in a slice
zstdcat webatla-all-data-com-*.jsonl.zst | wc -l

# One domain per line
zstdcat webatla-all-data-com-*.jsonl.zst | jq -r '.Domain' > domains.txt

# Selected columns to CSV
zstdcat webatla-all-data-com-*.jsonl.zst \
  | jq -r '[.Domain, .["Country by IP"] // "", .Registrar // ""] | @csv' > out.csv
```

#### Filter by technology

```
# Technologies is an array OR an object, handle both
zstdcat webatla-all-data-com-*.jsonl.zst \
  | jq -c 'select(.Technologies
      | if type=="array" then index("WordPress")
        elif type=="object" then has("WordPress")
        else false end)' > wordpress.jsonl
```

#### DNS and expiry

```
# Parked or dead domains
zstdcat webatla-dns-com-*.jsonl.zst \
  | jq -r 'select(.["DNS Status"] == "FALSE") | .Domain' > parked.txt

# Unique A records
zstdcat webatla-dns-com-*.jsonl.zst \
  | jq -r '.["DNS records"].A[]?' | sort -u > ips.txt

# Domains expiring in June 2026
zstdcat webatla-all-data-com-*.jsonl.zst \
  | jq -r 'select((.["Domain expiration date"] // "") | startswith("2026-06")) | .Domain'
```

#### Python

```
# Decompress first
zstd -d webatla-all-data-com-*.jsonl.zst

# Then stream line by line
import json

with open("webatla-all-data-com-2026-06-09.jsonl", encoding="utf-8") as f:
    for line in f:
        row = json.loads(line)
        t = row.get("Technologies")
        names = t if isinstance(t, list) else list(t) if isinstance(t, dict) else []
        if "WordPress" in names and row.get("Country by IP") == "DE":
            print(row["Domain"])
```

## Response codes and limits

Errors return a JSON body of the shape `{"error": "…"}`. Rate-limit headers are returned on every request.

| Code | Meaning |
| --- | --- |
| 200 / 206 | Success. 206 is a partial (resumed) download. |
| 302 | A browser session is missing on a download route. API keys get 401 instead. |
| 401 | Missing, invalid or IP-blocked API key. |
| 403 | No active paid access to this dataset. |
| 404 | Unknown dataset or entity, or the file is not uploaded yet. |
| 416 | Malformed or multipart Range request. |
| 429 | Rate limit exceeded. Back off and retry. |

Public endpoints are limited per IP. Authenticated endpoints are limited per key or per user, as listed above.

## Start building

Create a key in your account, or browse the datasets and pull a sample with your free account. Questions about a custom slice? Email support@webatla.com.

[Get an API key](https://webatla.com/account/api)
[View datasets](https://webatla.com/pricing)
