Yemen .ye Domain Observatory

Technical evidence and archival snapshot · 2026-08-25

Technical summary

The rebuilt corpus retains 1,476 registrable domains and 4,708 hostnames across 23,975 normalized observations. It recovers 62 domains beyond the normalized former curated list. Current checks found DNS records for 90.2% of the historical corpus but a root HTTP response for only 16.4%, demonstrating why present-day reachability must not define historical membership. A bounded, robots-aware pass produced 219 validated homepage WARCs.

Source: Recovery delta and observability stagesTables: data/domains.csv, data/observations.csv, data/snapshots/2026-08-25/dns.csv, data/snapshots/2026-08-25/http.csv, archives/manifests/2026-08-25-homepages.csv

Executed the former-list delta and same-denominator observability stages.

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

Registrable domains

1,476

Historical names retained regardless of present-day state.

Observed hostnames 4,708

Historical names retained regardless of present-day state.Source: Historical corpus summary and suffix distributionTables: data/domains.csv, data/hostnames.csv

Executed summary and public-suffix counts over the reviewed corpus.

SELECT COUNT(*) AS domains FROM domains; SELECT COUNT(*) AS hostnames FROM hostnames; SELECT public_suffix, COUNT(*) AS domains FROM domains GROUP BY public_suffix ORDER BY domains DESC

Observations

23,975

Normalized, source-linked evidence in the public corpus.

Source files/exports 48

Normalized, source-linked evidence in the public corpus.Source: Observation and source-family coverageTables: data/observations.csv, data/source_inventory.csv, analysis/source_family_coverage.csv

Executed evidence-scale and source-family coverage queries.

SELECT COUNT(*) AS observations FROM observations; SELECT COUNT(*) AS source_files FROM source_inventory; SELECT source_family, CAST(domains AS INTEGER) AS domains, CAST(share_of_corpus AS REAL) AS share FROM source_family_coverage ORDER BY domains DESC

Additional domains

Names recovered beyond the normalized former curated list.

Names recovered beyond the normalized former curated list.Source: Recovery delta and observability stagesTables: data/domains.csv, data/observations.csv, data/snapshots/2026-08-25/dns.csv, data/snapshots/2026-08-25/http.csv, archives/manifests/2026-08-25-homepages.csv

Executed the former-list delta and same-denominator observability stages.

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

WARCs captured

219

Validated one-response homepage WARCs.

Robots exclusions 9

Validated one-response homepage WARCs.Source: 2026-08-25 homepage WARC outcomesTable: archives/manifests/2026-08-25-homepages.csv

Executed archive-outcome counts over the validated public manifest.

SELECT status, COUNT(*) AS domains FROM warc_manifest GROUP BY status

Key findings

Structured suffixes dominate the recovered namespace

Most registrable domains are under com.ye, gov.ye, edu.ye, org.ye, or net.ye; 193 are directly under .ye. The corpus also preserves names even when current DNS or web checks fail.

Source: Historical corpus summary and suffix distributionTables: data/domains.csv, data/hostnames.csv

Executed summary and public-suffix counts over the reviewed corpus.

SELECT COUNT(*) AS domains FROM domains; SELECT COUNT(*) AS hostnames FROM hostnames; SELECT public_suffix, COUNT(*) AS domains FROM domains GROUP BY public_suffix ORDER BY domains DESC

Most retained domains sit under structured .ye suffixes

Source: Historical corpus summary and suffix distributionTables: data/domains.csv, data/hostnames.csv

Executed summary and public-suffix counts over the reviewed corpus.

SELECT COUNT(*) AS domains FROM domains; SELECT COUNT(*) AS hostnames FROM hostnames; SELECT public_suffix, COUNT(*) AS domains FROM domains GROUP BY public_suffix ORDER BY domains DESC

Most retained domains sit under structured .ye suffixes data
Public suffix	Domains
com.ye	772
gov.ye	233
ye	193
edu.ye	131
org.ye	86
net.ye	61

Historical presence and current web availability answer different questions

DNS responses remain common, while root web responses are comparatively rare. Conflating those measures would erase dormant, redirected, mail-only, or otherwise historically documented domains.

Executed the former-list delta and same-denominator observability stages.

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

Present-day web observability is much smaller than historical membership

Executed the former-list delta and same-denominator observability stages.

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

Present-day web observability is much smaller than historical membership data
Evidence stage	Share of historical corpus
Historical corpus	100%
DNS records present	90.2%
HTTP response	16.4%
WARC captured	14.8%

Archival exceptions are visible rather than silently dropped

Of 242 responsive roots, 219 were captured. Nine were excluded by robots policy and 14 remained connection timeouts after a resumable retry. Every captured WARC passed file and payload hashes, record-count checks, target-URL checks, and HTTP representation validation.

Source: 2026-08-25 homepage WARC outcomesTable: archives/manifests/2026-08-25-homepages.csv

Executed archive-outcome counts over the validated public manifest.

SELECT status, COUNT(*) AS domains FROM warc_manifest GROUP BY status

The bounded archive captured 219 of 242 responsive roots

Source: 2026-08-25 homepage WARC outcomesTable: archives/manifests/2026-08-25-homepages.csv

Executed archive-outcome counts over the validated public manifest.

SELECT status, COUNT(*) AS domains FROM warc_manifest GROUP BY status

The bounded archive captured 219 of 242 responsive roots data
Outcome	Domains
Captured	219
Robots exclusion	9
Retryable timeout	14

Scope, data, and metric definitions

A hostname is a normalized observed DNS-style name. A registrable domain is the longest applicable .ye public suffix plus one label. An observation is a unique domain/hostname/source/time/evidence/wildcard combination. DNS records present means at least one requested A, AAAA, NS, MX, CNAME, or SOA answer during the dated pass. HTTP response means a no-redirect root request returned any status code. None of these current metrics asserts registration status or ownership.

Source: Observation and source-family coverageTables: data/observations.csv, data/source_inventory.csv, analysis/source_family_coverage.csv

Executed evidence-scale and source-family coverage queries.

SELECT COUNT(*) AS observations FROM observations; SELECT COUNT(*) AS source_files FROM source_inventory; SELECT source_family, CAST(domains AS INTEGER) AS domains, CAST(share_of_corpus AS REAL) AS share FROM source_family_coverage ORDER BY domains DESC

Source-family coverage

Source: Observation and source-family coverageTables: data/observations.csv, data/source_inventory.csv, analysis/source_family_coverage.csv

Executed evidence-scale and source-family coverage queries.

SELECT COUNT(*) AS observations FROM observations; SELECT COUNT(*) AS source_files FROM source_inventory; SELECT source_family, CAST(domains AS INTEGER) AS domains, CAST(share_of_corpus AS REAL) AS share FROM source_family_coverage ORDER BY domains DESC

Source-family coverage
Source family	Domains	Corpus coverage
mixed_export	1,437	97.4%
compiled	1,414	95.8%
legacy_compiled	1,414	95.8%
legacy_whois_lookup	1,414	95.8%
legacy_rdap_lookup	1,366	92.5%
securitytrails	1,294	87.7%
networksdb	701	47.5%
certificate_transparency	128	8.7%
serpapi	72	4.9%
urlscan	28	1.9%

Methodology

Inputs were parsed by source type, normalized through IDNA, matched against the 2026-08-25 PSL rules, and deduplicated only at the complete observation-key level. Embedded scan/search times are retained; otherwise preserved file modification time is explicitly labeled as the date basis. Raw WHOIS/RDAP and proprietary exports are withheld, while their non-personal domain-name observations and input hashes remain auditable.

Source: Hashed source inventoryTable: data/source_inventory.csv

Executed source-family observation totals from the hashed inventory.

SELECT source_family, SUM(CAST(accepted_observations AS INTEGER)) AS accepted_observations FROM source_inventory GROUP BY source_family

Data-quality assessment

Executed the former-list delta and same-denominator observability stages.

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

Data-quality assessment
Dimension	Result	Evidence	Limitation
Historical inclusion	Pass	1,476 unique registrable domains, all with provenance	Complete for supplied inputs, not the registry zone
Source traceability	Pass	23,975 observations across 48 files/exports	Several lists are derived from overlapping upstream sources
Current DNS coverage	Dated	1,332 domains returned requested DNS records	DNS state is not registration status
Current web coverage	Dated	242 root HTTP responses without redirects	Timeouts and redirects were not pursued
Archive integrity	Pass with exceptions	219 WARCs passed hashes and record validation	9 robots exclusions; 14 retryable timeouts

Limitations and robustness

The corpus is complete for the supplied evidence after deterministic validation, not for the full registry history. Evidence families overlap and are biased toward publicly visible certificates, DNS, search, and web scans. Observation dates are acquisition evidence, not registration dates. DNS and HTTP timeouts are ambiguous. WARC coverage is a one-response homepage snapshot, not a whole-site crawl. These caveats are structural and should travel with any downstream analysis.

Executed the former-list delta and same-denominator observability stages.

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

Next steps

Run periodic DNS/HTTP snapshots without modifying the historical core.
Resume the 14 archival timeouts conservatively.
Query prior Common Crawl indexes to recover older captures without recrawling sites.
Add new evidence as provenance-linked observations, never as unsourced list edits.
Publish correction records instead of erasing disputed historical observations.

Source: Common Crawl data access

Further research questions

Which public institutions moved between structured suffixes or disappeared from current DNS?
How do hosting, certificate, and nameserver changes align with administrative events?
Which dormant domains have older public web-archive coverage?
Can official registry statistics or zone access quantify the remaining coverage gap?
What operational changes would improve transparency and continuity for the .ye namespace?

Source: IANA .ye delegation record

Sources

Historical corpus summary and suffix distributiondata/domains.csv · sqlite · 2026-08-25T13:29:13Z

Executed summary and public-suffix counts over the reviewed corpus.

SQL query

SELECT COUNT(*) AS domains FROM domains; SELECT COUNT(*) AS hostnames FROM hostnames; SELECT public_suffix, COUNT(*) AS domains FROM domains GROUP BY public_suffix ORDER BY domains DESC

Observation and source-family coveragedata/observations.csv · sqlite · 2026-08-25T13:29:13Z

Executed evidence-scale and source-family coverage queries.

SQL query

SELECT COUNT(*) AS observations FROM observations; SELECT COUNT(*) AS source_files FROM source_inventory; SELECT source_family, CAST(domains AS INTEGER) AS domains, CAST(share_of_corpus AS REAL) AS share FROM source_family_coverage ORDER BY domains DESC

Hashed source inventorydata/source_inventory.csv · sqlite · 2026-08-25T13:29:13Z
Executed source-family observation totals from the hashed inventory.
SQL query
```
SELECT source_family, SUM(CAST(accepted_observations AS INTEGER)) AS accepted_observations FROM source_inventory GROUP BY source_family
```

Recovery delta and observability stagesanalysis/research_summary.json · sqlite · 2026-08-25T13:29:13Z

Executed the former-list delta and same-denominator observability stages.

SQL query

SELECT COUNT(*) AS additional_domains FROM domains d WHERE NOT EXISTS (SELECT 1 FROM observations o WHERE o.source_id = 'curated_unique' AND o.domain = d.domain); SELECT 'Historical corpus' AS stage, COUNT(*) AS domains FROM domains UNION ALL SELECT 'DNS records present', COUNT(*) FROM dns_snapshot WHERE status = 'records_present' UNION ALL SELECT 'HTTP response', COUNT(*) FROM http_snapshot WHERE status = 'http_response' UNION ALL SELECT 'WARC captured', COUNT(*) FROM warc_manifest WHERE status = 'captured'

2026-08-25 DNS status countsdata/snapshots/2026-08-25/dns.csv · sqlite · 2026-08-25T13:29:13Z
Executed DNS status aggregation.
SQL query
```
SELECT status, COUNT(*) AS domains FROM dns_snapshot GROUP BY status
```
2026-08-25 HTTP status countsdata/snapshots/2026-08-25/http.csv · sqlite · 2026-08-25T13:29:13Z
Executed HTTP status aggregation.
SQL query
```
SELECT status, COUNT(*) AS domains FROM http_snapshot GROUP BY status
```
2026-08-25 homepage WARC outcomesarchives/manifests/2026-08-25-homepages.csv · sqlite · 2026-08-25T13:29:13Z
Executed archive-outcome counts over the validated public manifest.
SQL query
```
SELECT status, COUNT(*) AS domains FROM warc_manifest GROUP BY status
```
IANA .ye delegation record
Public Suffix List
Common Crawl data access