DNSSEC misconfiguration takes .fj zones offline

Around midnight UTC on March 8, hostnames under Fiji's country-code top-level domain (.fj) began failing for many users. Reports on Hacker News and the local fijivillage news site described widespread outages affecting websites and apps, including Vodafone's M-PAiSA financial service, which could not complete transactions.

Traffic data from Cloudflare customer zones in the .com.fj second-level domain shows HTTP traffic dropping by roughly 40% almost immediately at the start of the incident, with continued declines through the morning. Query error volumes from Cloudflare's 1.1.1.1 resolver for .com.fj hostnames also climbed sharply after midnight, pointing to bad answers from .fj servers. Radar data shows no connectivity disruption in Fiji during the same window, ruling out a network-level outage.

The symptoms pointed to a DNS-level failure, and the likely culprit was DNSSEC. Resolution attempts returned the extended DNS error message:

EDE: 9 (DNSKEY Missing): 'no SEP matching the DS found for fj.'

Extended DNS Error 9 is defined as "A DS record existed at a parent, but no supported matching DNSKEY record could be found for the child." In DNSSEC, the DS record at the parent zone contains a hash of the child zone's KSK, allowing resolvers to verify the child's authenticity. When the DS record and the zone's actual signing key do not match, validation fails.

The evidence indicates that around midnight UTC, the .fj zone began signing with a key not present in the root zone's DS record. This likely resulted from a scheduled key rollover that proceeded without first confirming that IANA had updated the DS record in the root zone, which is the required sequence for TLD key changes.

The DNSSEC failure also explains the mixed reports from users. Resolvers performing strict DNSSEC validation returned errors that broke browsing and applications, while resolvers with looser validation policies continued to resolve .fj hostnames without issue.

Cloudflare resolver metrics indicate the problem was resolved around 1400 UTC when the DS record was updated. A DNSSEC misconfiguration at a single domain is disruptive, but when it occurs at the ccTLD level, the impact extends to every zone beneath it. Such incidents are unfortunately not rare; a public log of DNSSEC outages lists multiple past examples of similar failures.