Introduction

KF Webs recently got access to the gTLD zone-files, of which we decided to have a closer look on the COM zone-file.KF Webs performs domain name services through the interface at passive12.net, a fully automated service for easy and quick maintenance and new registration of domain names.

All the work has been performed on a system purchased in 2001. It is a Dual PIII 1000MHz with a gigabyte of RAM and a terabyte of storage. Not the most powerful box in the world, but it gets the job done. We used MySQL 5.0.20 with MyISAM tables to perform the analysis. The zonefile was copied 2006-04-24

What is a zone-file

DNS is the abbreviated for of Domain Name System (DNS). When you visit a website such as KFWEBS.NET your computer sends a request to translate the domain name, kfwebs.net., from a human readable form into a computer readable form, referred to as an IP-address. In the time of writing the IP address of kfwebs.net looks like 213.161.224.2.

The DNS is hierarchical, both in its form and its function. Considering a domain such as kfwebs.net , the Top Level Domain (TLD) would be NET, while the second level domain name would be KFWEBS.

An interesting note is that the full domain name is really "KFWEBS.NET.". Notice the final dot and feel free to try to enter it in your Web User Agent's address-bar. Although this is technically the full name, the final dot is usually omitted, but it shows the hierarchical structure of the domain.

A ZONE-file is used on DNS servers in order to delegate control and to store information about domains. In the COM zone-file there are the two records:

SECURE-MY-INTERNET NS NS3.KFWEBS.NET.
SECURE-MY-INTERNET NS NS4.KFWEBS.NET.

These lines tells where to get more information about the SECURE-MY-INTERNET second level domain, and hence delegates control to the mentioned nameservers.

NS3.KFWEBS.NET. and NS4.KFWEBS.NET. again contains the information required for the computer to turn the human-readable form of the name into a computer readable form using a so-called A record. This zone-file again looks like:

SECURE-MY-INTERNET.COM. A 72.29.83.156

For more in-dept information this wikipedia entry is recommended

Getting started

The COM zone file, known as com.zone.gz on the FTP server is 810 MiB of size. Uncompressed it is 3.7 GiB. There was a total of 117,379,441 records, as each domain name has multiple records due to the nature of the DNS system using multiple nameservers. A minimum of two is required, although it is recommended to use at minimum of three nameservers and not more than seven.

About 870 thousand records were A records for such nameservers, so those got deleted, we also ran the zone file through some shell scripts in order for it to become a list of unique domain names, resulting in a 685 MiB file.

Loading into the database and indexing it was going at a rate of about 560,000 domain names a minute. It was loaded into a total of 11 MyISAM database tables in MySQL, then combined using a 12th table of the MERGE storage engine. The MERGE storage engine, also known as MRG_MyISAM was introduced in MySQL 3.23.25. You can read more in the documentations

At the present time the COM zone-file holds 51,268,278 active domain names, meaning that on average a domain name has 2.2895 nameservers.