Ike.ninja

(May 15) Several security issues were fixed in the kernel.

(May 20) Updated kernel packages that fix one security issue are now available for Red Hat Enterprise Linux 6.1 Extended Update Support. The Red Hat Security Response Team has rated this update as having [More…]

In September 2012 Netcraft reported that Amazon had become the largest hosting company in the world based on the number of web-facing computers. In the last eight months, the e-commerce company’s tally of web-facing computers has grown by more than a third, reaching 158k. The number of websites hosted on these computers has also increased, from 6.8M in September 2012 to 11.6M in May 2013, a 71% increase.

Although Amazon’s main business is still online retail, Amazon Web Services (AWS), its cloud computing division, has been growing in significance. In Amazon’s first quarter of 2013 the Other category (which still includes AWS along with other non-retail activity) was just under 5.0% of its revenue, up from 3.2% at the same point in 2011. The first publicly available AWS service was launched in 2004, but it was not until 2006 that Amazon launched its two core services S3 (data storage) and EC2 (per-hour rental of virtual computer instances). Since then, Amazon has been increasing the number of services provided: in 2012 alone, 159 new services and features were released.

Including its retail infrastructure, the number of web-facing computers at Amazon has grown more than thirty-fold in four years: in May 2009, Netcraft found 4,600 Amazon-controlled web-facing computers; in May 2013, Netcraft found 158k web-facing computers on 164k IP addresses. Netcraft estimates the number of computers behind a group of IP addresses by using a variety of heuristics based on the TCP/IP characteristics seen in the HTTP responses gathered. Hosted on those computers, there are more than 11.6M websites (or hostnames) which corresponds to 2.1M websites with unique content (active sites). Despite being the largest hosting provider by number of web-facing computers, it is dwarfed by Go Daddy, the largest hosting provider when considering the number of websites hosted. Go Daddy has 37M websites on just 23k web-facing computers: the high ratio of websites to web-facing computers may be indicative of Go Daddy’s role as a registrar, for which it has a large network of holding pages, and its inexpensive shared hosting platform.

EC2 – Elastic Compute Cloud

EC2, provides on-demand virtual-computer instances billed per hour and is currently available from all nine AWS regions. Each region may correspond to multiple physical data centres which are structured into “Availability Zones”. The two largest regions, US East (Northern Virginia) and EU West (Ireland), account for more than three-quarters of all EC2 usage as measured by Netcraft. Sydney, the newest AWS region, now accounts for just under 1% of all measured web-facing computers using AWS, having almost tripled in size in the past four months. In total, more than 156k instances power at least one hostname on 3M domains across the internet.

Launched in 2011, the GovCloud (US) region is specifically intended for more sensitive applications that require additional security and compliance with US regulations. As of May 2013, Netcraft found just 27 web-facing computers within the government cloud, some of which power www.grdregistry.org and www.govdashboard.com. Given its intended role, it would not be surprising if a large proportion of the computers used in the region are not web-facing.

Notable EC2 users include Netflix, a DVD rental and video streaming service, Instagram, a photo sharing application now owned by Facebook, and DuckDuckGo, a search engine.

Metric (EC2 Total)	February 2013	March 2013	April 2013	May 2013	Growth (4 month)
Web-facing Computers/Instances	141,960	145,648	152,041	156,225	10%
IP Addresses	144,625	148,837	155,712	160,884	11.2%
Domains	2,788,685	2,810,906	2,996,147	3,061,178	9.8%
Hostnames	9,489,496	9,938,480	10,649,545	10,925,661	15.1%

Many uses of EC2 such as batch data-processing will not be directly measurably over the internet: Netcraft measures publicly visible computers with corresponding DNS entries and which respond to HTTP requests. Netcraft’s Web Server Survey is run at Amazon from the Northern Virginia region, so the region may be over-reported due to services like latency based multi region routing which provide differing responses depending on topological location.

Geographic distribution of computers per EC2 region in May 2013

Data Centre (EC2 – Web Facing Computers)	February 2013	March 2013	April 2013	May 2013	Growth (4 month)
Asia Pacific (Singapore)	6,576	6,805	6,998	7,290	10.9%
Asia Pacific (Sydney)	499	739	1,129	1,427	186%
Asia Pacific (Tokyo)	7,342	7,595	8,065	8,601	17.1%
EU West (Ireland)	23,778	24,635	25,326	25,942	9.1%
South America (Sao Paulo)	2,115	2,263	2,396	2,655	25.6%
US East (Northern Virginia)	87,094	88,543	92,426	93,537	7.4%
US West (Northern California)	9,325	9,478	9,715	9,695	4%
US West (Oregon)	5,217	5,573	5,965	7,051	35.2%
GovCloud (Oregon)	14	17	21	27	92.9%

S3 – Simple Storage Service

S3 provides an online file storage service which can be managed programmatically via Amazon’s API. Files are logically grouped into containers called buckets which can be made public and accessible over HTTP but default to being private. As with EC2, Netcraft cannot track private use of S3 but is able to survey websites using S3 publicly to serve static files and even entire websites.

Metric (S3 Total)	February 2013	March 2013	April 2013	May 2013	Growth (4 month)
Domains	41,782	42,561	45,721	48,636	16.4%
Hostnames	124,454	127,370	132,962	138,588	11.4%

In May 2013, a total of 139k hostnames were found to be hosted directly on S3, either using a subdomain of s3.amazonaws.com or using a custom CNAME pointing to S3. Of these, 24.7k hostnames, or over 18.5k domains, point to an S3 bucket configured to serve an entire website, as does mediahackers.org. Many more websites are not hosted entirely on S3, but make use of the service to serve static files such as images, stylesheets, or file downloads.

One of the most widely referenced S3 hostnames is used for twitter badges bucket, which was once a common method to display twitter icons on a third-party website. Tumblr, a popular blogging platform recently acquired by Yahoo!, also makes use of S3 to host static media.

CloudFront

CloudFront is a Content Delivery Network which can be used to serve both dynamic and static content from 28 edge locations which are topologically closer to a site’s visitors. Caching content reduces the bandwidth and performance requirements on the website’s own servers and, by being topologically close to visitors, the latency associated with each HTTP request can be improved.

In the May 2013 survey, more than 63k hostnames were served via CloudFront, more than 60% of which point to an S3 bucket. Amazon uses CloudFront on its own websites, including imdb.com, and also uses it for serving images on Amazon.com. Other than Amazon itself, CloudFront users include: the Toronto Star, a Canadian newspaper, and Pirifrom, the makers of utility program CCleaner, are two of the most visited sites using CloudFront amongst users of the Netcraft Toolbar.

Metric (CloudFront Total)	February 2013	March 2013	April 2013	May 2013	Growth (4 month)
Domains	22,920	24,079	25,264	26,221	14.4%
Hostnames	55,578	57,817	60,475	63,203	13.7%

The number of CloudFront-dedicated IP addresses and computers cannot be easily measured as different results are obtained depending on the location of the request.

Route 53

Route 53, is a managed Domain Name System (DNS) hosting service. Route 53, named for the TCP and UDP port used for the protocol, hosts DNS records which map from human-readable hostnames to IP addresses. Integrated with the rest of AWS, it allows programmatic access to change DNS records in response to changes elsewhere in a customer’s infrastructure. As with CloudFront, Amazon have servers providing this service in edge locations outside of its 9 EC2 regions; Route 53 is available from 28 separate locations.

Metric (Route 53 Total)	February 2013	March 2013	April 2013	May 2013	Growth (4 month)
Domains	136,698	146,635	161,619	169,111	23.7%
Hostnames	3,493,986	3,662,195	3,831,910	4,068,053	16.4%

Over the past four months there has been a steady growth in the number of websites using Route 53 to host their DNS records: it now serves DNS records for 169k domains. Busy sites making use of this service include pinterest.com, a social photo-sharing website which is a heavy user of Amazon’s infrastructure; MediaFire, a file uploading and sharing service; and ow.ly a URL shortener.

Heroku

Heroku is Platform as a Service (PaaS) provider owned by Salesforce. Whilst not operated by Amazon, it makes heavy use of AWS services, especially EC2. Heroku provides an abstracted managed environment for web developers to deploy applications in a number of different languages. In May 2013, Heroku was serving 70K domains directly (not behind a CDN) across 4,786 computers.

Popular sites using Heroku include www.upworthy.com, a curated news website; help.github.com, a knowledge base for the popular git-based project hosting service; and Absolventa, a German job market.

Metric (Heroku total)	April 2013	May 2013	Growth (2 month)
Computers	4,293	4,786	11.5%
IP Addresses	4,408	4,972	12.8%
Domains	65,821	69,781	6%
Hostnames	1,094,578	1,102,663	0.7%

Heroku, as demonstrated in the results from Netcraft’s survey, has been available almost exclusively from the Northern Virginia EC2 region. In April, Heroku announced availability of its service in Europe from the AWS EU West region based in Ireland. Only a limited number of Heroku customers have had access to this region during a private beta phase which explains the currently low uptake: only 1% of the computers attributed to Heroku were in the region.

IP Addresses	April 2013	May 2013
US East (Northern Virginia)	4,374	4,915
EU West (Ireland)	33	56

Summary

The launch of the new AWS region, hundreds of new services, new partnerships, and multiple price reductions, are a clear indicator of the relentless growth of Amazon Web Services.

Netcraft provides information on the Internet infrastructure, including the hosting industry, and web content technologies. For information on the cloud computing industry including Microsoft Azure, Rackspace Cloud, and Google App Engine, please contact [email protected].