Porn sites are often technology leaders. They have to be. It is believed that porn is the most widely viewed content on the Internet. Just how much traffic do they serve? YouPorn, who serves free pornographic videos and is the second largest porn site on the Internet, hosts over 100TB of porn. At peak times, they serve over 100 million pages per day or about 950 terabytes of data over the course of 24 hours. Other than Google and Facebook, they dwarf everything else on the Internet.
Porn sites differ from regular websites in that visitors typically spend 15-20 minutes on the site (vs. 2-6 minutes for the average news website) and they typically serve video rather than text and images. As such, they deliver tremendous amounts of data over the Internet, often in unbelieveable bursts of traffic.
Most of the data served by Youporn is streaming video which accounts for 28 petabytes per month. During spikes in service, Youporn often serves 4,000 pages per second or about 100 gigabytes per second (the equivalent of 3 full DVDs streamed every second) and generates about 15GB of log data per hour.
Youporn won’t release too many details about their hardware infrastructure but they do acknowledge that they use a pretty traditional LAMP stack with web pages served using the high performance Nginx web server. Their primary data store is Redis (a key-value data store) programmed using PHP (they switched from Perl and MySQL some time ago). They use HAProxy (load balancing and intelligent distribution) and Varnish (reverse proxy and cache management) to load balance their servers. Other technologies include ActiveMQ (message bus), Symfony2 (development framework), and Syslog-ng (used for view counters). They are gunning for a capacity of 200+ million daily requests.
Their no-sql Redis data store itself server handles 300,000 queries per second with writes updated in real-time. When they first switched over to Redis, they had to add additional nodes, not because Redis couldn’t keep up but rather, because their network cards could not keep up with Redis.
It is believed that they generate monthly ad revenue of $120,000 and account for about 2% of the total traffic on the Internet.
Sources: ExtremeTech, High Scalability magazine, Wikipedia, Global Warming.
