Surviving the Slashdotting

Earlier in the year, David A. Smith and Andreas Raab put a download package containing an earlier version of the Croquet software (known as "Solar" Croquet) on a previous version of the Croquet Project website published on the Squeakland server. It quickly became necessary to shut the server down for two reasons: The first was that they were "slashdotted", that is, the web site http://slashdot.org mentioned us and our former low-capacity server. The site was immediately overrun. Since that earlier site was intended to be lightly used, the immediate widespread interest that we received became somewhat of a problem. The other reason is that the version that had been posted there was only intended for small scale distribution in order to get some comments from a few people. It was never intended that the earlier site and its download package would serve as a launch pad for the project.

In July of this year, I committed to a complete redesign and implementation the Croquet Project website and have been hosting it here at the University of Wisconsin since then. For most of that time it's been in the back of my mind that once the Jasmine developers' release was made available for download, a subsequent slash-dotting might bring our server to its knees. The obvious solution was to set up BitTorrent as an alternate download option for each file on the site. This would decentralize the traffic and make it possible for us to scale up to any level of demand for the files. For those of you who are unfamiliar with this great new technology, BitTorrent is a peer-to-peer protocol, that allows users to connect to each other directly to send and receive portions of a file. Imagine that, using BitTorrent to distribute a legitimate file...what next?

On the evening of October 11th (which was the promised date for the "Jasmine" developers' release), Mark McCahill and I were furiously building, testing, packing, and uploading the platform-specific images to the Wisconsin servers. With the assistance of Dave Schroeder here at Wisconsin, we were able to publish the .torrent files on the site and "seed" two fast Internet2-connected machines with copies of the download packages. All of this before the stroke of midnight. Later the next day, and in the midst of a slashdot effect of about one million hits on the server over the twenty-four hours following a Slashdot posting on the morning of the 12th, David P. Reed was able to perform a 'blindingly fast" download of a 60 megabyte download package to his box at MIT using BitTorrent. No problemo...