Archive for January, 2008
Catalog Choice registers half a million users! Monday, January 28th, 2008 by Matt
Catalog Choice on the Today Show.
On January 24, Catalog Choice saw its biggest day yet, when it was covered in a fantastic piece on NBC’s “The Today Show”:
Over the course of the day, the catalogchoice.org website saw over two million page views, and registered 60,000 new user accounts, bringing the total number of registered users, three days later, to over 500,000!
In addition, “Catalog Choice” was the number one search term for the day on Google:
Coping with the traffic.
Coping with a sudden increase in traffic, orders of magnitude more than typical, was a challenge. The front-end web application servers quickly became overloaded, and later the back-end DB server became overloaded (we were servicing over 2,000 DB queries per second!) Since it’s still not possible (with our hosting providers, at least) to bring on additional servers on-demand, we quickly made several modifications to the application:
We made a number of layout modifications in the application that would allow us to cache content to a far greater extent.
These same modifications also targeted the reduction of DB queries.
With these modifications, we were able to cope well with the secondary traffic surge.
Lessons learned.
It’s quite possible that Catalog Choice is now one of the largest Ruby on Rails applications running on the internet, in terms of number of users. Over the past few months of operation, we’ve learned some lessons:
Although not related to Rails, we’ve learned that it’s a good idea, especially for a site with this broad of a user base, to be conservative on the use of client-side technology. When originally launched, we had implemented elegant page transitions, catalog finder live type-ahead, and other similar UI features — all done with JavaScript (AJAX) in a way that gave the site a desktop-application feel. We considered this acceptable practice, as we were designing for IE 6/7, Safari 2/3 and Firefox 2/3.
However, when you have 500,000 users, even 1% on older browsers represents quite a large crowd! So we’ve since modified the site to work in a far more traditional manner, relying very little on client-side JavaScript, and where necessary, degrading very gracefully.
For hosting, our infrastructure, like many these days, is based on virtual machines. We have N number of front-end web application servers, each practically maxed out in terms of CPU and memory. Based on the experience with the Today Show traffic, we’re thinking now that it might be better to have 2N front-end servers, each with half the CPU and memory, since it’s a lot easier to quickly add CPU and memory to an existing server (to meet demand), than it is to bring on additional VMs. (This is, assuming 2N front-end servers with half the memory are roughly comparable in cost to N servers with double the CPU and memory, which might not be the case.)
It has been a very exciting experience to watch the site grow, analyze the usage patterns, and adjust the application and its user interface to not only improve the usability and user experience, but to adapt to the changing user profiles (i.e. now that over 500,000 of our visitors are no longer first-timers, and that we have over 1,000 merchants in the system.)
How the site is doing.
When the site first launched, the consumer response was (and continues to be) nothing short of amazing. It is clear that this site is meeting a very big need in the United States; that is the reduction of unwanted paper catalogs. The industry’s response was, expectedly, lukewarm, especially after the Direct Marketers Association (the DMA) issued an email to all its members to “Just say no!” to Catalog Choice.
However, with half a million vocal consumers behind it, Catalog Choice has become an influential heavyweight. A website feature we launched last week alerts users to which specific merchants have refused to honor their opt-out requests, and provides the merchants customer support telephone number, just in case the consumer would like to give them a call. Within 24 hours, after being inundated with phone calls from angry customers, we had merchants changing their minds :-)
A misconception in the industry (promoted by the DMA) is that Catalog Choice seeks to do away with catalogs altogether. That couldn’t be further from the truth. Catalog Choice is about doing away with just those catalogs that are unsolicited and unwanted.
All in all, Catalog Choice has been a fantastic project for MakaluMedia. We’re fortunate to be one of very few companies having the opportunity to build and operate such a large-scale Rails site, and a site that serves such a meaningful social purpose!