Archive for the 'Hosting' Category
Catalog Choice registers half a million users! Monday, January 28th, 2008 by Matt
Catalog Choice on the Today Show.
On January 24, Catalog Choice saw its biggest day yet, when it was covered in a fantastic piece on NBC’s “The Today Show”:
Over the course of the day, the catalogchoice.org website saw over two million page views, and registered 60,000 new user accounts, bringing the total number of registered users, three days later, to over 500,000!
In addition, “Catalog Choice” was the number one search term for the day on Google:
Coping with the traffic.
Coping with a sudden increase in traffic, orders of magnitude more than typical, was a challenge. The front-end web application servers quickly became overloaded, and later the back-end DB server became overloaded (we were servicing over 2,000 DB queries per second!) Since it’s still not possible (with our hosting providers, at least) to bring on additional servers on-demand, we quickly made several modifications to the application:
We made a number of layout modifications in the application that would allow us to cache content to a far greater extent.
These same modifications also targeted the reduction of DB queries.
With these modifications, we were able to cope well with the secondary traffic surge.
Lessons learned.
It’s quite possible that Catalog Choice is now one of the largest Ruby on Rails applications running on the internet, in terms of number of users. Over the past few months of operation, we’ve learned some lessons:
Although not related to Rails, we’ve learned that it’s a good idea, especially for a site with this broad of a user base, to be conservative on the use of client-side technology. When originally launched, we had implemented elegant page transitions, catalog finder live type-ahead, and other similar UI features — all done with JavaScript (AJAX) in a way that gave the site a desktop-application feel. We considered this acceptable practice, as we were designing for IE 6/7, Safari 2/3 and Firefox 2/3.
However, when you have 500,000 users, even 1% on older browsers represents quite a large crowd! So we’ve since modified the site to work in a far more traditional manner, relying very little on client-side JavaScript, and where necessary, degrading very gracefully.
For hosting, our infrastructure, like many these days, is based on virtual machines. We have N number of front-end web application servers, each practically maxed out in terms of CPU and memory. Based on the experience with the Today Show traffic, we’re thinking now that it might be better to have 2N front-end servers, each with half the CPU and memory, since it’s a lot easier to quickly add CPU and memory to an existing server (to meet demand), than it is to bring on additional VMs. (This is, assuming 2N front-end servers with half the memory are roughly comparable in cost to N servers with double the CPU and memory, which might not be the case.)
It has been a very exciting experience to watch the site grow, analyze the usage patterns, and adjust the application and its user interface to not only improve the usability and user experience, but to adapt to the changing user profiles (i.e. now that over 500,000 of our visitors are no longer first-timers, and that we have over 1,000 merchants in the system.)
How the site is doing.
When the site first launched, the consumer response was (and continues to be) nothing short of amazing. It is clear that this site is meeting a very big need in the United States; that is the reduction of unwanted paper catalogs. The industry’s response was, expectedly, lukewarm, especially after the Direct Marketers Association (the DMA) issued an email to all its members to “Just say no!” to Catalog Choice.
However, with half a million vocal consumers behind it, Catalog Choice has become an influential heavyweight. A website feature we launched last week alerts users to which specific merchants have refused to honor their opt-out requests, and provides the merchants customer support telephone number, just in case the consumer would like to give them a call. Within 24 hours, after being inundated with phone calls from angry customers, we had merchants changing their minds :-)
A misconception in the industry (promoted by the DMA) is that Catalog Choice seeks to do away with catalogs altogether. That couldn’t be further from the truth. Catalog Choice is about doing away with just those catalogs that are unsolicited and unwanted.
All in all, Catalog Choice has been a fantastic project for MakaluMedia. We’re fortunate to be one of very few companies having the opportunity to build and operate such a large-scale Rails site, and a site that serves such a meaningful social purpose!
Phenomenal response to Catalog Choice Tuesday, November 6th, 2007 by Matt
HARDLY A MONTH HAS PASSED since we announced the launch of Catalog Choice, a website that MakaluMedia was chosen to design and develop by an amazing team of funding and supporting organizations, and the response to the site has been extraordinary.
In less than 25 days, more than 100,000 users have registered, opting out of more than 800,000 catalogs. And for us, as user experience designers and product developers, one of the most exciting statistics is the near 50% conversion ratio, meaning that nearly one out of every two visitors to the site has registered.
Update November 21, 2007: We are about at the one month point now, and have over 180,000 users, opting out of nearly two million catalogs!.
Catalog Choice has also been featured on the national television networks CNN, NBC and ABC (see videos below), and has been well received in the press and blogosphere:
- The New York Times
- The Boston Herald
- The Chicago Tribune
- Guy Kawasaki
- Joel on Software
- Adam Engst at TidBITS
The success of the site has served to validate much of our beliefs regarding user experience design. Some of principles that stand out include:
- Simple is better. Less is more.
- A successful user experience derives from near obsessive attention to detail.
- Don’t outsource your user support. On the contrary, put your best people on it. People love getting a personal response from somebody that knows what they’re talking about. You’ll reap the benefits in trust, patience and tremendous value in terms of feedback to the process of continually improving the user experience.
We’ve also learned a wealth about effectively operating a high-volume Ruby on Rails application. (On one particular day, the ABC “Good Morning America” spot was the very top news article on Yahoo News, and we saw 20,000 new user registrations in a period of four hours.) Having a great hosting partner helps a lot!
Finally, this project is proof positive that talented and motivated people, supported by an effective agile process and paying a lot of attention to clear communications, can build a successful project, even if separated great distances geographically. Our Catalog Choice team includes MakaluMedia staff located in the United States, Germany, Spain and Ireland!
And with that, we bring this article to a close. Our next project milestones include one million catalog opt-outs, and hopefully down the road one millions users! And remember, if you’re located in the US, and ready to simplify your life while helping the environment, head on over to Catalog Choice and sign up for a free account:
Be sure to check out the following video clips.
- Catalog Choice on NBC
- Catalog Choice on CNN
- Catalog Choice on ABC Good Morning America
Postscript to the Slashdot Effect Thursday, June 1st, 2006 by Arto
The actual slashdotting is now well over, and I’m glad to say we weathered the storm without any incidents. With the help of our sysadmin, Niall, we distributed the load across three dedicated servers in geographically diverse locations, and none of the boxes even broke a sweat.
However, that’s not to suggest the servers were idling; on the contrary, they were each servicing up to hundreds requests per second. I usually keep a monitoring console open to the servers, and when CmdrTaco originally posted the announcement, it was immediately obvious that something had happened. The staff in our colocation facilities noticed, too — it didn’t take many minutes for the first e-mail alert to arrive. (I fancy they heard the dual-CPU fans suddenly spooling up to maximum effect, but of course, their monitoring systems just warned them of a possible DDoS attack.)
Here’s a nice bandwidth graph from one of our servers, covering the first 24 hours. I thought the start of the slashdotting might be obvious enough that I won’t bother to specifically point it out:
On the software side of things, I was especially pleased to discover that no adjustments to Lighttpd’s settings were necessary in order for it to handle massive concurrency.
The load averages remained very reasonable, under 15.0 even with the nightly full backup running, except for a human error that resulted in one of the servers becoming momentarily unresponsive with a load average of almost 500.0 (oops); this was remedied within a minute.
Unfortunately, we didn’t get to test our server systems’ capacity to the fullest under a simultaneous, combined Slashdot/Digg/Reddit assault, since both Digg and Reddit linked to the actual design preview, and the direct links pointing here from the comments didn’t bring in much traffic to constitute a hammering. Oh well, live to fight another day.