One final quick note on the Rockies ticket sales debacle - following up on my previous posting[1] on the subject. This note discusses how to including the humans in your system design can improve both your perceived availability and your customer satisfaction while cutting your costs.
Pacolian did tweak their application to be a little more friendly to their infrastructure and perhaps fixed some networking issues, so that people did eventually get tickets to the Rockies/Red Sox games in Denver. But, many, many people got told they were going to get tickets, but the system was still so slow and under such heavy load that it timed out many or even most users before they could pay for their tickets. So, even the second try was highly unsatisfactory, and increased infrastructure costs. Although you could argue that the system didn't crash, having it be so slow as to be unusable certainly creates the perception of unavailability in the minds of many users.
If you want to sell tickets to a hot event, there are many ways to do it. For any system you come up with, you can expect the users who use it to try and game the system somehow to increase their odds.
What are the goals of selling tickets to events?
- sell all the tickets
- fairness
- discourage professional ticket scalpers
- limit number of tickets sold to any one party
- make "gaming" the system harder
- minimize infrastructure costs
- minimize customer frustration
The way Pacolian did it, was to allow people to queue up in the web site and then sell the tickets first-come-first served beginning at a certain time. Although this might sound reasonable because it's modeled after how tickets are sold at box offices in the real physical world, it's a bad idea in the Internet. This encourages exactly the behavior that took their systems down, maximized customer frustration, and infrastructure costs, while still giving an edge to professional scalpers. In their system, if you want tickets, obviously you open as many browser windows at a time as you can, hit the system as hard as you can at exactly the same time to increase your chances of getting in line first. This creates a situation similar to what operating system people sometimes call thundering herd[2] [3] behavior. The difference here is that the thundering herd loop included people and browsers. It makes kind of a comical sight if you think about it...
Another method which I seem to recall hearing that other vendors use is a lottery system. Such a system might work like this (details may need tweaking):
- Announce a registration period of 8-18 hours where people can register for which games they want to buy tickets for, and which price ranges they're willing to buy.
- Anyone registering during this time can register once, and give an email address where they can be reached, and which can be accessed from this computer. Warn people not to do this on shared computers.
- Each computer gets one cookie and one email address for registering. If you try and re-register the same browser or use the email address multiple times, you're rejected.
- At the end of the registration period, winners are randomly selected, and notified by email including a random cookie which is correlated to the cookie given to their browser earlier. (you have to have both the cookie and the email to purchase a ticket).
- They have a specified period of time to purchase the tickets, and any given credit card number can't be used more than once per game. Any tickets not paid for by that time get sold to other applicants.
This means that there is no advantage to registering early or late. As long as you register, you have the same chance as anyone else. The peak load on such a lottery system system is probably one or two orders of magnitude below that of the Pacolian thundering herd design.
The result of this is:
- System availability goes up,
- Customer satisfaction goes up
- Infrastructure costs go down.
- The possibility of gaming the system still exists, but is probably no worse than the original Pacolian system.
All in all, a big win. To be fair, I didn't think of this idea myself, but before the Rockies/Pacolian debacle, I'd never put much thought into ticket sales either. Because I'm not an expert in this area, there are no doubt many improvements that an expert would make to my proposal to make it harder to game. However, from an availability perspective, this application design is much more robust than the original Pacolian system because it takes into account the motivations of the humans that are part of the computer system.
References
[1] http://techthoughts.typepad.com/managing_computers/2007/10/the-cost-of-un-.html
[2] http://catb.org/jargon/html/T/thundering-herd-problem.html
[3] http://en.wikipedia.org/wiki/Thundering_herd
Comments