OmniTI's Surge. Year one. Done. Awesome.
This past Thursday and Friday (and much of Wednesday honestly), OmniTI went where it never went before: we ran a conference. Surge 2010 was, in fact, a unique conference despite its close relevance to others in the industry. This conference was assembled to answer one question: "How do I build systems that perform well and scale to meet demand?" That may seem like a simple question that someone has asked before, but I am an engineer and, as such, use an engineering definition of "system" and have a fearful respect for what "demand" can really be.
I'd like to shout out a huge CONGRATULATIONS to all of the OmniTI team involved in orchestrating this event. People stopped no fewer than fifty times during the two day event to tell me how professional they thought the event was: specifically the polish of the brand, the beautifully consistent quality and composure of the conference itself, the venue, the proficiency of the speakers and fact the the content presented was all on-topic and well oriented to, what I shall call, the Internet Architect Practitioner.
A system is a set of connected things or parts forming a complex whole and as such, it isn't "a computer." Systems, particularly complex systems, are not only fascinating, but also happen to be ubiquitous between all of us and every useful website on the planet. Designing a system isn't about writing a piece of software or running a computer, it is about so much more. In the context of Internet computing, it is about the connectedness of many components including logical, electrical, optical and even mechanical. Bryan Cantrill did a spectacular job enlightening the crowd to the fascinating intricacies of failures in complex systems.
In fact, in talking to Cantrill, he suggested that each talk should be required to have at least one real-world failure described. I personally believe that learning from another's success is far less effective than learning from their failures. Brian was describing his impression of Tom Daly's excellent talk on how Dyn leverages AnyCast and partway through he said to me: "Given the complexities of this system, I have to think when this thing goes south, it must go really south." True enough, Tom described a rather intricate cascading failure all summed up with two words: fascinating and educational.
Demand is, in the beginning and the end, the bedfellow of supply in the racy world of economics. The interesting part in Internet systems (as opposed to economics) is how the system reacts to conditions in which demand exceeds supply. In economics, inventory is exhausted and future prices rise. In our systems, resources are depleted and experience nose-dives. If this condition persists, we have a natural economic scenario where that demand for a crappy experience will not erode and general demand will fall bringing back balance. However, we can "just make more." We can add more resources and ensure that resources aren't depleted. And, most importantly, if you architect your systems well, you can do this both successfully and cost-efficiently. Demand on the Internet has a dark side in that while it is bounded, it is bounded by the population of planet Earth. Ouch.
Artur Bergman presented details on how Wikia serves its global audience both successfully and (very) cost-effectively. Artur Bergman never fails to make me think just a bit outside of the box I'd been thinking in; always refreshing and always useful. I've been struggling with how to accomplish the real-time variant of what Artur has done aggregating actual user-perceived performance; the problem isn't that hard, the trick is scaling it as a service. The idea that one can leverage Google Analytics to do this, for free, in hour-relative real-time is damn spiffy.
I learned a tremendous amount at Surge 2010. My job, amongst others, is to keep abreast of the current technologies and methodologies used to power the largest websites in the world (top 500). I take that knowledge and extract or adapt those techniques and help companies in the top 10000 compete more effectively. I read academic papers, ACM and IEEE journals every week. I talk to colleagues. I read "trendy" tech outlets like hackernews and reddit. I gather as much knowledge as possible. But, what do I do with it?
My goal is to share that knowledge with my team at OmniTI. I've struggled with how exactly to do this in the past. I have to say that my most successful effort to date was Surge 2010. Almost everyone from OmniTI attended and many told me that they left Surge feeling educated and (more importantly) inspired by the truly enthralling industry in which we all work.