July 2008

The ever-reliable highscalability.com has a whole evening’s worth of detailed reading about GAE piled into one place, including lots of reallllly interesting details on the difference between BigTable and RDBMS.

This article is ancient history by blog standards, being from all the way back in April, but it gives a start at answering a question that was hot – but ultimately unanswered – at the recent San Francisco CloudCamp. In talking about how the enterprise adoption of cloud computing would depend in large part on how risks were addressed, shifted and mitigated, analogies were repeatedly drawn to past moves to “outsourcing something critical”. Examples ranged from the more recent shift from in-house to outsourced data centers, exemplified by Exodus Communications, to more ancient, classic tech shift examples such as utility power. However, the analysis that night didn’t quite get past the “Hmm, that’s an interesting point” stage. Kirkpatrick’s post takes the next step.

The question is “Can cloud computing smite down evil zombie botnet armies?”

The answer, IMHO, is “No”.

For the blissfully uninitiated, botnets are overlay networks in which compromised hosts on the Internet are harnessed up to some master command server to order the botnet to attack targets on the Internet, e.g. enabling a distributed denial of service attack. It’s also a popular resource management tool to marshall hosts for use by spammers. Here’s a solid backgrounder on the subject.

The core research  idea – Self Cleansing Intrusion Tolerance – is an interesting security research topic. It starts from the premise that there will always be some attack that is more sophisticated than your defenses, so all hosts should eventually be assumed to be compromised over time, and restarted at some last know secure state. The “assume compromise” premise is realistic, if unpopular, and now we have modern tools which have caught up to the classic security good practice of “reinstall from a day 0 backup in the event of a security compromise”. With virtualization, there’s a ready means to return to day 0. SCIT takes this to an extreme, constantly reboot alternating slices of your virtual server farm, so that any malware has only a minimal time to work before it is removed in favor of a fresh install.

As an aside, such an approach would require having a way to know that a virtual host is not in the middle of servicing a user (human or otherwise) connection before shutting down, or the farm will have a built-in “flakiness” quotient in which a percentage of all user connections will be intentionally broken in the name of the greater good each hour, which is not such an elegant solution for routine use.

Nonetheless, the basic idea, of taking advantage of the built-in day 0 backup inherent in virtualization on a routine basis, is sound. Viewing it as a silver bullet against botnets and worms is not.

A hearty malware infestation is moving at a much faster rate than the 1 minute reboot cycle proposed. Some malware would simply reinfest a portion of, or even all of the same virtual servers every minute, with the remainder of the 60 second window being enough to launch outbound attacks. We’re talking generally about small programs performing complete operations in chunks of a few seconds or subseconds at a time.

Restarting might even be a boon for malware writers, since they can do some damage to other hosts and then know that their tracks will disappear in a minute. And an autoreboot pattern on a large virtual farm will be noticeable remotely, and then the botnet C&C software can be modified to a) flag the autorebooting hosts as such; b) perhaps have policy-based reinfection of same (if that’s even necessary, given the speed at which infestations can move); b) and policy might include selling botnet space in an autorebooting farm as a separate service at a different rate – “1000 forensics-proof temporal zombies for $49.95!”.

Computer installations all tend towards collecting cruft over time, with malware as a malicious and extreme form of cruft. Virtualization offers the convenient opportunity to periodically clean out the crap from a system, including the evil variants, so the general idea of regularly dropping back to a known good version is worth exploring. Its already being done on the client side in the world of thin clients. But the bots will adapt and propagate happily onward…