A Heap of App Engine Reading
The ever-reliable highscalability.com has a whole evening’s worth of detailed reading about GAE piled into one place, including lots of reallllly interesting details on the difference between BigTable and RDBMS.
The ever-reliable highscalability.com has a whole evening’s worth of detailed reading about GAE piled into one place, including lots of reallllly interesting details on the difference between BigTable and RDBMS.
This article is ancient history by blog standards, being from all the way back in April, but it gives a start at answering a question that was hot - but ultimately unanswered - at the recent San Francisco CloudCamp. In talking about how the enterprise adoption of cloud computing would depend in large part on how risks were addressed, shifted and mitigated, analogies were repeatedly drawn to past moves to “outsourcing something critical”. Examples ranged from the more recent shift from in-house to outsourced data centers, exemplified by Exodus Communications, to more ancient, classic tech shift examples such as utility power. However, the analysis that night didn’t quite get past the “Hmm, that’s an interesting point” stage. Kirkpatrick’s post takes the next step.
The question is “Can cloud computing smite down evil zombie botnet armies?”
The answer, IMHO, is “No”.
For the blissfully uninitiated, botnets are overlay networks in which compromised hosts on the Internet are harnessed up to some master command server to order the botnet to attack targets on the Internet, e.g. enabling a distributed denial of service attack. It’s also a popular resource management tool to marshall hosts for use by spammers. Here’s a solid backgrounder on the subject.
The core research idea - Self Cleansing Intrusion Tolerance - is an interesting security research topic. It starts from the premise that there will always be some attack that is more sophisticated than your defenses, so all hosts should eventually be assumed to be compromised over time, and restarted at some last know secure state. The “assume compromise” premise is realistic, if unpopular, and now we have modern tools which have caught up to the classic security good practice of “reinstall from a day 0 backup in the event of a security compromise”. With virtualization, there’s a ready means to return to day 0. SCIT takes this to an extreme, constantly reboot alternating slices of your virtual server farm, so that any malware has only a minimal time to work before it is removed in favor of a fresh install.
As an aside, such an approach would require having a way to know that a virtual host is not in the middle of servicing a user (human or otherwise) connection before shutting down, or the farm will have a built-in “flakiness” quotient in which a percentage of all user connections will be intentionally broken in the name of the greater good each hour, which is not such an elegant solution for routine use.
Nonetheless, the basic idea, of taking advantage of the built-in day 0 backup inherent in virtualization on a routine basis, is sound. Viewing it as a silver bullet against botnets and worms is not.
A hearty malware infestation is moving at a much faster rate than the 1 minute reboot cycle proposed. Some malware would simply reinfest a portion of, or even all of the same virtual servers every minute, with the remainder of the 60 second window being enough to launch outbound attacks. We’re talking generally about small programs performing complete operations in chunks of a few seconds or subseconds at a time.
Restarting might even be a boon for malware writers, since they can do some damage to other hosts and then know that their tracks will disappear in a minute. And an autoreboot pattern on a large virtual farm will be noticeable remotely, and then the botnet C&C software can be modified to a) flag the autorebooting hosts as such; b) perhaps have policy-based reinfection of same (if that’s even necessary, given the speed at which infestations can move); b) and policy might include selling botnet space in an autorebooting farm as a separate service at a different rate - “1000 forensics-proof temporal zombies for $49.95!”.
Computer installations all tend towards collecting cruft over time, with malware as a malicious and extreme form of cruft. Virtualization offers the convenient opportunity to periodically clean out the crap from a system, including the evil variants, so the general idea of regularly dropping back to a known good version is worth exploring. Its already being done on the client side in the world of thin clients. But the bots will adapt and propagate happily onward…
Here’s an interesting poll on whether their users are interested in paying software monthly, by The Escapist magazine. As of this moment, 9:30 PM July 4th (why the he** am I blogging?), 81.5% (22) said no and only 18.5% (5) said yes. According to the site,
The Escapist Features cover digital entertainment culture with a progressive editorial style, with articles and columns by the top writers in and outside of the digital entertainment industries. A weekly publication, the Features’ magazine-style updates offer content for a mature audience of entertainment enthusiasts, industry insiders and other “NetSet” readers.
So the site is definitely not your typical enterprise software focused sites. However, I would assume some of the gamers are also corporate workers. It’s interesting that large percentage of their audience (though the sample is fairly small) are not interested in SaaS.
So would you be interested in paying monthly for software, aka SaaS? Leave a comment and let us know what SaaS product you are currently using or planning to use.
There’s a lot of debate about how much can be abstracted away by “the cloud”, but on the other side there is always a concrete implementation to create a service. Here’s an interesting case study around a highly scalable consumer SAAS application - Facebook’s Chat service. The article is a bit older (May), but it’s still timely, in showing how choice of implementation language matters in architecture. And Erlang is one of those programming languages that usually only comes up in rarefied telecom circles or when software engineers want to demonstrate that the feathers in their plume are brighter than others. Seeing it outside of its normal hiding places and showcased on a stage like Facebook is interesting in its own right.
Sam Charrington, one of the organizers of CloudCamp, put up a review of the three conferences. Found a picture of me (at the end of the table talking) in one of the sessions. (Thanks Sam!)
Stacey Higginbotham over at GigaOM wrote an interesting piece on 10 Reasons Enterprises Aren’t Ready to Trust the Cloud. Even though I agree that some of these points are valid reasons on why enterprises are hesitant in moving into the cloud, I have to wonder whether Stacey meant to be provocative (read: flame bait) on the piece. Also, the piece seems to be quite opinionated and lack support in many cases. Let’s drill down on it a bit.
1. It’s not secure.
I have written extensively on this blog (here and here) regarding the security concerns of SaaS and cloud computing. However, saying that the cloud is not secure is definitely a stretch. I would like to see some supporting evidence on this. The only major “security breach” I’ve seen is probably the Salesforce case.
In addition, none of the regulations or industry mandates, including HIPAA, GLBA, SOX, PCI, FISMA, etc etc, say anything about not allowing data to be outside of the corporate firewalls. In fact, many of the enterprises in the affected industries are already outsourcing some of their critical data. For example, financial companies using credit card processing services such as ViewPointe. There’s also plenty of hospitals using external services. PCI also has a specific section on hosting providers. Again, no regulation or mandate explicitly state that data cannot leave the corporate firewalls.
What CIOs/CSOs care mostly about is that cloud (application or platform) providers must meet their security requirements, there’s transparency in the security and operational practices, and that they can audit the provider or review the appropriate audit reports from the provider. The issue comes down to trust.
2. It can’t be logged.
Again, this is really about auditability, especially for compliance. This is definitely an area that’s lacking and cloud providers would be wise to do more in this area. Again, I wrote about that here: 4. Access Audit - Who has accessed my data and where’s my access logs?
3. It’s not platform agnostic.
Seriously though, is this really an issue? We are still in the world of multiple OS platforms, including different variants of Linux, Microsoft Windows, Mac OS X, Sun Solaris, IBM AIX, HP UX, etc etc etc. Is platform agnostic really that critical? Just like in the on-premise world, enterprises would be wise to evaluate the cloud platforms they plan to use based on a predefined set of requirements. Also, is supporting multiple cloud platforms really a concern that will prevent enterprise adoption?
4. Reliability is still an issue.
Again, I agree that reliability is a concern. However, that’s a concern regardless of what you decide. You have to worry about reliability if you choose to go with your own data center or cloud. You have to worry about reliability if you choose to partner with a data center provider to hose your gears. You have to worry about reliability if you choose to go into the cloud. Heck, you have to worry about reliability even if you just host your gears in your own IT network.
Stacey said “Even inside an enterprise, data centers or servers go down, but generally the communication around such outages is better and in many cases, fail-over options exist.” I am sorry, but by definition, the cloud platforms usually have these capabilities built-in already. A single server or multiple servers failing is usually not going to affect your cloud applications or platforms.
I believe the real issue is service level agreement. Are the cloud providers providing adequate SLAs and do the CIOs feel comfortable with the SLA that they are getting?
5. Portability isn’t seamless.
No disagreement here. Currently there’s not an enterprise version of the data portability standard. That can turn many enterprises away if they have no way of retrieving/migrating their data if they choose to go with another provider.
6. It’s not environmentally sustainable.
Again, a good issue to raise. However, I would like to see some evidence to show that creating and maintaining your own data center is more efficient than going into the cloud. There will always be excess capacity in order to handle spikes, regardless whether you build your own data center or go into the cloud.
7. Cloud computing still has to exist on physical servers.
No disagreement that data locality is an important consideration when moving into the cloud. I wrote about it in a previous blog. However, that just means enterprises should be aware of this issue and make sure that’s part of their requirement for evaluating the cloud vendors. This however does not mean enterprises won’t adopt because of this concern.
8. The need for speed still reigns at some firms.
The increase in bandwidth to home and offices is one of the main reasons why clouds are hot these days. However, I agree with Stacey that speed is definitely a concern for certain types of applications. At CloudCamp, during Jeff Barr’s AWS feedback session, the first hot topic that came up was how do people move a HUGE amount of data into the cloud and back. People talked about shipping hard drives as a solution to this type of problem.
However, this is not going to be an issue for most enterprises in the US, UK and countries with adequate bandwidth. Take for example applications such as Salesforce.com CRM, NetSuite, and many others, these applications do not require the need to transfer large amount of data back and forth so they are ideal for delivering via the web.
So again, a valid concern, but not a show stopper.
9. Large companies already have an internal cloud.
Again, I would like to hear more evidence from Stacey to back this up. I agree that most enterprises already have IT infrastructure in place, but most of these infrastructures are not considered clouds. My conversations with enterprises, including discussion from CloudCamp, is that enterprise IT groups are stretched thin and they can’t respond fast enough to business requirements. When the business require certain applications to do their job, they have to go provision hardware, software, space, etc and that process can take months. Going with the cloud allows them to quickly react to the business requirements and makes it a win-win situation.
Even if enterprises have their internal clouds, does that mean that shouldn’t consider external clouds? Enterprises should, and will, always weigh the cost/benefits to determine what’s the right solution for them.
10. Bureaucracy will cause the transition to take longer than building replacement housing in New Orleans.
Agreed. In big companies that’s ways going to be the case. No one is suggesting that all enterprises will move into the cloud overnight. Many of the enterprises are just starting to experiment with the cloud to see what can and cannot be done. This is healthy and it’s the right approach. A good example is New York Times using Amazon EC2 to convert millions of articles and TIFF image into PDF files.
Again, enterprises are adopting the cloud, just cautiously.

Coté over at Redmonk just went nuts
and posted 5 articles on his views and observations of last week’s cloud conferences (CloudCamp, Structure 08, Velocity).