Cloud Computing

Please take the following 2-minute survey to help us understand your hadoop environment better.

[poll id="4"]

Today, the major use of cloud computing for enterprises are still in its infancy (heck the whole cloud computing space is in its infancy). Most enterprises use cloud computing for testing, development and other peripheral tasks. However, most, if any, are using the clouds for production use. This is fairly similar to the virtualization space, where early use of the virtualization technology are for testing and development. Ten years later, we are seeing more and more enterprises adopt virtualization for production use and virtualization has become main stream.

In the past month or so I have talked to a lot of people in the cloud computing and virtualization space. Many of these folk are working at/on startups that solves one of the many challenges for Enterprise cloud computing. What are these challenges? I have tried to summarize them here (in no particular order).

Data Governance

I’ve written extensively about the need for data governance in previous posts. In essence, enterprises have a ton of sensitive data that requires access monitoring and protection. Data (and information generated from the data) is the life blood of many enterprises, the loss of control will not be acceptable. Whole markets (read: DLP) are created to protect the enterprise data and information. On top of all that, enterprises must comply with many of the regulations that require data governance. By moving the data into the cloud, enterprise, for now, will lose some capabilities to govern their own data set. They would have to rely on the service providers to guarantee the safety of their data.

I hate to invoke the ILM acronym but much of data governance is about

• Creation and Receipt
• Distribution
• Use
• Maintenance
• Disposition

So who’s tackling this problem? As far as I know, nobody is and nobody really can except for the service providers themselves. It is really up to the service providers such as Amazon, Google and Salesforce to provide guarantees that customer data are safe and access to data are restricted and protected.

Manageability

There are some great IaaS/PaaS out there, including Amazon’s web services (S3, EC2, EBS, etc), Google’s App Engine, Salesforce’s Force.com, Joyent, etc. However, most of these are raw infrastructures and platforms that do not have great management capabilities. This is not unusual. Throughout computing history, raw capabilities will generally appear on the market first, then management of these raw capabilities become a differentiator when competition heats up. Just look at the blade server and virtualization spaces as these are great examples of that trend. The hypervisor was the key technology that enabled enterprise virtualization; however, that piece is now being given away (see VMware’s ESXi) and management capabilities becomes the main differentiator.

Cloud computing is no different. An example of missing management capabilities for cloud infrastructures is auto-scaling. Amazon EC2 claims to be elastic; however, it really means that it has the potential to be elastic. Amazon EC2 will not automatically scale your application as your server becomes heavily loaded. It is still up to the developer to manage that scalability problem.

So who’s tackling this problem? Many startups have recognized the need for management early on and have built management capabilities on top of the existing cloud infrastructure/platforms. RightScale is one of the early pioneers in this space. Their solution solves many of the management issues such as auto-scaling and load balancing.

Monitoring

Monitoring, whether is for performance or availability, is critical to any IT shop. We are not talking about just how much CPU or memory the machines are using. We are talking about performance of transactions and disk IO and others. CPU and memory usage are misleading most of the time in virtual environments. The only real measurement is how long your transactions are taking and how much latency there are. According to High Availability‘s article on latency:

Amazon found every 100ms of latency cost them 1% in sales. Google found an extra .5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition.

So who’s tackling this problem? Hypernic’s CloudStatus is one of the first to recognize this issue and developed a solution for it. They started with monitoring of Amazon’s web services, then recently added monitoring for Google App Engine. In addition, RightScale’s solution can also provide monitoring for the virtual machines under their management.

Reliability and Availability

I won’t beat the dead “Gmail down, EC2 down, etc down” horse here. But the truth of the matter is enterprises today cannot reasonably rely on the cloud infrastructures/platforms to run their business. There’s almost no SLAs provided by the cloud providers today. Even Jeff Barr from Amazon said that AWS only provides SLA for their S3 service. I haven’t researched the SLA issue so not sure how true that is. But if it’s true, I think this will be one of the biggest factor, if not the biggest factor, in enterprise adoption. Can you imagine enterprises signing up cloud computing contracts without SLAs clearly defined? It’s like going to host their business critical infrastructure in a data center that doesn’t have clearly defined SLA.

We all know that SLAs really doesn’t buy you much. In most cases, enterprises get refunded for the amount of time that the network was down. No SLA will cover business loss. However, as one of the CSOs I met said, it’s about risk transfer. As long as there’s a defined SLA on paper, when the network/site goes down, they can go after somebody. If there’s no SLA, it will be the CIO/CSO’s head that’s on the chopping block.

So who’s tackling this problem? Well, again, no one is today as far as I know. Maybe some startup will come up with clever idea to provide SLA as a third party vendor (read: cloud insurance.) Or maybe the cloud providers will grow/wake up and actually do something to encourage the enterprise adoption.

Virtualization Security

Security is a huge area that encompasses many different things, including the standard enterprise security policies on access control, activity monitoring, patch management, etc. On top of that, virtualization security is something that most enterprises are just starting to grasp but don’t fully understand. Many IT people still believe that the hypervisor and virtual machines are safe. Recent presentations from Blackhat has demonstrate that we shouldn’t sleep so tight at night. As IT shops get more educated on the virtualization security issues, it will become one of the factors they will consider when they move into the cloud. Access control and monitoring of the virtual infrastructure will be on top of their mind.

So who’s tackling this problem? There are quite a few startups like Reflex, Blue Lane and Catbird that are creating privileged VAs that claim to protect the VAs running on VMware’s ESX servers. However, ensure you do your research on the performance of these solutions first before adopting one of them. Other startups (unnamed) are creating interesting solutions in protecting the actual virtual infrastructure themselves, e.g., how do you protect and monitor access to the ESX servers? how do you control and monitor the movement of virtual machines using live migration or VMotion.

—

Cloud computing is here to stay. It will be the next big wave and will be adopted by enterprises. However, the industry as a whole needs to answer some of these challenges and ease the enterprises’ concerns.

Other interesting reads for Enterprise Cloud Computing are:

• EC3: Enterprise Class Cloud Computing
• CIOs On Cloud Computing
• Enterprise cloud computing gathers steam