GGBLOG - In The Clouds

Today I'd like to talk about EC2 (elastic compute cloud) which is Amazons implementation of a compute cloud which really just a bunch of computers linked together (a cluster).

Where things start to get interesting over other plain old compute clusters is the E (elastic) part.

Amazon gives you a nice set of tools to bring computing nodes online on demand which is why they call it elastic.

I remember back in my days as a web developer when the 911 attacks occurred, even though we had 12 servers in our cluster to do static content serving, there was such a spike in load that the poor operations guy had to rush down to our hardware vendor and pick up some new machines and plug them into the cluster. This whole process took hours, It would have been pretty nice to have EC2 back then to bring the nodes online through a nifty little firefox extension.

There is alot of energy around grid based frameworks like Grid Gain, Hadoop, Coherence and Gigaspaces. These frameworks come into play by allowing you to take advantage of EC2 by allowing you to scale your applications up and out and providing linear scalability.

What does this mean?

Most applications out there are built on tiered and layered architectures. As a example (see pic below) a j2ee app is typically has a web browser, webserver, app server, integration server then back end server. Thats an awful lot of services/servers to go through to get any work done.

























In my experience the biggest problem to this architecture that there is always a glass ceiling when it comes to the scalability of these solutions, eg. the database or sap runs out of capacity

Grid based frameworks attempt to turn this on its head by allowing you break your applications into a structure which fully supports parallel processing.