Friday, November 13, 2009

Lower Your Electricity Bill With GridGain

Recently, when talking to a company that creates consumer electronics, an interesting idea popped up. This company is really struggling in minimizing their electricity bill and I was asked if GridGain could help. Interestingly enough, GridGain can help. Let's suppose you have two data centers, A and B, and data center A has low electricity costs between 12am and 12pm and data center B has low costs between 12pm and 12am. All you need to do to make sure that either data center is used during low cost time window is create GridLowElectricityTopologySpi as follows:


@GridSpiInfo(author = "Your Name", version = "1.0")
@GridSpiMultipleInstancesSupport(true)
public class GridLowElectricityTopologySpi
extends GridSpiAdapter
implements GridTopologySpi {
/** Auto-inject logger. */
@GridLoggerResource
private GridLogger log = null;

/**
* Start SPI.
*/
public void spiStart(String gridName) {
if (log.isInfoEnabled() == true) {
log.info(startInfo());
}
}

/**
* Stop SPI.
*/
public void spiStop() {
if (log.isInfoEnabled() == true) {
log.info(stopInfo());
}
}

/**
* Given whole grid topology return only nodes
* selected for execution of this task.
*/
public Collection<GridNode> getTopology(
GridTaskSession ses,
Collection<GridNode> grid) {
int hour = Calendar.getInstance(Calendar.HOUR_OF_DAY);

List<GridNode> top = new ArrayList<GridNode>(grid.size());

for (GridNode node : grid) {
String segment = node.getAttribute("segment");

if (hour <= 12) {
if (segment.equals("A")) {
top.add(node);
}
}
else if (segment.equals("B")) {
top.add(node);
}
}

return top;
}
}

Now, the nodes from data center A need to have the following in their configuration (the configuration for data center B is analogous).

<bean
id="grid.cfg"
class="org.gridgain.grid.GridConfigurationAdapter"
scope="singleton">
...
<property name="userAttributes">
<map>
<entry key="segment" value="A"/>
</map>
</property>
...
</bean>

This is all that needs to be done to lower your electric bill. Now all tasks will be routed to the data center with lowest electricity cost.

Enjoy!

 

Friday, November 6, 2009

GridGain Is Presenting at Dallas JUG, November 11

GridGain will be presenting at Dallas Java User Group at Dallas, TX, on Wednesday, November 11th. If you are around the Dallas area and interested in Cloud Computing - come see the presentation. As usual we'll have lively discussion on clouds and grids and as always plenty of live coding that you won't see anywhere else - live cloud application in 10 minutes from scratch!. We also going to show some sneak peek preview of GridGain 3.0, including live Cloud Demos.

Wednesday, September 23, 2009

Endless Conversations with Cucumber

There is a new kid on the block, Behavior Driven Development, or BDD, so naturally I rushed to check it out. I immediately came across Cucumber, a BDD testing framework for Ruby. The Cucumber site is fairly informative and you pretty much immediately understand that Cucumber is about writing "Given->When->Then" test cases (or as they are called Scenarios). Unlike other test frameworks, developer can really "talk" to Cucumber. You can start implementing a simple use case as follows:

Given that a user arrives to any page
When no session exists
Then user is redirected to Login page
Sounds pretty straight forward - one of the pros of using Cucumber is that your test scenarios are pretty much self-documenting. Once you write this scenario, you start executing it in Cucumber which will keep failing until it succeeds. When you will run it the first time, Cucumber will not understand any of this. As you keep going, you will have to define "that a user arrives to" in Cucumber in order for the first step to pass. Then you will have to define "no session exists", and then "user is redirected to". So gradually you will be explaining to Cucumber what it is that your test case is doing.

My initial impression was that this is way too cool. Looking from the outside, you can basically read the code in plain English which simplifies readability and understanding of what it is you are testing. But once I started looking at it a bit deeper, it occurred to me that testing this way on a serious project can be really dangerous. Here is why.

Cucumber Imposes Endless DSLs
This comment may go towards BDD in general. Basically Cucumber allows you to create an endless Domain Specific Language (DSL) for your tests. Any new or existing developer can just "speak" his mind to Cucumber which will inevitably lead to more and more new "key phrases" for the overall DSL. In my view this may significantly hinder reusability of test cases and pretty soon you may end up in a position where you have thousands upon thousands of unmaintainable Cucumber scenarios. If you look at GridGain test cases for example (and we have about a thousand of test cases), we managed to keep them very concise only because they are mostly reusing existing test components available to them. One may argue that you can do the same in Cucumber, but this would be against overall Cucumber philosophy in my view.

Enterprises Need Structure
Yes, enterprise projects need structure and developers need to be constrained by available libraries and APIs. You cannot just let every developer go willy-nilly by allowing him to "talk freely" to the underlying testing framework. I have nothing against a specific DSL for testing (on the contrary, I would encourage it), but this DSL should be well defined and used by everyone in the company.

I should mention, however, that on a relatively small project, testing the Cucumber way may be very productive. Since small projects are exactly the sweet spot for Ruby, Cucumber may very well be the best testing framework available for Ruby. But as the project will grow, using Cucumber will become more and more painful and unmaintainable.

So, here is the test scenario for GridGain that you could create in Cucumber (forgive the sarcasm, but I couldn't resist):
Given that GridGain is a large enterprise project
When I write test scenarios in Cucumber
Then I will shoot myself in the foot

P.S. I realize that this post may stir some disapproval from Cucumber community. In the interest of full disclosure I want to mention that I have not used Cucumber on a real project and simply am expressing my opinion here which is not based on actual project experience.

 

Monday, August 31, 2009

GridGain - Simplicity and Developer Usablity

I just came across a pretty old blog about GridGain where a user explained his experiences with GridGain and had a brief Q&A via comments.

When asked about difference between GridGain and Globus I think he gave a spot-on answer:

"No, I’ve never worked with the Globus Toolkit. From what I’ve read, Globus is more about collaboration of organizations (forming "virtual organizations") and solving all the problems that come with it like interoperability and security concerns.

GridGain on the other hand has a strong focus on simplicity and developer usability. I think it’s perfect for "local" grids with perhaps a few dozens (or hundreds?) of nodes. You can certainly build a more comprehensive solution on top of GridGain, but the local computational grid use case is where it is best suited at the moment.

In my concrete case at work we’ve got a highly secured internal network and need a convenient solution for distributing the workload to boost throughput. Globus looks like overkill, but I’m confident that GridGain is the right thing for us."
Although this conversation took place about 2 years ago and GridGain had since evolved into a Cloud Development Platform with aplenty of new features, our (quoting) "strong focus on simplicity and developer usability" has not changed. Even with such features in the pipeline as data grid and hands-free self-scaling on the clouds, GridGain developers will still enjoy the same simple deployment-less process:
Change -> Compile -> Run On Cloud
Take a look at some of our Online Screen Casts and find out for yourselves.

Friday, August 21, 2009

What is a Native Cloud Application

With the latest push towards the cloud in the industry and some M&A actions in cloud area we are beginning to be faced more and more with a simple question - what does it mean to build an application for a cloud? At GridGain we strongly believe that for an application to take a real advantage of the cloud it should have some of the following characteristcs:

  • Massively Parallel Computations
  • Massively Parallel Data
  • Auto-Scalability On the Cloud
I like to call applications with such characteristics Native Cloud Applications, or NCAs. With our upcoming GridGain 3.0 release we are planning to build a Cloud Development Platform that will allow you easily build and deploy NCAs on any of the existing cloud providers. So what do the above mentioned characteristics really mean?

Massively Parallel Computation
An application that supports massively Parallel Computation is a application that can simply distribute computations in an environment such as cloud, where resources can be added or removed at any moment either manually or automatically. If you add a resource, your application has to take immediate advantage of it and start distributing computations onto it (the vice versa should apply whenever you remove a resource). In GridGain we achieve massively parallel computations with our innovative MapReduce implementation, Automatic Peer Class loading, and Adapter Discovery and Communication SPIs.

Massively Parallel Data
Well, you may think that most clouds already have distributed storage, so you can just use it. The reality here is that storage like S3 on Amazon provides a distributed disk storage. It does not provide distributed In-Memory storage like a distributed cache or data grid. Google GAE does provide it to a certain extent, but as a cloud, it is less flexible to use than Amazon. So, in order to take a full advantage of the cloud your application In-Memory storage must dynamically increase whenever new resources get added and vise versa. GridGain 3.0 is coming with a comprehensive dynamically partitioned data grid solution that does just that.

Auto-Scalability On The Cloud
A NCA must be able to automatically instruct the cloud to scale up or scale down depending on current load and latency characteristics. In GridGain 3.0 user will be able to specify dynamic SLAs or strategies that will allow to add any kind of resource to a cloud, from addtional memory or CPU to starting or removing an arbitrary number of images.

Although we are adding such a complex and comprehensive feature set, our emphasis on ease of use and ease of deployment has not changed. Stay tuned for our upcoming release.

Monday, August 17, 2009

GridGain presents at Chicago Java User Group, August 18th

GridGain will be presenting at CJUG - Chicago Java User Group at Chicago, IL, on Tuesday August 18th. If you are around the Chicago area and interesting in Cloud Computing - come see our presentation. As usual we'll have lively discussion on clouds and grids, get a preview on our upcoming GridGain 3.0 release, and as always plenty of live coding that you won't see anywhere else - live grid application in 10 minutes from scratch!

Hope to see you there!

Tuesday, August 4, 2009

GridGain Is Growing And Growing Strong

Last month was a very good month for GridGain as we achieved a few important milestones. First and foremost, we crossed a threshold of 1 million starts. Since GridGain product was first launched 2 years ago (around August 2007), GridGain nodes were started over a million times across the globe. This is a very conservative number... I am sure the real number is at least 30% higher. Here is a graph of GridGain starts since the beginning:

The vertical axis has number of starts, and the horizontal axis shows time in months. According to these stats, a GridGain node is started at least every 30 seconds somewhere around the globe.

Another pretty cool stat is our forums usage. We have arguably the most active user community among all the products in the same space. Our forums have over 3,000 messages and over 80,000 views which is very good for a product in this space.

An lastly, there is no bigger reward for any Open Source contributor than kind user testimonials. Here is a few of the latest testimonials we received from our users and customers:

Thanks a lot for your help and for the great product. I'm starting to use Gridgain for one of our production systems and up to this point I had only joy! Keep up the good work.
--jjoubert
I am a big fan of GridGain. We at [...] implemented our solutions with GridGain capability. It is very impressive to my clients whenever we did presentations. Excellent products which helps us solving big problem for us. [...] offers software on SaaS market space which requires huge computing power when number of clients increase. GridGain provides the answer to our problem.
--Charles Hsiao

Thanks for an amazing product. I had considered several others, but GridGain is the most intuitive and well-documented.
--Andrew Hallock
You can find more user testimonials here. It's the usage stats and testimonials like this that keep us motivated to continually make GridGain the best Open Cloud Platform out there.