Deploying User Resources In Grid Environment

Recently I had to help a client who is using GridGain to process XML messages from external sources. The business case is that multiple external sources send XML messages that take a while to process and grid architecture was chosen to provide timely parsing/processing and ensure overall redundancy and automatic fail-over. XStream was chosen as a XML-to-object mapper.

The best thing about this solution was that even though XStream does need some initialization, with GridGain you are able to start up bare stand-alone remote grid nodes and deploy XStream resources onto remote nodes using Peer Class Loading automatically. This can be achieved with @GridUserResource injection in GridGain.

Here is how the code for the deployed XStream resource looks. Note the @GridUserResourceOnDeployed and @GridUserResourceOnUndeployed annotations that control the resource lifecycle on remote node.

GridXStreamResource {
    // XStream instance used for parsing.
    private XStream xstream = null;

    // Gets fully initialized instance of XStream.
    public XStream getXStream() {
        return xstream;
    }

    // Callback invoked once whenever a task 
    // is deployed on a processing grid node.
    @GridUserResourceOnDeployed
    private void onDeployed() {
        xstream = new XStream();

        // Initialize for processing certain XML bean classes.
        xstream.processAnnotations(MyXmlBean1.class);
        xstream.processAnnotations(MyXmlBean2.class);
    }

    // Destructs the resource.
    @GridUserResourceOnUndeployed
    private void onUndeployed() {
        // Give to GC.
        xstream = null;
    }
}

Below is the code for GridTask and GridJob that will automatically deploy all code including GridXStreamResource onto remote nodes.

public class XmlProcessingTask extends GridTaskSplitAdapter<byte[], Boolean> {
  @Override
  protected Collection split(int gridSize, byte[] payload) {
      // We only had to create one job per message without splitting it.
      return Collections.singletonList(new GridJobAdapter<byte[]>(payload) {
          // Inject XStream instance used for parsing messages.
          @GridUserResource
          private transient GridXstreamResource xstreamRsrc = null;

          public Serializable execute() {
              byte[] payload = getArgument();

              // Parse XML message.
              MyXmlBean1 bean = (MyXmlBean1)xstreamRsrc.getXStream().
                  fromXML(new ByteArrayInputStream(payload));

              // Add processing logic here.

              return true;
          }
      });
  }

  public Boolean reduce(List<GridJobResult> gridJobResults) {
      // Simply return result from remote job.
      return gridJobResults.get(0).getData();
  }
}

Here is the code that executes the above task on the grid:

// XML payload received from external source.
byte[] payload = ...;

GridTaskFuture<Boolean> future = grid.execute(XmlProcessingTask.class, payload);

// Wait for result.
boolean success = fugure.get();

Now, to run it, simply startup standalone GridGain nodes by executing gridgain.sh or gridgain.bat script that comes with installation and watch your code automatically deploy and execute on remote nodes.

Enjoy!