1. Did you ever wish you could take a data structure you are familiar with and distribute it over grid? For example, why not take java.util.concurrent.BlockingDeque and add something to it on one node and poll it from another node? Or why not have a distributed primary key generator which would guarantee uniqueness on all nodes? Or how about a distributed java.util.concurrent.atomic.AtomicLong which can be updated and read from any node on the grid? GridGain gives you such capability. What GridGain did is actually take most of the data structures from java.util.concurrent framework and made sure they could be used in distributed fashion.

    In this blog I want to show how flexible GridGain distributed queues are. On top implementing java.util.Collection interface and supporting different modes of operation, like collocated vs. non-collocated, or bounded vs. ubounded modes, you can actually control how elements are ordered within queues. GridGain supports FIFO, LIFO, and Priority based queues out of the box.

    FIFO queues (first-in-first-out) are the most traditional queues where elements are added from the tail and polled form the queue head. LIFO queues (last-in-first-out) resemble more of stack features instead of queues. In LIFO queues elements are added and polled from the tail.

    But the most interesting queue type is Priority queue where user can control the order of the elements. Priority queues order elements within the queue based on priority attribute specified by the user. Priority attribute of a queue element is annotated via @GridCacheQueuePriority annotation. Here is an example of how priority queue can be created and used.
    public void priorityQueueExample() {
        Random rand = new Random();
    
        Grid grid = G.grid();
    
        // Initialize new unbounded collocated priority queue.
        GridCacheQueue<PriorityItem> queue = 
            grid.cache().queue("myqueue", PRIORITY);
    
        // Store 20 elements in queue with random priority.
        for (int i = 0; i < 20; i++) {
            int priority = rand.nextInt(20);
    
            queue.put(new PriorityItem(priority, "somedata-" + i));
        }
    
        PriorityItem item = null;
    
        int lastPriority = 0;
    
        do {
            item = queue.poll();
    
            // Ensure the elements are correctly ordered based on priority.
            assert lastPriority <= item.priority();
    
            lastPriority = item.priority();
        }
        while (item != null);
    }
    
    ...
    
    // Class defining sample queue element with its priority specified via
    // @GridCacheQueuePriority annotation attached to priority field.
    private static class PriorityItem implements Serializable {
        // Priority of queue item.
        @GridCacheQueuePriority
        private final int priority;
    
        private final String data;
    
        private SampleItem(int priority, String data) {
            this.priority = priority;
            this.data = data;
        }
    
        public int priority() {
            return priority;
        }
    }
    

    Read more about GridGain queues here.
    1

    View comments

  2. When working in distributed environment often you need to have a consistent local state per grid node that is reused between various job executions. For example, what if multiple jobs require database connection pool for their execution - how do they get this connection pool to be initialized once and then reused by all jobs running on the same grid node? Essentially you can think about it as a per-grid-node singleton service, but the idea is not limited to services only, it can be just a regular Java bean that holds some state to be shared by all jobs running on the same grid node.

    In GridGain versions 2.x and earlier this approach was handled by using @GridUserResource annotation to annotate fields within GridTask or GridJob classes to specify singleton beans. However, this approach was dependent on GridDeploymentMode configuration and, for ISOLATED or PRIVATE deployment modes, resource could be initialized multiple times, once per GridTask. This forced users to use various hacks in their logic and generally was not very convenient to use.

    Starting with GridGain 3.0, GridNodeLocal per-grid-node local storage was introduced. The name was borrowed from ThreadLocal class in Java, because just like ThreadLocal provides unique per-thread space in Java, GridNodeLocal provides unique per-grid-node space in GridGain. GridNodeLocal implements java.util.concurrent.ConcurrentMap interface and is absolutely lock-free. In fact, it simply extends java.util.concurrent.ConcurrentHashMap implementation and, therefore, inherits all the methods available there.

    Here is an example of how GridNodeLocal could be used to create some user specific singleton service from a simple GridGain job:
    final Grid grid = G.start(..);
    
    // Execute runnable job on some remote grid node.
    grid.run(BALANCE, new Runnable() {
      public void run() {
        GridNodeLocal<String, MySingleton> nodeLocal = grid.nodeLocal();
    
        // 1. Check if f someone already created our singleton service. 
        MySingleton service = nodeLocal.get("myservice");
    
        if (service == null) {
          // 2. Create new singleton service.
          MySingleton other = 
            nodeLocal.putIfAbsent("myservice", service = new MySingleton(..));
    
          if (other != null) 
            service = other;
        }
    
        // Perform operations with our singleton service.
        ...
      }
    });
    0

    Add a comment

About me
About me
- Antoine de Saint-Exupery -
- Antoine de Saint-Exupery -
"A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away."
Blog Archive
Blogs I frequent
Loading
Dynamic Views theme. Powered by Blogger.