An easy-to-manage network cluster is a cluster in which all nodes are equal and can be brought up with identical configuration. However, even though all nodes are equal, it often still makes sense to assign application-specific roles to them, like "workers', "clients", or "data-nodes". In Apache Ignite, this concept is called cluster groups.
Apache Ignite is an In-Memory Data Fabric composed of multiple distributed components with Clustering APIs serving as the main backbone for the rest of the components, including Data Grid, Compute Grid, and Service Grid. I am one of the committers to this project and generally blog quite a bit about it.
You can create virtual cluster groups in Ignite based on any application-specific custom filter. However, to make things easier, Ignite comes with some predefined filters.
Select Remote Nodes
Here is how you can execute a simple closure on all remote nodes. Remote nodes include all cluster members, except for the member who is starting the execution.
final Ignite ignite = Ignition.ignite();
IgniteCluster cluster = ignite.cluster();
// Get compute instance which will only execute
// over remote nodes, i.e. not this node.
IgniteCompute compute = ignite.compute(cluster.forRemotes());
// Broadcast to all remote nodes and print the ID of the node
// on which this closure is executing.
compute.broadcast(() -> System.out.println("Hello Node: " + cluster.localNode().id());
Select Worker Nodes
You can assign application specific roles to cluster members, like "masters" and "workers", for example. This can be done via user attributes specified on node startup. For example, here is how you can bring up a cluster node with "ROLE" attribute set to "worker":
IgniteConfiguration cfg = new IgniteConfiguration();
Map<String,String> attrs = Collections.singletonMap("ROLE", "worker");
cfg.setUserAttributes(attrs);
// Start Ignite node.
Ignite ignite = Ignition.start(cfg);
Then here is how you would execute a closure only over nodes with role "worker":
IgniteCluster cluster = ignite.cluster();
// Get compute instance which will only execute over "worker" nodes.
IgniteCompute compute = ignite.compute(cluster.forAttribute("ROLE", "worker"));
// Broadcast to all "worker" nodes and print the ID of the node
// on which this closure is executing.
compute.broadcast(() -> System.out.println("Hello Node: " + cluster.localNode().id());
Custom Cluster Groups
And finally, you can create custom cluster groups based on any user-defined predicates. Such cluster groups will always only include the nodes that pass the predicate. For example, here is how we would create a group of cluster nodes that have CPU utilization less than 50%:// Nodes with less than 50% CPU load.
ClusterGroup readyNodes = cluster.forPredicate((node) -> node.metrics().getCurrentCpuLoad() < 0.5);
// Broadcast to all nodes with CPU load less than 50% and
// print the ID of the node on which this closure is executing.
compute.broadcast(() -> System.out.println("Hello Node: " + cluster.localNode().id());
For more on cluster groups, visit Ignite Cluster Groups documentation.
I ran into Apache Ignite last night, over casual browsing. How does this compare to Mesos?
ReplyDeleteHi Dmitriy,
ReplyDeleteIf I want to perform some operations synchronously while some asynchronously on the same cache, what is the recommended way to do so?
Thanks
Ignite has both, synchronous and asynchronous APIs for every distributed operation. Feel free to use either one.
DeleteHi Dmitriy,
DeleteCan I run ignite with two master nodes within a cluster, where if there is network glitch between these nodes they still keep serving(add/update/delete) data independently also sync back when network is fine?
I really need your help if we can do this ?
This is not handled in Ignite, but GridGain has data center replication which behaves just the way you have described. More info here: https://www.gridgain.com/products/software/enterprise-edition/data-center-replication
Delete