Apache Ignite - In Memory Performance with Durability of Disk

Ignite is the in-memory computing platform
that is durable, strongly consistent, and highly available
with powerful SQL, key-value and processing APIs

Starting with 2.1 release, Apache Ignite has become one of a very few in-memory computing systems that provides its own distributed persistence layer. Essentially, users do not have to integrate Ignite with any type of 3^rd party databases (although such integration is supported), and start using Ignite as a primary storage of their data on disk and in memory.

So, what makes Ignite data storage unique? Let us look at a few important features provided by Ignite. You will probably notice that some of these features can also be seen in other data storage systems. However, it is the combination of these features in one cohesive platform that makes Ignite stand out among others. 

1. Durable Memory

Ignite durable memory component treats RAM not just as a caching layer, but as a complete fully-functional storage layer. This means that users can turn the persistence on and off as needed. If the persistence is off, then Ignite, just like always, can act as a distributed In-Memory Database or as an In-Memory Data Grid, depending whether you prefer to use SQL or key-value APIs. If the persistence is turned on, then Ignite becomes a distributed, horizontally scalable database that guarantees full data consistency and is resilient to full cluster failures. On top of that, the data is stored in off-heap memory so there are no GC pauses even on large data sets.

2. Complete SQL support

With the latest release, in addition to SQL querying, Ignite added support for DDL and DML, allowing users to interact with Ignite using pure SQL without writing any code. This means that users can create tables and indexes, insert, update, and query data using only SQL. Having such complete SQL support makes Ignite a one-of-a-kind distributed SQL database.

3. ACID compliance

Data stored in Ignite is ACID-compliant both in memory and on disk, making Ignite a strongly consistent system. Ignite transactions work across the network and can span multiple servers. This makes Ignite stand out from the eventually consistent NoSQL systems that hardly support any type of transactions.

4. Collocated Processing

Most traditional SQL and NoSQL databases work in a client-server fashion, meaning that data must be brought to the client side for processing. This approach requires lots of data movement from servers to clients and generally does not scale.  Ignite, on the other hand, allows for sending computations to the data, moving only the light weight compute functions across the network. As a result, Ignite scales better and minimizes data movement. When collocated, all the data processing happens locally on the node that stores the data, and only the result is brought back to the user.

5. Scalability and Durability

Ignite is an elastic, horizontally scalable distributed system that supports adding and removing cluster nodes on demand. Ignite also allows for storing multiple copies of the data, making it resilient to partial cluster failures. If the persistence is enabled, then data stored in Ignite will also survive full cluster failures. Cluster restarts in Ignite can be very fast, as the data becomes operational instantaneously directly from disk. As a result the data does not need to be preloaded in memory to begin processing, and Ignite caches will lazily warm up resuming the in memory performance.