Thursday, April 15, 2021

System design: Redis | Distributed cache | Data Access Architecture | High availability |Memory Architecture in Redis Enterprise Software

April 15, 2021

Here is the article. 

Data Access Architecture 

    Consistency and Durability 

    Redis Enterprise Software (RS) comes with the ability to replicate data to another slave for high availability and persist in-memory data on disk permanently for durability. With the WAIT command, you can control the consistency and durability guarantees for the replicated and persisted database in RS. Any updates that are issued to the database are typically performed with the following flow shown below; Application issues a write, Proxy communicates with the correct master “shard” in the system that contains the given key, The acknowledgment is sent to proxy once the write operation completes The proxy sends the acknowledgment back to the application.

    Database Persistence with Redis Enterprise Software 

    All data is stored and managed exclusively in either RAM or RAM + Flash Memory (Redis on Flash) and therefore, is at risk of being lost upon a process or server failure. As Redis Enterprise Software is not just a caching solution, but also a full-fledged database, persistence to disk is critical. Therefore, Redis Enterprise Software supports persisting data to disk on a per-database basis and in multiple ways. Persistence can be configured either at time of database creation or by editing an existing database’s configuration.

    Discovery Service 

    The Discovery Service provides an IP-based connection management service used when connecting to Redis Enterprise Software databases. When used in conjunction with Redis Enterprise Software’s other high availability features, the Discovery Service assists an application cope with topology changes such as adding, removing of nodes, node failovers and so on. It does this by providing your application with the ability to easily discover which node hosts the database endpoint. The API used for discovery service is compliant with the Redis Sentinel API.

    Redis OSS Cluster API Architecture 

    Redis OSS Cluster API reduces access times and latency with near-linear scalability. The Redis OSS Cluster API provides a simple mechanism for Redis clients to know the cluster topology. Clients must first connect to the master node to get the cluster topology, and then they connect directly to the Redis proxy on each node that hosts a master shard. Note: You must use a client that supports the OSS cluster API to connect to a database that has the OSS cluster API enabled.

    High Availability 

      Database clustering 

      Open source Redis is a single-threaded process to provide speed and simplicity. A single Redis process is bound by the CPU core that it is running on and available memory on the server. Redis Enterprise Software (RS) supports database clustering to allow customers to spread the load of a Redis process over multiple cores and the RAM of multiple servers. A database cluster is a set of Redis processes where each process manages a subset of the database keyspace.

      Database replication 

      Database replication provides a mechanism to ensure high availability. When replication is enabled, your dataset is replicated to a slave shard, which is constantly synchronized with the master shard. If the master shard fails, an automatic failover happens and the slave shard is promoted to be the new master shard. When the old master shard recovers, it becomes the slave shard of the new master shard. This auto-failover mechanism guarantees that data is served with minimal to no interruption.

      Rack-zone awareness in Redis Enterprise Software 

      Rack-zone awareness is a Redis Enterprise feature that helps to ensure high-availability in the event of a rack or zone failure. When you enable rack-zone awareness in a Redis Enterprise Software (RS) cluster, you assign a rack-zone ID to each node. This ID is used to map the node to a physical rack or logical zone. The cluster can then ensure that master shards, corresponding slave shards, and associated endpoints are placed on nodes in different racks/zones.

    Memory Architecture in Redis Enterprise Software 

    Redis Enterprise Software has multiple memory mechanisms in its architecture, from RAM to Flash Memory, to having databases that span them both.

      Memory Management with Redis Enterprise Software 

      Redis Enterprise Software (RS) manages node memory so that data is entirely in RAM for improved database performance. RS is designed to handle memory management to optimize database performance - better than OS memory management. If not enough RAM is available, RS prevents adding more data into the databases. RS protects the existing data and prevents the database from being able to store data into the shards. You can configure the cluster to move the data to another node, or even discard it according to the eviction policy set on each database by the administrator.

      Redis on Flash 

      Redis on Flash (RoF) offers users of Redis Enterprise Software and Redis Enterprise Cloud the unique ability to have large Redis databases but at significant cost savings. Where standard Redis databases must all be in RAM, Redis on Flash enables your Redis databases to span both RAM and dedicated flash memory (SSD). Whilst keys are always stored in RAM, RoF intelligently manages the location of their values (RAM vs Flash) in the database via a LRU-based (least-recently-used) mechanism.

No comments:

Post a Comment