Stack Exchange Redis Connection Pooling

Summary

The Stack Exchange Redis client for C# is fast and reliable.  Unfortunately, it’s difficult to find quality information on the details of this client.  In this blog post I’m going to demonstrate how to setup a Redis connection in C# using the Stack Exchange Redis client.  This connection technique will reuse connections instead of constantly creating new connections for each thread communicating with the Redis server.

My Test Setup

I’m using Redis for windows, which you can download from here:  
https://github.com/ServiceStack/redis-config/tree/master/bin/windows.

I just downloaded a zip copy of the program, then installed it as a service on my PC, using this command (from inside the bin/windows directory of the unzipped redis source): 

redis-server --service-install

Notice that there are two minus signs in front of the “service-install” command switch.  Then I startup up the windows service manager and clicked on the “Start” button.  The windows service name is “redis” (as expected). 

The Test Program

I setup a test program consisting of a parallel for loop that will simulate a system that uses asynchronous processes.  An example would be a website where each connection would read from Redis to retrieve cached data.  Your first step is to use NuGet to install the StackExchange Redis client.

My first connection object looks like this: 

public class RedisConnection
{
    private static ConnectionMultiplexer redis;
    private static IDatabase db;
    public static IDatabase RedisServerDB
    {
        get
        {
            if (db == null)
            {
                redis = ConnectionMultiplexer.Connect("localhost:6379");
                db = redis.GetDatabase();
            }
            return db;
        }
    }
}

I used a lazy loading scheme to connect to the redis server if the connection is not already created.  Then I called this from this code: 

static void Main(string[] args)
{
    for (int i = 0; i < 100000; i++)
    {
        string[] redisKeyList = new string[] {
            "a", "b", "c", "d", "e", "f", "g", "h", "i",
            "j", "k", "l", "m", "n", "o", "p", "q", "r",
            "s", "t", "u", "v", "w", "x", "y", "z" };
        Parallel.ForEach(redisKeyList, currentKey =>
        {
            RedisConnection.RedisServerDB.StringSet(currentKey, "some data to store at this location in redis");
            string ignoreMe = RedisConnection.RedisServerDB.StringGet(currentKey);
        });
    }
}

Before you run this program, start a command window and use the following netstat command to see the number of concurrent connections: 

netstat -n | find "ESTABLISHED" | find "6379"

Now run the program and run the above command in the command window a few times.  You’ll see the number of connections increase until there are two per parallel process in progress (one outbound and one inbound connection)

One issue with this method is that IIS will keep connections open for a few seconds before clearing the connection.  As web traffic increases, the number of connections will increase and tie up resources on your web server

Many articles explain that Stack Exchange Redis does not have a connection pooling capability.  However, there is a work-around for this problem and it’s simple and involves the “Lazy” keyword.  Here’s an example of using the lazy load method 

public static class RedisConnectionPool
{
    private static Lazy<ConnectionMultiplexer> Connection = new Lazy<ConnectionMultiplexer>(
            () => ConnectionMultiplexer.Connect("localhost:6379"));
    public static IDatabase RedisServerDB
    {
        get
        {
            return Connection.Value.GetDatabase();
        }
    }
}

Change the “RedisConnection” calls in the main code above into “RedisConnectionPool” and run the program.  Now execute your netstat command and look at the number of connections.  There should be 4: 

TCP    127.0.0.1:6379         127.0.0.1:51835        ESTABLISHED
TCP    127.0.0.1:6379         127.0.0.1:51836        ESTABLISHED
TCP    127.0.0.1:51835        127.0.0.1:6379         ESTABLISHED
TCP    127.0.0.1:51836        127.0.0.1:6379         ESTABLISHED

Hat tip and more information of this technique, go here:
http://gigi.nullneuron.net/gigilabs/setting-up-a-connection-with-stackexchange-redis/

Where to get the Sample Code

As normal, I have posted the test code used for this blog post on my GitHub account.  You can click here to download it.

Leave a Reply