Friday 28 February 2014

Playing with redis-cluster for non-linux-gods, step by step

 

Redis-cluster is now available in in beta, and I do love me some redis. There’s a great tutorial available for working with redis cluster, but I found it a bit fiddly to get it working: I’m normally a Windows person - we use CentOS in production, but we have an excellent sysadmin team who know how to make linux work, but that doesn’t help me so much for playing with stuff as a developer. Hence thought I’d write up the step-by-step instructions to get it working if you’re not a linux god.

As an aside for Windows developers

For a lot of local development purposes, the “Microsoft Open Technologies, Inc” port of redis is available on nuget, which makes it really easy to run. I usually just:

PM> Install-Package Redis-64

to install it into a Visual Studio solution (via the Package Manager Console), and then create two solution-level files: redis-cli.cmd and redis-server.cmd, with (respectively)

@packages\Redis-64.2.6.12.1\tools\redis-server.exe
@packages\Redis-64.2.6.12.1\tools\redis-cli.exe

This makes starting a redis server or console just a "right-click, Run" away. However! At the time of writing the nuget binary is based on 2.6.12, we need 3.0 for cluster (technically 2.9.50, but close enough).

Installing your server OS

For the purposes of this tutorial, I’m starting from a vanilla Ubuntu Server 13.10, installed into Hyper-V by mounting the iso; for VM configuration, I’m using 4GB memory, 2 CPUs, 127GB disk, and access to the external network (we’ll need files). For the OS install, I’ve simply used all the default options (except obviously my location and user details, and I enabled automatic security updates), and let it do its thing. This can take a little while (quite a bit longer than Ubuntu Desktop, but it seems a preferable choice). A long wait, a reboot and a login later, you should see your server OS:

image

Working on the server

You can work directly in the VM console if you like, but I find it awkward because copy and paste doesn’t work, so I’m going to use putty on my regular desktop to talk to the server. First we need to install a ssh server (for more complete configuration options and recommendations, see help.ubuntu.com):

$ sudo apt-get install openssh-server

We will also need our server’s ip address:

$ ip addr

which should provide a 192.* address – for me 192.168.0.15.

Now we can switch to putty and login (by entering the ip address as the host):

image

Note that the first time you log in you will get a security warning about the certificate – this is expected and is because openssh has been installed with a newly created certificate that we don’t yet trust. We know that this is our server, so we can just say yes, and proceed to enter our username and password – so now we’re logged in via a putty terminal:

image

Tip: right-clicking pastes whatever is in the clipboard!

Getting a build environment

Ubuntu Server doesn’t start with a lot installed; we’ll need a few tools…

$ sudo apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion pkg-config

(I bet you’re glad of that working copy/paste about now, eh?)

Installing Redis

The package manager versions of redis are universally ancient, so the recommended option is direct install. The overall instructions are on the download page, but we need to replace the stable link with the beta link (which is the Download link on the 3.0.0 Beta line, currently). So:

$ wget https://github.com/antirez/redis/archive/3.0.0-beta1.tar.gz
$ tar xzf 3.0.0-beta1.tar.gz
$ cd redis-3.0.0-beta1 $ sudo make install

At this point we now have a usable redis-server, yay! You can test this trivially by starting a default server:

$ redis-server

image

and then in another console session (or from your Windows host if you have redis-cli available – the version of redis-cli on nuget is fine for this):

$ redis-cli –h 192.168.0.15 PING

to which the server will reply, rather unoriginally, PONG. You can use ctrl+c back in the other session to stop the redis-server. Note that you don’t usually run server-based redis-server sessions directly like that, but it is a handy smoke test.

Getting redis-trib.rb Working

The most convenient way to configure a redis cluster is with the redis-trib.rb tool. Ubuntu Server doesn’t ship with the necessary Ruby tools installed, so we’ll need to install them. The easiest way to configure Ruby is with rvm. Now, you might think “apt-get install rvm”, but DO NOT DO THIS. Here’s the joy I snagged from a previous Ubuntu install (hence the slightly different appearance):

image

Which takes us to this very helpful stackoverflow.com answer. Fortunately, we can avoid that pain by getting it right first time:

$ sudo curl -L https://get.rvm.io | bash -s stable --ruby --autolibs=enable --auto-dotfiles
$ gem install redis

Now redis-trib.rb should work:

$ cd src
$ ./redis-trib.rb

image

Now you should be fully setup and configured, ready to follow the tutorial. Note that you probably want to use the non-loopback ip when configuring the cluster, so that external clients don't get confused by MOVED replies:

$ ./redis-trib.rb create --replicas 1 192.168.0.15:7000 192.168.0.15:7001 192.168.0.15:7002 192.168.0.15:7003 192.168.0.15:7004 192.168.0.15:7005

Transferring Configuration Files

If, like me, you don’t usually use terminal text editors, you may find it easier to transfer configuration files (discussed more in the tutorial; note I’ve added “daemonize yes”, which makes it run as in the background rather than blocking the console) from your host, for example:

image

The putty package includes the pscp utility for this; first you need to know your full directory on the linux server:

$ pwd

And then you can recursively copy the files from your host setup to the server:

pscp -r d:\cluster-test marc@192.168.0.15:/home/marc/redis-3.0.0-beta1

which should show the multiple files being copied:

image

And now, continuing to follow the tutorial, we should be able to start the 6 nodes successfully:

image

Outro

Hopefully you should have everything here that you need to get a working install so you can follow the tutorial. The same steps should also be helpful to anyone wanting to configure a vagrant setup. Enjoy.