One of Redis’s best unsung features is its wire protocol. It’s the reason that Redis has one of the largest ecosystems of high-quality client libraries. The Redis wire protocol is remarkably simple, which makes it easy to build a client that implements all of Redis’s major features. It’s also designed in a way that makes it easy to write fast and efficient client libraries.
RESP (REdis Serialization Protocol) is the name of the text-based protocol that Redis clients and servers use to communicate with each other over TCP. All communication between server and client consists of five basic types:
When writing and reading RESP, we don’t have to do any escaping or encoding of our keys or values. The only part of RESP we need to actually parse is the simple metadata included with each RESP object.
All RESP objects begin with a prefix character and end with a line terminator
(except arrays, which don’t include their own line terminator). The simplest
example of a RESP object is the OK simple string response:
+OK\r\n(RESP is a human-readable protocol but for clarity’s sake I’ll explicitly write
out out the line breaks (\r\n) in all RESP examples.)
In the above simple string, + is the simple string prefix, OK is the body of
the simple string, and \r\n is the line terminator that marks the end of this
simple string.
To read a simple string, we read up to the next \r\n line
terminator, returning the preceding bytes (up to +) as the returned reply
string. This works because RESP simple strings cannot include
any newline characters.
Errors and integers are formatted similarly to simple strings, but they use
different prefixes. Errors are prefixed with -:
-ERR unknown command 'GETT'\r\nAnd integers are prefixed with ::
:99\r\nBulk strings are unique in that they have two parts. A length specification and a body:
$13\r\nHello, World!\r\n$ is the bulk string prefix, 13 is the number of bytes in the actual string
body, and then \r\n terminates the length specification. Hello, World! is
the 13 byte string body, and that is also terminated with \r\n (which is not
part of the string body).
Because the bulk string container provides an exact length for the body, we never have to parse the actual string body to find the end — we can read another 13 bytes (plus the final line terminator) from the socket without inspecting the contents of the string. This means we can use any data that we want in the body of the string without encoding or escaping the contents.
Arrays also begin with a length specification, except the length field indicates the number of objects in the array rather than the number of bytes in the array’s contents.
*2\r\n$3\r\nfoo\r\n$3\r\nbar\r\nIn the above example, we have an array that contains two bulk strings (foo and
bar). While the array length specification doesn’t allow us to skip ahead the
way that we can when reading a bulk string, it does make it easy to implement
arrays in our client. We read the array size, read that many
more objects, and then return all of those objects in the final array.
All Redis commands are sent as arrays of bulk strings. For example, the command “SET mykey ‘my value’” would be written and sent as:
*3\r\n$3\r\nSET\r\n$5\r\nmykey\r\n$8\r\nmy value\r\nThe prefixed lengths used in RESP allow parsers to be implemented without needing to do complicated state modeling or multiple passes through the data, allowing for very fast parsers to be built.
There’s a number of details I haven’t covered here, such as null bulk strings and null arrays. For more information, the official documentation (as usual) has a very readable and comprehensive documentation page for RESP.
A great way to understand the protocol better is to try implementing a simple client yourself. Check out our guide to reading and writing the Redis protocol in Go next.