Retwiz demonstrates a simpler, scalable clone of Twitter
Retwiz cropped up recently whilst i was browsing the usual technology blogs. It seems like a really clever implementation of a Twitter style application utilizing the Redis database (and supporting PHP client libraries). Redis is a non-relational, key-value database that offers some pretty impressive features:
Redis is a key-value database. It is similar to memcached but the dataset isn’t volatile. Values can be strings, exactly like in memcached, as well as lists and sets with atomic operations to push/pop elements. In order to be very fast but at the same time persistent the whole dataset is taken into memory and from time to time and/or when a number of changes to the dataset are performed it is written asynchronously on disk.
Now although i don’t know that much about databases, i could see how this sort of implementation could be seriously scalable. The killer feature here seems to be speed (the database engine manages to intelligently load the dataset into memory every now any then, making performing queries a lot speedier), the Google Code page boasts some pretty impressive benchmarks:
Redis is pretty fast!, 110000 SETs/second, 81000 GETs/second in an entry level Linux box.
====== SET ======
100007 requests completed in 0.88 seconds
50 parallel clients
3 bytes payload
keep alive: 1
58.50% <= 0 milliseconds
99.17% <= 1 milliseconds
99.58% <= 2 milliseconds
99.85% <= 3 milliseconds
99.90% <= 6 milliseconds
100.00% <= 9 milliseconds
114293.71 requests per second
====== GET ======
100000 requests completed in 1.23 seconds
50 parallel clients
3 bytes payload
keep alive: 1
43.12% <= 0 milliseconds
96.82% <= 1 milliseconds
98.62% <= 2 milliseconds
100.00% <= 3 milliseconds
81234.77 requests per second
The Twitter based example shows how to build a simple system based around the Redis database, most notably focusing on how a typical application supported by Redis could be horizontally scaled to many clustered servers, as there is no one point where the entire dataset is required at once.
Redis includes client libraries for Ruby, Python, Erlang and PHP, so I’ll be checking this out over the next week or so as it seems a pretty clever concept, something which I’ve not seen before (although I’m sure there are other similar implementations out there?). Retwiz is written in PHP, therefore i might try and develop something similar in my preferred language, Ruby.
Grab the current Redis source here: http://redis.googlecode.com/files/redis-beta-6.tar.gz
Grab the source of the Retwiz example app here: http://code.google.com/p/redis/downloads/list (The english isn’t perfect but the write up is a really good explanation of the basic implementation details).
Shorthand Aliases for RAILS_ENV
Sometimes i find myself referring to the current rails environment in my views and controllers like this:
Although this serves its purpose, its ugly. There is a nicer way to access this value across your Rails applications:
# shorthand aliases for (RAILS_ENV == "environment")
def production?
RAILS_ENV == "production"
end
def staging?
RAILS_ENV == "staging"
end
def development?
RAILS_ENV == "development"
end
# and register these methods for use in the views too
helper_method :production?
helper_method :staging?
helper_method :development?
This now provides us with three helper methods to use within the controllers, then we make them available as view helpers so we can do neater conditional view statements.
Leopard and Capistrano 1.4.1 Recipes
Recently whilst installing a new Rails stack on Leopard i ran into some issues with a Capistrano 1.4.1 recipe. I was getting the following error:
It took me a while to come to the conclusion that the problem was some strange incompatibility with Leopard and Capistrano 1.4.1 (more to the point, one of the dependent Gems, NetSSH). It seemed to be a conflict with how Leopard and NetSSH store SSH known host keys (but i didn’t have time to check this out properly).
To fix this problem, you have to uninstall Capistrano (or maybe just the dependent gem, NetSSH, but i did all versions just to ensure that a newer version wasn’t taking precedence over an older version, this included the ‘cap’ and ‘capify’ links).
I also uninstalled the other dependencies, just to make sure. The correct stack looks something like:
net-scp (1.0.0)
net-sftp (1.1.1, 1.1.0)
net-ssh (1.1.2, 1.0.10)
net-ssh-gateway (1.0.0)
Now i just have to update all my recipes to roll on Capistrano 2.x. Hopes this saves someone some time!