TL;DR
I recently got a new MacBook Pro and I wanted to do some benchmarking. Naturally I turned to the leaderboard library I maintain which already had a base set of benchmarks that I could compare the results to on my new computer. You may also recall I’ve written about leaderboards before here on #AltDevBlogADay. In this blog post I’ll discuss two “knobs” I tweaked to get more performance out of the leaderboard library, the underlying interpreter and an underlying client library. While there was definitely a speed improvement when switching the underlying interpreter to a more recent version, there was a more significant speed improvement when switching the underlying client library.
TWEAKING KNOBS
The benchmarks were run on a 2.5 GHz Intel Core i7 MacBook Pro with 8 GB of memory under OS X Lion 10.7.2 using Redis 2.4.5. They measured the total amount of time to rank 10 million people in a leaderboard using sequential scores, total amount of time to rank 10 million people in a leaderboard using random scores and the average time to retrieve an arbitrary page from the leaderboard. The basic benchmarking code can be found in the README on the leaderboard project page.
Switching Ruby Interpreters
Below are the benchmark results from switching between different Ruby interpreters, Ruby 1.8.7, Ruby 1.9.2 and Ruby 1.9.3. Values are in seconds.
Ruby 1.8.7
Time to rank 10 million people in a leaderboard (sequential scores): 794.157420158386
Time to rank 10 million people in a leaderboard (random scores): 849.301838159561
Average time to retrieve an arbitrary page from the leaderboard (50,000 requests): 0.00165219999999999
Ruby 1.9.2
Time to rank 10 million people in a leaderboard (sequential scores): 656.387526
Time to rank 10 million people in a leaderboard (random scores): 748.955826
Average time to retrieve an arbitrary page from the leaderboard (50,000 requests): 0.0011321999999999895
Ruby 1.9.3
Time to rank 10 million people in a leaderboard (sequential scores): 651.057383
Time to rank 10 million people in a leaderboard (random scores): 719.157958
Average time to retrieve an arbitrary page from the leaderboard (50,000 requests): 0.001079199999999996
Summary
Ruby 1.8.7 to Ruby 1.9.3 (sequential scores): 18% improvement
Ruby 1.8.7 to Ruby 1.9.3 (random scores): 15% improvement
Ruby 1.8.7 to Ruby 1.9.3 (average page request): 34% improvement
Switching Redis Client Library
I decided not to benchmark Ruby 1.8.7 or Ruby 1.9.2 in tweaking the client driver “knob”.
Below are the benchmark results using Ruby 1.9.3 with the hiredis-rb client library. Values are in seconds.
Ruby 1.9.3 and hiredis
Time to rank 10 million people in a leaderboard (sequential scores): 472.544572
Time to rank 10 million people in a leaderboard (random scores): 549.911350
Average time to retrieve an arbitrary page from the leaderboard (50,000 requests): 0.0003803999999999928
Summary
Ruby 1.8.7 to Ruby 1.9.3/hiredis (sequential scores): 40% improvement
Ruby 1.8.7 to Ruby 1.9.3/hiredis (random scores): 35% improvement
Ruby 1.8.7 to Ruby 1.9.3/hiredis (average page request): 76% improvement
Ruby 1.9.3 to Ruby 1.9.3/hiredis (sequential scores): 27% improvement
Ruby 1.9.3 to Ruby 1.9.3/hiredis (random scores): 23% improvement
Ruby 1.9.3 to Ruby 1.9.3/hiredis (average page request): 64% improvement
WRAPUP
It seems every aspect of video game technology is tweaked to maximize performance, whether it be frames per second or requests per second. Fiddling with very basic “knobs” here, I was able to get decent improvements in writing to and reading from a leaderboard of non-trivial size. Further speed improvements could potentially be realized by using a socket-based connection versus a TCP-based connection as well as faster hardware optimized for a server environment versus an off-the-shelf laptop.
Thanks for indulging my leaderboard nerdery yet again. You can find more hilarity over on my Twitter account, @CzarneckiD.