Wednesday, March 31, 2010

CouchDB Relaximation

‹prev | My Chain | next›

Getting back to node.js, I think I will explore some more of the node.js things linked to CouchDB. Several folks were kind enough to provide links in response to a post from last week. One of the first on the list is relaximation.

The relaximation script establishes a number of node.js clients to perform concurrent reads and writes against a CouchDB server. It then performs a statistical analysis of the results, providing a nice graph output. By default, it creates 50 writing processes and 200 reading processes. Other defaults can be seen in the help:
cstrom@whitefall:~/repos/relaximation/tests$ ~/local/bin/node compare_write_and_read.js --help
-w, --wclients :: Number of concurrent write clients per process. Default is 50.
-r, --rclients :: Number of concurrent read clients per process. Default is 200.
-u, --url1 :: CouchDB url to run tests against. Default is http://localhost:5984
-v, --url2 :: CouchDB url to run tests against. Default is http://localhost:5985
-1, --name1 :: Name of first comparative. Required.
-2, --name2 :: Name of first comparative. Required.
-d, --doc :: small or large doc. Default is small.
-t, --duration :: Duration of the run in seconds. Default is 60.
-i, --poll :: Polling interval in seconds. Default is 1.
-p, --graph :: CouchDB to persist results in. Default is http://couchdb.couchdb.org/graphs
-r, --recurrence :: How many times to run the tests. Deafult is 10.
I am still running on my netbook here, which only has a single CouchDB server. I do have the VMs lying about, so let's see how a bare metal CouchDB server compares to a VM CouchDB server:
cstrom@whitefall:~/repos/relaximation/tests$ ~/local/bin/node compare_write_and_read.js --name1 netbook --name2 vm-on-netbook --url2 http://couch-011a.local:5984
{"time":1.006,"writes":{"clients":10,"average":63,"last":52},"reads":{"clients":0,"average":null,"last":null}}
{"time":2.001,"writes":{"clients":18,"average":155,"last":63},"reads":{"clients":9,"average":154,"last":33}}
{"time":3,"writes":{"clients":28,"average":259,"last":13},"reads":{"clients":18,"average":209,"last":211}}
{"time":4.002,"writes":{"clients":33,"average":461,"last":200},"reads":{"clients":27,"average":236,"last":113}}
{"time":5.001,"writes":{"clients":45,"average":629,"last":134},"reads":{"clients":35,"average":371,"last":158}}
{"time":6.003,"writes":{"clients":45,"average":640,"last":56},"reads":{"clients":46,"average":229,"last":113}}
{"time":7.002,"writes":{"clients":46,"average":1003,"last":526},"reads":{"clients":55,"average":343,"last":16}}
...
... Lots and lots and lots of output
...
{"time":58.002,"writes":{"clients":50,"average":2759,"last":23},"reads":{"clients":200,"average":2636,"last":13}}
{"time":59.008,"writes":{"clients":50,"average":2741,"last":32},"reads":{"clients":200,"average":2578,"last":30}}
http://couchdb.couchdb.org/graphs/_design/app/_show/compareWriteReadTest/f4eb5ba38837ce71c2ced8e583004e4f
That is one pretty graph. Kudos on the use of a <canvas> graph.

It is interesting looking at the graph seeing that read/writes both peak at about 30 seconds, then decrease slightly until ~40 seconds of heavy pounding at which point things stay nice and steady. I would have expected things to get optimized relatively quickly and then to stabilize. Perhaps the slight trend downward is a result of reaching some threshold for number of documents in the database. Grist for IRC conversations tomorrow.

Another interesting thing to note is that the VM outperforms the localhost CouchDB server. It took me a while to remember that the VM and localhost CouchDB servers are at different versions (0.11-pre vs. 0.10). It seems clear that there were some optimizations added between 0.10 and 0.11. All the more reason to upgrade.

The last thing to note on the graphs is that the VM/0.11 graphs are nice and smooth while the localhost/0.10 graphs are jagged (even after 10 runs). I am not sure of the reason for this, but the last few runs against this DB resulted in errors similar to:
[Thu, 01 Apr 2010 01:55:23 GMT] [debug] [<0.25812.14>] httpd 500 error response:
{"error":"unknown_error","reason":"normal"}


[Thu, 01 Apr 2010 01:55:23 GMT] [info] [<0.25873.14>] Stacktrace: [{mochiweb_request,send,2},
{mochiweb_request,respond,2},
{couch_httpd,send_response,4},
{couch_httpd_db,do_db_req,2},
{couch_httpd,handle_request,5},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]
Too many of those could have skewed the statistics.

Looking through the code, there is oodles to learn. It still astounds me how much one can do with Javascript. This guy even implements his own OptionParser in Javascript. Crazy stuff.

That was fun and even more educational than I expected. The concurrent clients seem a nice use of node.js that I would not have otherwise thought of. Tomorrow, I think I would like to explore some of the node.js libraries that interact with the _changes API in CouchDB.

Day #59

2 comments:

  1. I implemented the optionparser because, at the time, there wasn't one for node. As far as I know there still isn't a good one.

    The graph is a couchapp which is also in relaximation called "graphs" and it uses the flot jquery plugin to do the graphing.

    We're using all of this at couch.io to compare different performance tweaks that we try in git branches and a few other kinds of performance comparisons. It's cool to see someone else get some use out of it, I'll be sure to improve on the docs and get some more automated stuff up later :)

    One thing you want to do is always run CouchDB with delayed_commits off. This means that responses won't return until they are written to disc, without it responses return immediately and are batched every second. With delayed_commits off the writes are batched efficiently under concurrent load but single writer performances looks terrible which is why the default is on.

    ReplyDelete
  2. @mikeal Thanks for all of the pointers!

    "As far as I know, there still isn't a good one." Classic. Still good fun to read through.

    I reran the script with delayed_commits off and found results more in line with what I expected:

    http://couchdb.couchdb.org/graphs/_design/app/_show/compareWriteReadTest/f4eb5ba38837ce71c2ced8e583005c44

    No idea why the v0.10 was so fast writing, but there were lots of errors in there.

    Thanks again for the node.js help and pointers -- they've help a lot!

    ReplyDelete