Sunday, July 7, 2013

Unit Testing PUTs in Dart (the Hard Way)


Today, I would like to explore unit testing HTTP PUT operations in Dart. I am expanding the test coverage of HipsterSync, the persistence layer in Hipster MVC. As of last night, I can POST new records in there, so today, I need to build on that to PUT new records on top of those newly POSTed records.

As an aside, I am not overly fond of true RESTful interfaces. For instance, I doubt that I will even go as far as supporting POSTing of existing records. I am more or less content with claiming REST-like support in my applications and that ought to be more than sufficient for the exploration that I have planned. Anyhow…

I start with a new unit test group for PUT:
    group("HTTP put", (){
      var model_id;

      setUp((){
        var completer = new Completer();

        // Will obtain the model_id and complete the completer here...

        return completer.future;
      });
    });
This group opens with a setUp() block that returns a future. This is the Dart unit test way to signal that the setup involved is asynchronous and that subsequent tests need to wait until an asynchronous event occurs. The HipsterSync class is rife with futures, so I could have returned a future directly from one of its methods. But I need to grab the model_id after a successful POST, so I manually instantiate my own future here.

The rest of the setup is doing just that. I POST a new record, with a low-value in the attributes, and then complete my local completer when the server responds successfully:
    solo_group("HTTP put", (){
      var model_id;

      setUp((){
        var completer = new Completer();

        var model = new FakeModel();
        model.url = 'http://localhost:31337/widgets';
        model.attributes = {'test': 1};
        HipsterSync.
          call('create', model).
          then((rec) {
            model_id = rec['id'];
            completer.complete();
          });

        return completer.future;
      });
    });
Since Dart provides no mechanism to stub out HttpRequests from the browser, I need an actual server to respond to that call. So in this case, the URL http://localhost:31337/widgets is not just test scenery—it is making an actual call to the test server that I have been building and that runs under continuous integration.

With that, I am ready to write my actual test. Both the create and update operations in HipsterSync are implemented in the call() static method. This is done to mimic the Backbone.js API and to make it easier to swap out the persisting behavior (e.g. to support persisting data in localStorage). The only difference between the two calls are that the MVC model being persisted needs an ID in the URL and the call() requires “update” as the first argument instead of “create.” So the test ends up looking very similar to the setup:
    group("HTTP put", (){
      var model_id;
      setUp((){ /* ... */ });

      test("it can PUT on top of existing records", (){
        var model = new FakeModel();
        model.url = 'http://localhost:31337/widgets/${model_id}';
        model.attributes = {'test': 42};

        HipsterSync.
          call('update', model).
          then(
            expectAsync1((response) {
              expect(response, containsPair('test', 42));
            });
          );
      });
    });
Since call() returns a Future in both cases, I still need to account for asynchronous behavior in my test, hence the expectAsync1() call in there. The expectAsyncN() methods will poll in the test until the expected asynchronous call (with the supplied arity) is invoked. In this case, I expect that the response will be a HashMap containing the key-value pair of test & 42 (instead of the originally created test & 1).

Since I am exploring this kind of testing for the first time, my test server (also written in Dart) does not support HTTP PUT. In test_server.dart, I add PUT support to my simplistic router:
handleWidgets(req) {
  var r = new RegExp(r"/widgets/([-\w\d]+)");
  var id_path = r.firstMatch(req.uri.path),
      id = (id_path == null) ? null : id_path[1];

  if (req.method == 'POST') return createWidget(req);
  if (req.method == 'GET' && id != null) readWidget(req);
  if (req.method == 'PUT' && id != null) updateWidget(id, req);

  notFoundResponse(req);
}
Then write the supporting updateWidget() method:
updateWidget(id, req) {
  HttpResponse res = req.response;

  if (!db.containsKey(id)) return notFoundResponse(req);

  List<int> data = [];
  req.listen(
    (d) {data.addAll(d);},
    onDone: () {
      var body = new String.fromCharCodes(data);
      var widget = db[id] = JSON.parse(body);

      res.statusCode = HttpStatus.OK;
      res.write(JSON.stringify(widget));
      res.close();
    }
  );
}
I am experimenting a little here with the Stream nature of HttpRequest. I should be able to listen to the request stream to read in the body of the request, which contains my JSON payload. Except I can't.

It seems that I am too used to node.js streams here—or at least too used to node.js socket streams. In my Dart server, I am continuously listening to the request stream, adding the byte contents to a data list. I expect that, once the request has completed, my onDone function will be invoked, at which point I can reassemble the bytes, parse them as JSON and store them in my local DB.

What I find is that, by the time my onDone callback is invoked, the response stream is already closed. I can tell this because my server crashes with:
➜  test git:(http-test-server) ✗ dart test_server.dart
Server started on port: 31337
Uncaught Error: Bad state: StreamSink is closed
Stack Trace:
#0      _StreamSinkImpl._controller (io_sink.dart:78:7)
#1      _StreamSinkImpl.add (io_sink.dart:25:5)
#2      _IOSinkImpl.write (io_sink.dart:127:8)
#3      _HttpOutboundMessage.write (http_impl.dart:278:20)
#4      updateWidget.<anonymous closure> (file:///home/chris/repos/hipster-mvc/test/test_server.dart:88:16)
...
Update: see below for the real explanation of this behavior.

If isDone is only invoked once the response stream is closed, then it is useless to me in this case. So I may as well make use of the iterable nature of HttpRequest streams to reduce this to the value that I want:
updateWidget(id, req) {
  HttpResponse res = req.response;

  if (!db.containsKey(id)) return notFoundResponse(req);

  req.
    reduce((prev, d)=> new List.from(prev).addAll(d)).
    then((data) {
      var body = new String.fromCharCodes(data);
      var widget = db[id] = JSON.parse(body);

      res.statusCode = HttpStatus.OK;
      res.write(JSON.stringify(widget));
      res.close();
    });
}
Only that still will not work. Again, by the time I try to write to the response stream, it is closed:
➜  test git:(http-test-server) ✗ dart test_server.dart
Server started on port: 31337
Uncaught Error: Bad state: StreamSink is closed
Stack Trace:
#0      _StreamSinkImpl._controller (io_sink.dart:78:7)
#1      _StreamSinkImpl.add (io_sink.dart:25:5)
#2      _IOSinkImpl.write (io_sink.dart:127:8)
#3      _HttpOutboundMessage.write (http_impl.dart:278:20)
#4      updateWidget.<anonymous closure> (file:///home/chris/repos/hipster-mvc/test/test_server.dart:88:16)
...
Update: see below for the real explanation of this behavior.

It seems that I have to write responses as soon as I get data. Except event that does not work. What. The…

Update: And then I realize that my updateWidget() PUT handler has been correct all along. The problem was the router, which needed to return the call to updateWidget() to guard against the code executing the notFoundResponse() method:
handleWidgets(req) {
  var r = new RegExp(r"/widgets/([-\w\d]+)");
  var id_path = r.firstMatch(req.uri.path),
      id = (id_path == null) ? null : id_path[1];

  if (req.method == 'POST') return createWidget(req);
  if (req.method == 'GET' && id != null) return readWidget(req);
  if (req.method == 'PUT' && id != null) return updateWidget(id, req);

  notFoundResponse(req);
}
Bleh. Sometimes I can get tunnel vision while trying to finish up these posts. Both the onDone stream approach and the reduce iterable approach work in addition to the the vanilla stream approach as well.


Day #805

No comments:

Post a Comment