Hookbox is a new Python-based comet server with support for Websockets. It’s main features include:

  • Support for named channels on which data is published/subscribed.
  • Server-side Websocket and WSGI support via the eventlet concurrent networking library.
  • Client-side Websocket support via js.io.
  • Fall-back to a custom comet protocol when websocket support is not available in the browser.
  • A RESTful api to interact with the channels.
  • Easy integration with web frameworks like Quixote and Django, again via web hooks.
  • A built-in admin interface to view the state of active channels and users.

It has been authored by some of the original people behind Orbited and represents a significant simplification in the process of writing websocket-enabled web apps primarily because it includes its own message broker.

Experiment: Graphic EQ

To better understand Hookbox, I decided to write a web-based graphic EQ which updates itself based on data produced on the server (strictly speaking, a graphic EQ is a control instrument, but my example presents it as a measurement instrument so perhaps calling it a web-based spectrum analyzer would be more appropriate). Checkout the video below to see it in action:

Hookbox Graphic EQ Demo from Salman Haq on Vimeo.

The top-half of the clip shows the equalizer bands dynamically updating in Firefox (v3.6.8) while the bottom-half shows the Firebug pane displaying the live network calls. If you look carefully, you will see that in this particular experiment all the network requests used the CSP protocol and not the websocket protocol. This is expected because websocket support is only available in version 4 or later of Firefox.

The video capture was a bit sluggish so it’s worth mentioning that it actually ran much smoother than it appears in the video. This is due to the fact that graphics acceleration is improperly configured on my quad-core box. (Side note: Each time I upgrade to the latest Ubuntu release, my video and printer settings need to be tweaked significantly to recover optimal performance. Since I upgraded to Lucid Lynx, I have not been able get the video performance I would like).

Trials

The experiments were repeated using three data publishing rates: 10 Hz, 100 Hz, and 1000 Hz. I should note that the producer.py process used time.sleep() which is not very precise on systems running a non-realtime kernel which typically lack high-precision timers. At the 10 and 100 Hz rates, the browser was able to keep up with the deluge of data and rendered the bands quite smoothly. At the 1KHz rate, all of my system’s cores reached maximum loading and Firefox rendered the bands rather slowly. The memory consumption did not jump between these trials and the network bandwidth increased slightly but never crossed 30 kbps in the steady-state.

These observations suggest that the experiments were CPU-limited rather than memory or bandwidth limited as the data publishing rate was increased. This is fitting with my earlier remark that graphics acceleration on the test system is mis-configured, which means that the display rendering was done by the CPUs rather than the onboard video card and hence the high cpu consumption.

I failed to repeat these experiments with Google Chrome due to this bug which I consequently filed on github. But given that V8 is a faster javascript engine, I would expect Chrome to perform better.

Code

The source code of the experiment is available in my fork of the hookbox project on github (I expect it will be pulled into Michael Carter’s main branch soon). It consists of fewer than 100 lines of Python, fewer than 30 lines of Javascript, and took about a couple of hours to get working satisfactorily. That’s not bad considering that it was an introductory play-date with Hookbox.

The equalizer widget consists of seven sliders from the jQuery-UI library, lined up vertically alongside each other. To update the sliders, a call back is attached to the hookbox subscription object’s “onPublish” event which looks like this:

                // see index.html for the full code.
                subscription.onPublish = function(frame) {
                  // adjust each slider from the data values in the frame's payload.
                  // the payload is expected to be an array of seven integers (one for
                  // each slider).
                  $("#eq > span").each(function (index) {
                        $(this).slider("value", frame.payload[index]);
                  });
                };

The data production and publishing loop is also very simple. Seven random integer samples are produced, and sent via a HTTP POST to the Hookbox server, which then publishes it to all subscribers of that channel:

   # see producer.py for the full code.
    while True:
        # generate seven random data points
        # and publish them via the HTTP/REST api at 10 Hz.
        # pop = range(0,100) (defined earlier)
        # 'values' is a dictionary with some other fields required by hookbox (also defined earlier).
        values["payload"] = random.sample(pop, 7)
        data = urllib.urlencode(values)
        req = urllib2.Request(url, data)
        # publish the samples!
        resp = urllib2.urlopen(req)
        time.sleep(0.1)

The common and most important thing between these two snippets is the “payload” object. In this example, it is an array of seven integers, ranging from 0 to 99 that is produced by the aptly named producer.py which also publishes it via the hookbox server and is received by the javascript running in the browser. The rest of the code in my example is quite self-explanatory and is a good way to learn Hookbox (and even Quixote).

Conclusion

While still in its infancy, Hookbox looks like a very promising project. From a developer’s perspective, it allows you to get something up and running very quickly. Its simplicity derives from the web hook api which also concerns me a bit. Using HTTP for inter-process communication, as in my example producer.py communicates with the hookbox server to publish data, doesn’t seem like the best choice for low-latency high-throughput applications. Perhaps Hookbox might introduce a web socket interface for inter-process communication so that the data can travel from the producer to the consumer using only one protocol? Overall, I’m quite impressed with Hookbox and really hope that it gains momentum. And lastly, don’t forget to read Michael Carter’s introduction to the project on cometdaily.com!

EDIT 8/13/2010: Another Hookbox demo by me, Scope. See this video of it in action.

EDIT 8/16/2010: If you have questions about the source code or have problems with the demo, please contact me via the Hookbox mailing list for the benefit of other Hookbox users and developers.