Tag Archives: rsmb

My #lca2011 talk – with moving pictures!

I’ve already written about my talk at LinuxConf Australia a little while ago (slides on Slideshare). Now, using some newfangled magical technology with moving pictures and recorded sounds, the video is online – and here it is, in case you would like to check out the full 40 minute talk and brief Q&A (sorry, but I can’t resize the embedded player!)

Advertisements

OS X mosquitto “bites”…

In my post last week about the new MQTT support coming to WebSphere MQ I very briefly mentioned that there are some third-party tools that already implement MQTT. One of those I pointed to is the very neat mosquitto broker, a project started by Roger Light.

mosquitto has been around for a while now and is aiming to replicate the functionality of the Really Small Message Broker that is on IBM alphaWorks. One of the neat things about it, from my point of view, is that it there is an Ubuntu PPA repository, so with a couple of apt commands, I can install a running MQTT broker and build my own applications independently (NB there are packages for other Linux distros too, as well as Windows). When I want to do some “heavy lifting” or share data with my ESB, I connect up my local mosquitto broker to pass messages across to WebSphere MQ through the new telemetry channels – because MQTT supports a concept of bridges, and both RSMB and mosquitto both include support for bridging.

I noticed that there wasn’t yet a version available for Mac OS X but figured that it shouldn’t be too difficult to compile and run it on that platform. As it happens, it did turn into a bit of an adventure for a couple of reasons, but at least I learned from the experience. If you’re desperate to build yourself a version to try some MQTT development on the Mac, here’s what I had to do to get it going on Snow Leopard:

  1. Installed mercurial, and a GUI for it called Murky (which requires the hg command line tool from the base mercurial package). The sources for mosquitto are in bitbucket, a Mercurial repository… this is optional of course as I could have just used a source tarball.
  2. Grabbed the latest mosquitto source from bitbucket.
  3. Modified the Makefiles throughout the mosquitto tree to build libraries with a .dylib instead of a .so extension (the default on OS X), and also changed the -soname parameter to -install_name which the OS X version of gcc understands.
  4. At this point the compile was starting to show progress… but failing due to missing symbols… the offender being one from sqlite, _sqlite_enable_load_extension. Turns out that the version of sqlite shipped in OS X 10.6.x is 3.5.4 but it does not have extension loading functionality built in, as evidenced by nm -g /usr/lib/libsqlite3.dylib | grep 'sqlite3_enable'
  5. Downloaded sqlite3.8.0, configured it to install to /usr/local (to avoid overwriting the default OS X shipped version), and built and installed it with no issues.
  6. At this point the compile was pretty smooth, once I modded Makefile link and include lines to point to the new version of sqlite in /usr/local. The only thing that failed was documentation, but that was “optional” 🙂
  7. Trying to start the broker failed… because it was trying to load the sqlite3-pcre extension.
  8. Installed git (the source for the sqlite3-pcre extension is in a git repository).
  9. Grabbed the source for sqlite3-pcre and built and installed it using:
    gcc -shared -o pcre.so -L/usr/local/lib -lsqlite3
              -lpcre -Werror pcre.c -I/usr/local/include
    sudo mkdir /usr/local/lib/sqlite3
    sudo cp pcre.so /usr/local/lib/sqlite3
  10. The final issue was that the path to the pcre extension is hard-coded into mosquitto/src/conf.c so I modded that to point at the version in /usr/local and recompiled. I’m assuming that this would not generally be required, but it worked as a hack to get me going!
    D’oh. Just realised that this is precisely what the ext_sqlite_regex variable in the mosquitto.conf file is for. Shouldn’t have bothered!

So that was it. Being fair, if I hadn’t been feeling my way through that, I would have installed git and mercurial, grabbed all the lib sources for sqlite3 and pcre and built them, built mosquitto, and been good to go. At this point, the broker and test clients are runnable (assuming the library paths are set up appropriately):

DYLD_LIBRARY_PATH=/usr/local/lib ./mosquitto
DYLD_LIBRARY_PATH=../lib ./mosquitto_pub -t test/andy
        -m "hello world"

If you are interested in seeing this in action, here’s a short (and silent, but annotated) screencast:

The Java GUI application you see in the screencast is the test client shipped in an old IBM SupportPac, IA92, a Java implementation of MQTT. The final release of the WebSphere MQ Telemetry component for WebSphere MQ will contain something similar, considerably enhanced and integrated into WebSphere MQ Explorer.

In other news, Roger recently announced version 0.8 of mosquitto, which now has slightly different packaging and includes C, C++ and Python clients. I hope to give these a test drive shortly!

Experiments with PHP and MQTT

Over the past few days I’ve been playing around with combining lightweight messaging and PHP. There are a couple of reasons for this, but the primary one is that I’d like to extend my prototype iPhone CurrentCost monitoring web application to display more up-to-date information about the state of my home energy usage. I’d planned to do this for a while, but recently Mark Taylor created his own version of the iPhone interface (PDF link) and he has got current readings on the front page. Clearly, I have to compete 🙂

Actually, in my system, I’d like to do things a different way. The heart of my setup is a Really Small Message Broker. At the moment, data from the CurrentCost meter comes in over the USB connection and is then published in pieces, or on topics, to the RSMB (temperature and energy readings are separate). These published messages are then read by a script which is subscribing to the topics and squirrelling the historical data into an rrdtool database; and also being pushed up to our IBM broker “in the interweb cloud” via an MQTT broker bridge connection.

So in theory, having the up-to-date information in the web UI should be a simple case of grabbing the MQTT publications on each topic and displaying them. The way I’ve coded things (and would prefer to do things), this involves having the ability to subscribe to MQTT publications from PHP.

I’m not at the end of the road yet, but I do have a starting point.

howitworks.png

I’ve got a front-end test page which currently uses Prototype to send an Ajax request to a server-side PHP script (yes, I have had jQuery recommended to me, and I may well look into that instead of Prototype, but this works).

The server-side script uses the Simple Asynchronous Messaging PHP library. SAM is a wrapper which enables a variety of messaging transports to be supported in PHP, such as MQTT, WebSphere MQ or WebSphere Platform Messaging. Just one thing: I found that in order to get the most recent SAM release to work on Ubuntu on my MPC-L, I had to install IBM’s XMS client SupportPac (for some reason, it won’t build without it, even though it is “optional”) and I also had to delete a spurious empty line from the end of /usr/share/php/SAM/php_sam.php to prevent header issues. Other than that, it was all good.

The script is really simple and basically uses all of the defaults to create a connection to my local RSMB over MQTT. The advantage of this being server-side is that I don’t have to open my RSMB to the Internet, the PHP code can connect via localhost. Once that’s done, it creates a subscription on the topic I’ve asked for, and receives the first data that comes along, then echoes it back to the front-end. I could make it auto-updating with Ajax.PeriodicalUpdater too, but there’s no need to put a load on my server.

Wanna see a quick demo? 😉

I’m quite pleased with the way this is working. There’s some more plumbing to do, and I’ll almost certainly extend the server-side piece to allow two-way communications (publish as well as subscribe) as well as finer-grained control over the options. As a proof-of-concept though, I think this is looking good.