In my post last week about the new MQTT support coming to WebSphere MQ I very briefly mentioned that there are some third-party tools that already implement MQTT. One of those I pointed to is the very neat mosquitto broker, a project started by Roger Light.
mosquitto has been around for a while now and is aiming to replicate the functionality of the Really Small Message Broker that is on IBM alphaWorks. One of the neat things about it, from my point of view, is that it there is an Ubuntu PPA repository, so with a couple of apt commands, I can install a running MQTT broker and build my own applications independently (NB there are packages for other Linux distros too, as well as Windows). When I want to do some “heavy lifting” or share data with my ESB, I connect up my local mosquitto broker to pass messages across to WebSphere MQ through the new telemetry channels – because MQTT supports a concept of bridges, and both RSMB and mosquitto both include support for bridging.
I noticed that there wasn’t yet a version available for Mac OS X but figured that it shouldn’t be too difficult to compile and run it on that platform. As it happens, it did turn into a bit of an adventure for a couple of reasons, but at least I learned from the experience. If you’re desperate to build yourself a version to try some MQTT development on the Mac, here’s what I had to do to get it going on Snow Leopard:
- Installed mercurial, and a GUI for it called Murky (which requires the hg command line tool from the base mercurial package). The sources for mosquitto are in bitbucket, a Mercurial repository… this is optional of course as I could have just used a source tarball.
- Grabbed the latest mosquitto source from bitbucket.
- Modified the Makefiles throughout the mosquitto tree to build libraries with a .dylib instead of a .so extension (the default on OS X), and also changed the
-sonameparameter to-install_namewhich the OS X version of gcc understands. - At this point the compile was starting to show progress… but failing due to missing symbols… the offender being one from sqlite,
_sqlite_enable_load_extension. Turns out that the version of sqlite shipped in OS X 10.6.x is 3.5.4 but it does not have extension loading functionality built in, as evidenced bynm -g /usr/lib/libsqlite3.dylib | grep 'sqlite3_enable' - Downloaded sqlite3.8.0, configured it to install to /usr/local (to avoid overwriting the default OS X shipped version), and built and installed it with no issues.
- At this point the compile was pretty smooth, once I modded Makefile link and include lines to point to the new version of sqlite in /usr/local. The only thing that failed was documentation, but that was “optional” š
- Trying to start the broker failed… because it was trying to load the sqlite3-pcre extension.
- Installed git (the source for the sqlite3-pcre extension is in a git repository).
- Grabbed the source for sqlite3-pcre and built and installed it using:
gcc -shared -o pcre.so -L/usr/local/lib -lsqlite3 -lpcre -Werror pcre.c -I/usr/local/include sudo mkdir /usr/local/lib/sqlite3 sudo cp pcre.so /usr/local/lib/sqlite3 - The final issue was that the path to the pcre extension is hard-coded into mosquitto/src/conf.c so I modded that to point at the version in /usr/local and recompiled. I’m assuming that this would not generally be required, but it worked as a hack to get me going!
D’oh. Just realised that this is precisely what the ext_sqlite_regex variable in the mosquitto.conf file is for. Shouldn’t have bothered!
So that was it. Being fair, if I hadn’t been feeling my way through that, I would have installed git and mercurial, grabbed all the lib sources for sqlite3 and pcre and built them, built mosquitto, and been good to go. At this point, the broker and test clients are runnable (assuming the library paths are set up appropriately):
DYLD_LIBRARY_PATH=/usr/local/lib ./mosquitto
DYLD_LIBRARY_PATH=../lib ./mosquitto_pub -t test/andy
-m "hello world"
If you are interested in seeing this in action, here’s a short (and silent, but annotated) screencast:
The Java GUI application you see in the screencast is the test client shipped in an old IBM SupportPac, IA92, a Java implementation of MQTT. The final release of the WebSphere MQ Telemetry component for WebSphere MQ will contain something similar, considerably enhanced and integrated into WebSphere MQ Explorer.
In other news, Roger recently announced version 0.8 of mosquitto, which now has slightly different packaging and includes C, C++ and Python clients. I hope to give these a test drive shortly!
A good writeup, thanks. I think it’s worth getting those changes you’ve made for compiling libraries into the Makefiles proper.
A few clarifications: the minimum version of sqlite3 needed by mosquitto is 3.6.14.1, so even if it’d had extension support it wouldn’t have worked.
The failure building the documentation is, as you say, mostly optional š The documentation is already built in the released versions, it’s only in the repository that this is an issue. The latest versions of the man pages are always at http://mosquitto.org/man/ as well.
Happy to send you my modded Makefiles, I’d just note that I’ve done nothing to make the changes conditional… It was all in-place hackage just to get a build to work. No issues with sharing though! š
Interesting point in sqlite, I couldn’t see a min version requirement in the mosquitto docs I read but maybe that was me not looking hard enough. Very pleased with how cleanly that built, anyway. Since sqlite is part of Apple’s core data API set I didn’t want to overwrite their shipped version so figured a separate local copy would work best. If we wanted to do a formal build / installer for OS X we might wrap sqlite and -pcre into the distribution, I thought.
[…] Take a look over at OS X mosquitto “bites”… […]
Ah… I think the minimum requirement on sqlite is only documented in the packaging scripts now I think about it. That should change!
Does mosquito have enough functionality to be a drop-in replacement for rsmb? I’d prefer to be using an open source broker š
I think it does at this point – now that it can do bridging I think it’s more-or-less equivalent. Check with Roger š
Chris, there are still some differences, but I believe that these are the less used options in rsmb. The list of configurations options it doesn’t support are listed below – if any of these are important to you, please let me know!
autosave_on_changes
bind_address
clientid_prefixes
connection_messages
listener
retained_persistence
trace_level
addresses
idle_timeout
keepalive_interval
notifications
notification_topic
round_robin
start_type
threshold
try_private
mount_point
I’ve put this information onto the mosquitto page on the MQTT Wiki, for completeness (couldn’t see it in the docs, but you may have added this somewhere)
Good idea. The list is kept in mosquitto.conf – I’ve put a note on the wiki page to that effect.
Just visiting. Thanks for the Posterous subscribe on Reveln Consulting. Cool to see you sets of interests as well as what’s happening with your WordPress experiences. My webfolks have done some helpful things keeping me up to speed. Bookmarking you on Delicious. Bestest, –Deb
Is this still the best way to build an run mosquitto on Snow Leopard? If not, is there a new set of instructions for getting an MQTT server running on Mac OS?
Things have moved on a bit since Andy’s post. You’ll be better off using cmake to generate makefiles now. sqlite isn’t required either which helps.
Assuming you have cmake installed, try:
cd
mkdir build
cd build
cmake -i ..
make
The -i flag for cmake puts it into interactive mode – this lets you disable the optional sqlite support. It’s only there for backwards compatibility so if you’ve not used mosquitto on this machine before then you don’t need it.
There’s also a “homebrew” build for mosquitto at https://github.com/mxcl/homebrew/pull/3824 but I’m not sure how up to date it is.
Feel free to get in touch directly or on the mailing list if you have any problems.
[…] for download. While googling about this I found two post (Mosquitto and libmosquitto in OSX and OS X mosquitto ābitesāā¦) that were really helpful leaded me in the correct direction. In a nutshell what I needed to do to […]