Tag Archives: development

A little bit of Spring and MQTT

I’ve been involved with Spring (the Java framework, not the season…) for a couple of years now – since I joined VMware in 2012 through the former SpringSource organisation in the UK – and I’ve remained “involved” with it through my transition to Pivotal and the framework’s evolution into Spring 4 / Spring.IO under Pivotal’s stewardship.

To be clear, although I’ve been a “Java guy” / hacker through my time at IBM, I have never been a hardcore JEE coder, and my knowledge of Spring itself has always been limited. My good buddy Josh (MISTER Spring, no less) has done his best to encourage me, I briefly played with Spring Shell last year with @pidster, and the brilliant work done by the Spring Boot guys has been helpful in making me look at Spring again (I spoke about Boot – very briefly, as a newcomer to it – at the London Java Community unconference back in November).

Taking all of that on board, then, I’m still an absolute Spring n00b. I recognise a lot of the benefits, I see the amazing work that has gone into Spring 4, and I’m excited by the message and mission of the folks behind the framework. I would say that though… wouldn’t I? :-)

[considering this is my first blog post in a while, I'm taking a while to get past the preamble...]

This week, I chose to flex my coding muscles (!) with a quick diversion into Spring Integration. With a long history in WebSphere Integration back in my IBM days, this was both something of return to my roots, and also a learning experience!

With the new Spring.IO website (written with Spring, and hosted on Pivotal Web Services Cloud Foundry, fact fans!), the Spring team introduced the new Spring Guides – simple and easy-to-consume starter guides to the different technologies available in the Spring family. I knew that the team had added MQTT support via Eclipse Paho, so I thought I’d take a look at how I could modify a Spring Integration flow to take advantage of MQTT.

Incidentally, there’s complete reference documentation on the MQTT support, which is helpful if you’re already familiar with how Spring Integration works.

The resulting simple project is on Github.

Once I’d figured out that it is useful to ensure the latest versions of the various modules are listed as dependencies in the build.gradle file, it wasn’t too hard to adapt the Guide sample. In the example, the docs lead a developer through creating a new flow which searches Twitter and then passes tweets through a transformer (simply flattening the tweet sender and body into a single line of text), into an outbound file adapter.

The bulk of the changes I made were in the integration.xml file. I wanted to replace the file output with the tweets being published to an MQTT topic. To do that, I added the int-mqtt XML namespace information to the file, and configured an outbound-channel-adapter. It was also necessary to add a clientFactory bean configuration for a Paho MQTT connection. You’ll notice that, by default, my example posts results to the test broker at Eclipse (iot.eclipse.org port 1883 – an instance of mosquitto for public testing). Full information on how to build and test the simple example can be found in the project README file.

Thanks to my colleague Gary Russell for helping me to figure out a couple of things, as I was a Spring Integration newcomer!

About these ads

First week with VMware and Cloud Foundry

Hello, VMware.

Well – that was bracing!

I don’t expect to be posting “week notes” like this on a regular basis, but as a one-off it seems like a nice way to encapsulate just how much happened in the first week of my new role.

Tuesday

Joined VMware. Met new colleagues in London office. Started to look at the User Account and Authentication component in Cloud Foundry. Ran samples against cloudfoundry.com, modified the documentation. Issued first GitHub pull request :-)

Wednesday

More hacking on samples. Updated the Cloud Foundry Google+ page. Agreed to present Cloud Foundry at the London Real-Time hack weekend for the RabbitMQ guys. Watched (and tweeted) the live webcast of the Cloud Foundry first birthday event in San Francisco – very exciting news! “more clouds, more community, more code“, including a broader range of partners, a new governance process around the cloudfoundry.org Open Source project, and the announcement of BOSH being released to the community, too – multi-cloud deployment, here we come!

Thursday

Trip to the VMware Frimley office for some HR stuff. First call with full Developer Relations team for event planning. Briefing with the two directors I work for. Nothing to see here, move along…

Friday

Setup new laptop (custom order from Apple so there was a small delay). Prepped for demo at London Real-Time. Started making a lot of noise on VMware Link (aka Socialcast, the social sharing/discussion platform) internally :-)

Saturday

My first speaking gig – a Lightning Introduction to Cloud Foundry, taking Chris Richardson’s much more comprehensive Boot Camp presentation and cramming the essentials into ~15 min including a live demo, for a bunch of hackers at London Real-Time.

Oh, and it was caught on video.

Four or five of the hacks ended up running on Cloud Foundry, too, which I think was rather nice :-) I was also interviewed on realtime and the importance of cloud at the event, but I’ve not seen that video appear just yet.

Monday

More laptop setup, HR stuff. Prep for Scala Days. Started to improve a sample app (Ruby/Sinatra) I’d used in the past by adding Twitter Bootstrap and restructuring the code.

Tuesday/Wednesday

Scala Days in London – helping to man the sponsor stand talking about Cloud Foundry, answering questions, and meeting many new colleagues from the US who were presenting on Spring, Scala, and Cloud Foundry (including an announcement that Play 2.0 framework support and standalone apps are coming to cloudfoundry.com Real Soon Now). Recorded a podcast interview with Uhuru about what a Developer Advocate does.

Summary

I’m pleased that I was able to be so productive so quickly. I’d had a little previous experience with Cloud Foundry but it’s a testament to how quick it is to learn the basics and get moving that I was able to rapidly start playing with a bunch of code. It was also exciting to be out on Github on my first day – not something I could have done in a former life… it’s nice to be working in an organisation that is innovating with Open Source at this level.

There’s much to learn, and to be honest, a couple of the key aspects of Cloud Foundry actually make it more challenging (and interesting) for me to get to grips with. It’s open, and with BOSH, can potentially target different IaaS offerings (initially vSphere and the beginnings of AWS support; a hackathon yesterday aimed at adding OpenStack to the list) – so suddenly I need to know about those. It’s a polyglot platform, which means I need to broaden my language knowledge – I’m already making a start on Ruby and node.js, to complement existing Java and PHP knowledge.

It’s also exciting to learn more about what VMware does, the layers of technology that they offer, and their vision. My previous experience has primarily been with the desktop virtualisation technology, but there’s a huge and vibrant community around the server-side virtualisation tools, and products like Socialcast, Sliderocket and Zimbra in the collaboration space too.

There’s a lot of exciting stuff happening in this space. It’s thrilling to be here. Thanks to all of my new colleagues for a warm welcome and support – looking forward to working with you!

Eclipse Paho gets started…

Since the announcement of Eclipse Paho (an Open Source project under the Machine-to-Machine umbrella at Eclipse) there has been a fair amount of excitement in the MQTT community about the availability of IBM’s C and Java client code under an Open Source license.

The initial proposal and setup stages have taken a little while, but this week the initial availability of the C client code was announced on the Paho mailing list (Java will follow shortly).

PahoHALF

Paho Quickstart

This is not intended to be a comprehensive guide – better documentation etc will emerge over time – but I thought I’d post a quick guide as a kickstart for anyone wanting to give it a look. I did this on 64-bit Ubuntu 11.10 – similar steps will apply on other Linux or UNIX platforms (note, the initial code contribution has a Makefile with rules which should work on UNIX, Windows, or z/OS).

Install the necessary packages to build code. NB git is for grabbing the source from Eclipse; build-essentials is a metapackage providing gcc etc on Ubuntu; and doxygen and optional graphviz are used for generating the documentation.

sudo apt-get install git build-essentials doxygen graphviz

Get the code from the git repository:

git clone git://git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.c.git

Quick build for the client library and documentation:

cd org.eclipse.paho.mqtt.c.git/src
make -f ../build/Makefile all
doxygen ../doc/DoxyfileV3ClientAPI

Once these commands complete, you should be left with subdirectories called <platform> and docs. In my case, <platform> was 64-bit Linux, so I had a binary at linux_ia64/libmqttv3c.so. There’s no “make install” rule at the moment, nor is there a rule to compile the docs so I had to run doxygen directly. In the future it would be nice to automate all of that, and also to build some test applications.

Opening docs/html/index.html in a browser reveals very nice documentation describing the client library, including some examples of how to use it. For example, in docs/html/pubasync.html there’s a complete listing for an asynchronous publisher application. I extracted that code into pubclient.c and decided to check that it worked!

gcc -Wall pubexample.c -L./linux_ia64 -lmqttv3c -lpthread -o pubexample

That command successfully built a binary called pubexample. All I needed to do was test it. The sample application assumes that an MQTT broker is available on localhost port 1883 – if you want to change that, simply modify the value of the static variable ADDRESS in pubexample.c – in my case I simply apt-get installed the mosquitto and mosquitto-clients packages onto my system, but I could equally have unzipped and run Really Small Message Broker – both start on port 1883 by default if not given alternative configuration.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./linux_ia64/
./pubexample
Waiting for publication of Hello World!
on topic MQTT Examples for client with ClientID: ExampleClientPub
Message with token value 1 delivery confirmed

It was trivial to test that a subscriber (mosquitto_sub in my case) also received the publication. Job done!

Getting involved, and other news on Paho

I mentioned that the Java client contribution should appear soon. One other piece of news this week is that the project’s sandbox broker implementation – based on mosquittohas been spun up. That was posted on the Paho mailing list, and if you want to get involved you should definitely subscribe to that; start to track the Eclipse Bugzilla for Paho; watch the Paho wiki; keep an eye on the source repositories; etc.. I’m already thinking about getting an OS X build rule sorted out. If you want to test your sample code now, you’ve got the option of a local broker, the Eclipse Paho sandbox, the mosquitto sandbox, or various other implementations.

Oh – and please leave a comment on this post if you find this information interesting, or want to discuss where things are with Paho. I’ll be hanging out on the mailing list as well.

What about Bob? (or Andy, even!)

Well, although I’ve left IBM, I’m delighted that MQTT is now going Open Source – in fact that was one of the things that I really wanted to help to achieve before I moved on. I am really pleased that I will be able to continue to contribute to both Paho and the broader Eclipse M2M Industry Working Group. I’ll be helping to update the mqtt.org community site, and heading over to EclipseCon in Virginia in a couple of weeks’ time to talk about M2M and work with our friends from the Koneki project. If you are attending EclipseCon please come say hi to me – and you may be interested in Wes Johnson’s session on MQTT and Eclipse tools.

There are very cool times ahead!

Community, telephony, and prototypes: Make-a-thon

Warning: long post! the first in a series covering some of the events I’ve attended or been involved with lately.

Background

At January’s London Internet of Things meetup, I had the privilege to hear Haiyan Zhang speak with passion about various topics, including how she had collaborated with hackspaces in Japan in the aftermath of last year’s earthquake and subsequent nuclear disaster.

It was only the first time I’d come across Haiyan, so I was surprised but delighted that she invited me to the OpenIDEO Make-a-thon this past weekend, after tweeting about events like the London Green Hackathon.

I had only a vague idea what to expect of the Make-a-thon. When I saw some of the project briefs being published ahead of the day, I knew that it would be a little different to hackathons and other tech events I’d been to in the past. The briefs spanned issues such as improving local communities, bike safety, and several supporting campaigns by Amnesty International to use technology to support human rights activities.

My initial impression that it would not be a “run of the mill” tech event was reinforced when I arrived at the IDEO offices in Clerkenwell on Friday afternoon – it was a very different crowd to the ones I typically encounter – full of product designers, makers, human factors specialists, as well as web coders and developers. I rocked up with a bunch of Nanodes and other electronics with a vague thought of doing something hardware-related, but in the event I didn’t get that far!

IDEO’s typical approach to design revolves around prototyping and directed brainstorming, and in the event we divided into 8 teams of around 6 each, with diverse skills but with common interests around the briefs on offer. Friday afternoon was spent first understanding and exploring the brief, and then rapidly prototyping a rough idea before presenting it to the rest of the group. Saturday was spent refining the idea and producing an “experience prototype” which was intended to have been tried out “in the real world” if possible.

Evolution of the “Karma Phone”

Several of the briefs interested me, but I joined the team focused on the concept of Postcode Gangs – how could we build something to develop and improve community facilities within a postcode – essentially an arbitrarily-delineated area – in London? We spent some time brainstorming ideas around what “makes” a community before needing to rapidly decide on something to build for our rough prototype.

IDEO Makeathon

“What if” – there was a ringing phone in the middle of the street – and on the end of that phone, someone who knew something, had something to offer, or who was needed something? “What if” – we could create a new local hub with current and historical information about an area, enabling people to explore and meet their neighbours?

So the first prototype of what came to be called Karma Phone involved a lamppost (named, erm, Dan!) with a phone on it, which would randomly ring as people passed by – people could call it with a need and that others could then try to address. On the other side of the lamppost (also known as, Dan’s back) we imagined a large touch display with information about current events, realtime information, historical maps, and so on.

Karma Phone – The Outcome…

The team changed overnight, as Hayley and James were not able to stay for Saturday, Lydia joined us, and Victoria could only be involved for a short time on Saturday. We weren’t all convinced that a ringing phone would be answered, that the system wouldn’t be abused, that there wasn’t a social barrier around providing home address and asking for help, etc. How could we get an actual phone into the street and ringing, too? So, Saturday morning involved some rapid rethinking of what we wanted to build!

We settled on what turned out to be a subtle evolution of the original idea – a public phone which could act as a hyperlocal information service and skills exchange.

While we were brainstorming how to hack a physical phone, run a long cable into the street, use a mobile chained to a metal box, etc etc I remembered Twilio, which I’d been following for a long time, but never had the chance to hack on in anger. Within about 30 minutes I’d demonstrated the ability to initiate calls between two users from a web page, to the rest of the team.

Karma

Steve and Dan set about implementing the web UI; Tim started working on a physical enclosure; Victoria and Lydia managed to source a real “traditional” phone handset; and I remained hard at work writing PHP to talk to Twilio.

A couple of minor wrinkles along the way:

  • network issues meant that I had to use Tim’s phone to tunnel through to my webserver’s console, since it was apparently impossible via the event wifi. Evidently IDEO had just had a network provider change, so it was just an awkward time, but I lost some time fiddling with hosting in the early part of Saturday.
  • at a certain point on Saturday afternoon, I realised that attempting to call from the Twilio web client on the iPad was never going to work… since it requires Flash. I thought of a number of workarounds, but the one that finally stuck was that we were able to use Skype on the iPad, and use the skype:// URI scheme to launch the app from the web client. It wasn’t seamless as we needed Skype credit, and also had to tap an extra “call” button in order to start the call, but it was good enough for a prototype.
  • I’d wanted to make the web app, a standalone launchable web app on iOS. Weirdly, adding the usual meta tags to the page header to instruct iOS to treat the app as standalone launchable, meant that it was no longer possible to invoke Skype from within the web UI… so I backed off from that idea. The only cosmetic issue that presented was an inability to hide Safari “furniture” like the header, but that wasn’t a big problem for a prototype.

Here’s how the final system hangs together:

KarmaPhone-overview.png

Impressive Outcomes…

I spent so long coding and tweaking on Saturday (the commented and documented code is here – ignore how short it might seem – it was an intense few of hours!) that I missed most of the physical assembly. Tim and Dan did an amazing job of creating an enclosure for the iPad and handset. It might have been made from foam board, a box folder, and vinyl, but the final result was beautiful. And most importantly – it was fully functional!

IMG_5753

We would have loved to get the prototype out on the street for public testing (I suspect none of us more than Steve and Lydia!), but time worked against us. The final experience prototype was presented as a live demo with willing audience volunteers – one example call going to an answering service, and the other redirected to the local expert on Scotch Eggs (Tim!).

I’m happy to say that Karma Phone won Best Digital Prototype at the event, and I was (apparently!) Best Tweeter. Nice accolades :-)

So – conclusions? I really enjoyed the way we worked together as a team of very unique and different talents; and seeing the Karma Phone prototype realised so brilliantly. However, I also think the experience of the Make-a-thon was humbling… listening to the experiences of people illegally detained abroad, and seeing some truly brilliant ideas from all 7 of the other teams, was wonderful.

A huge thank you to everyone involved in the first IDEO Make-a-thon – a really unique hackday. The IDEO team in particular looked after us brilliantly, with superb facilities, a great welcome, and more-than-adequate quantities of the hacker staples (coffee, sweets, pizza and beer).

Read a full recap including information on all of the project briefs on the OpenIDEO site. There’s a gigantic set of photos from the IDEO team, and a much smaller one from me shot on an iPhone at lower quality.

Tim, Dan, Hayley, Victoria, Steve, James, Lydia – Thank You. It was a pleasure!

All of the other teams – you rocked. You did great things. I salute you!

Podcasting with UUPC (and podcats)

On Monday I was surprised and delighted to be asked to join the team from the Ubuntu UK Podcast (aka @uupc) as a guest presenter. I’ve been a listener and friend of the show for a long time, I knew several of the regular team, and since I work not far from “Studio A” it was fairly straightforward to jump in at short notice.

I had a blast recording with Tony, Laura and Mark! The amount of work and polish that goes into each show is fantastic, I can tell you. I was also very impressed with their studio setup. My own podcast is recorded using Skype, and although I do own a small mixer, it’s really nothing compared to Tony’s much larger desk. Each of us had a proper mic, too. Oh, and there was a Very Large Clock. There was also a small experiment in streaming to a small audience (the magic-fu was served up on Twitter and in the #ubuntu-uk-podcast IRC channel, so you should hang out in those places in case it happens again!).

Most importantly, we were supported by the UUPC PODCATS.

I gave a brief intro to myself at the start of the show. If you’ve just discovered me via my guest appearance on UUPC then feel free to follow me on Twitter or read more about me here on my site.

Below, I’ll just recap and expand on my Linux credentials, because it’s not something I’ve really blogged about before.

My Linux experience

I started out using Linux with some early SuSE version in the late 90s just after leaving university. Then I switched to RedHat, which was in the phase of spinning out the Fedora project, and I did some packaging for a few projects there for a while. I helped out on a bunch of projects around that time, like OpenUT (the initial Unreal Tournament port to Linux – there’s a special thanks to me in the credits for the Linux version!), the brilliant Anjuta IDE, and the Bible software GnomeSword (which is now known as Xiphos). I spent a lot of time helping to do things like triage bugs, coordinate releases, polish the UI for GNOME apps like Anjuta, and basically to some extent “project managing” alongside the actual project leaders, freeing them up to code on their projects while I took on a more technical coordination role communicating to different developers, helping with test, etc.. At the time I was a middleware developer for a large company so I had an appreciation of how things like CVS worked, how to do releases and release notes, working with users etc. – things that are sometimes missed on OSS projects, although all of these things have become better over time. It was a great way for me to deepen my UNIX skills and hone my development abilities too. Plus, I built some great relationships and friendships from working with the community.

Times change, and I had to take a step back from all of that for quite a long time. Although I had a Linux box at home as a server (on dial-up for a long time… yikes!), I otherwise wandered the wastelands of Windows XP and then got a Mac. Fundamentally I believe I see the good and bad aspects in most systems; I believe it’s important to at least try something, and not dismiss it; I did go through a strongly anti-Microsoft phase but with things like XBox 360 Live, a few items in the “Live” family of software, and their new phone operating system, I do have respect for what they’ve done. I struggle a little on the whole “openness” thing – my tendency and first preference is absolutely towards open standards, open source and free collaboration, but then along comes Apple with stuff that just… works… and is so… shiny… and… I’m almost willing to suspend that view. And then I smell coffee and come back to my senses :-)

So what am I doing now? Well, last year I switched to Ubuntu on my work laptop, a Lenovo Thinkpad, and I’ve been through Jaunty, Karmic, Lucid and now Maverick. I do NOT have a Windows partition, I run my whole work life in Ubuntu. I’ve got a Viglen MPC-L (previously featured on UUPC, of course!) running home and weather monitoring with some software called MQTT which I’m involved with as part of my job. I run Linux on my “set-top box”, an Acer Aspire Revo inspired by Popey, with Boxee and other bits. I have a netbook, a first-gen Acer Aspire One running UNE Lucid until they sort out Unity to a point where I feel it’s usable. And, as I mentioned on the show, I’ve recently picked up a cheap (actually, total bargain) Android handset as a development platform- my main phone is one of the Apple devices, sorry! The new phone is an Orange San Francisco, which is a rebadged ZTE Blade – I’ve flashed it to run Android 2.2 and moved a bunch of apps into SD storage, and it’s a lovely little device. Apart from that, I help to do a few things for internal Linux apps we use at work on Ubuntu, and I buzz around Launchpad largely helping to improve quality via bug reports etc. Oh, and I’m lined up to speak at LinuxConf AU in January! (very excited about that!)

So that’s me. I do Linux, I do Ubuntu, and I do a whole bunch of other stuff. Thanks again to the UUPC gang for not throwing me out of the studio! And thank you, dear reader, for listening…