Tag Archives: MQTT

Signals, messages, clouds… pigeons?

… AKA, how a flock of pigeons ūüź¶ is connecting my past, present and future technology interests.

If I look back at the past 10 years or so of my career, I believe there’s a continuum of interest – from my passion for¬†/ community building around application messaging systems (MQ, MQTT, and others), to building lightweight cloud-based services (Cloud Foundry and API platforms), and now, to working on what you can choose to see as a large scale, cloud messaging platform (Twitter).

Twitter is far more than that, of course. It’s also a platform that humans interact with more so than any of the other systems I’ve worked on in the past. From¬†e-commerce to¬†core banking systems, I’m pretty sure that none of the things I’ve helped to build in the past have had the same scale and impact in the world every day that Twitter has done. It also reflects my passion for people and communities – hell, I’m a guy with a History degree playing at being a technologist, and jumped into social media before the term even really existed, what else would you think I care about? OK, OK, yes I’ve also always been an early adopter of shiny new technologies… but my excitement with technology is almost always about what it enables us to do as humans every day.

So what’s the point of this post, other than to muse on this?

Well, Twitter also represents a nice confluence of my passion for the Internet of Things, sitting between those messaging systems and those cloud apps that I mentioned, and people.

Twitter is real-time, and live. By creating and curating my Twitter timeline, I¬†follow the people (@), topics (#), and well – the things – that I care about most. I receive breaking news, follow along with what my friends and colleagues are up to, and learn about what is happening in the world in real-time. It’s a platform for both messages, and signals. The engineering and support efforts required to support¬†all of this happening, with rapid¬†response times and a solid user experience, are pretty impressive – hats off to my colleagues in those departments.

Twitter is also conversational, which means that it can make a great home for all those conversational interactions we’re hearing about with the resurgence of bots, helpers, and personal agents. In fact, we can think back to not too long¬†after the Twitter API came about, to the Tweeting plant asking for water when the soil was dry, to see that these kinds of applications are no strangers to the platform.

I’ve spoken about these topics a number¬†of times over the past couple of years, most notably at our first Flight conference in 2014, where I covered flood alerting, plants, ferries, sports events, houses, earthquakes, and more!

Soon after starting work at¬†Twitter, I ended up helping a project to connect river sensors online. Right now, you can find local river stations in the UK on an interactive map, and then (if you’re interested) you can follow the ones that matter to you. It’s pretty cool stuff, and can help with flood alerting and monitoring as well as local awareness.

Perhaps¬†you’re interested in the environment around the Dublin Bay Buoy?

Talking of the environment, last year we ran a conceptual contest out of Twitter UK, inviting organisations to think about new ways to use the platform and build applications #PoweredByTweets. One of the winning entries was the idea that pigeons in London could be instrumented with tiny backpacks measuring air quality. At the exhibition of the winning entries in September, there was a display that mocked up how this could work.

Yesterday, the @PigeonAir patrol took flight for real in London, and for another day or two will act as a temporary set of mobile beacons that can report back air quality via Twitter conversations.

I love this – and quite honestly, I remember telling people how barmy I thought it was when we first started looking at it!

Another interesting data point to add to all of is that social signals from Twitter may be more effective than some FEMA models, according to recent research

So what’s next? Who knows. My mind is constantly racing with ideas, and others come up with far, far more interesting ones!

This is not a story about Twitter.

This is the story of how humans used technology in creative ways, to improve their lives and their environments.

 

Some random reflections on 2014

As time has moved on, I’ve lavished¬†less attention¬†on my blog, which is a shame… “back in the day” I enjoyed writing for it, and gained a lot of value from doing so. It’s of no particular surprise to me that I’ve spent less time writing here in 2014 than in any of the previous years; but it is a regret. I blame my schedule, a general change in the way I interact online, and a lack of inspiration. Actually, that last one isn’t quite true: I’ve often been inspired, or felt the need to blog, but have found myself mentally blocked. I need to get over that!

Anyway… 2014, looking back… a little bit of a year in review.

LEGO AndyThe major life change this year was my move to Twitter, which has been very exciting and energising. I’m thrilled¬†to have been invited to work with a team of exceptional people under Jeff Sandquist. In particular, this year I’ve had the short-lived opportunity to work with three brilliant and talented guys¬†I want to say “thank you” to, for making my transition to my new role such a pleasure: Taylor Singletary, Sylvain Carle and Isaac Hepworth. A special shout-out too to my close friend and colleague based in London, Romain Huet, without whom I would have found the past nine months much less fun or¬†easy-to-navigate! The whole team has been just amazing to work with, as have all my wonderful colleagues at the Twitter office in London #gratefulpipes

The work we’re doing on the Developer and Platform Relations team at Twitter is something I’m incredibly passionate about. Connecting with the third-party community and acting as the face and voice of Twitter with those developers, listening to them and responding to their concerns, is the reason I joined the¬†company.

I’ve been involved in the launch of a couple of APIs (most notably the Mute API), and I’m getting to work on much¬†of the external API surface, which plays well with my background and developer experience. We’ve completely relaunched our developer-facing website and forums in the past few months, which the whole team has worked hard on. I’m happy to see the focus¬†of discussion on the developer forums substantially improved now that we’ve moved to the Discourse platform – the user experience is far better than we had with the previous solution.

Most importantly, this past quarter we launched Fabric, our new free mobile SDK and platform for iOS and Android, and delivered a swathe of improvements to the developer experience for mobile enthusiasts. We also ran our first mobile developer conference, Flight – I was excited to be there, and I’m looking forward to seeing that experience continue in 2015.

My background in the Internet of Things and MQTT space has partly carried over into my new life at Twitter, and I’ve had the opportunity to speak at a couple of events (including Flight) about how Twitter’s platform plays into that space. However,¬†I’ve substantially stepped back from playing a major role in the MQTT community this year; a decision in part driven by the need to refocus on my new role, partly due to¬†some personal hostility and “burnout” with a couple of specific issues, but mostly because – I’m no longer “needed”! It has been incredibly satisfying to see the MQTT community grow over the past few years. The standardisation of the protocol at OASIS, the large number of implementations, and the ability of many other much smarter people to pick up the kinds of speaking engagements I was previously doing as a matter of course – all of these things make me immensely proud to have helped to lay the foundations for the success of that community over the past six¬†years or so.

I’ve also been very happy to see the success of the Cloud Foundry platform and the people involved – having devoted the previous two years of my career to that nascent Open Source community, it is just fantastic to see it take off and the Foundation get started. Nice work to everyone involved.

I’ve again thoroughly enjoyed my speaking opportunities this year, and the chance to broaden my range. Obviously that has included a lot about the Twitter API and developer platform, and lots again about IoT; but I’ve also spoken on wearables, developer advocacy, and API management. I’m very happy that I got to be a part of the first Twitter Flight conference – one of my speaking career highlights.

Personally, I’ve tried to stay healthy this year (no heart scares, no falls…!); although my travel schedule has been demanding again (TripIt tells me I covered¬†66613 miles in the air). That did at least include a couple of trips for fun, rather than being all about¬†business:-)

The next year looks to be busy with more events to speak at (and organise!), and much more to do around the Twitter platform. As an historian, a sociologist and someone with a keen interest in the intersection of technology and people, I’m very excited to be a part of this¬†wave of change.

Happy New Year – here’s to 2015!

Latest interviews, and upcoming speaking schedule

Just a quick note to comment that I’ve added links to two recent interviews on the media section of my Speaking and Media page:

I’ve been at QCon London this week and also did a couple more interviews, again one on each of these topics – it may take slightly longer for these to emerge, though. Let me know if you have any feedback on these, always interesting to hear!

It may seem hard to believe, and indeed looking back at my schedule it hardly seems it, but I’ve been avoiding too many speaking gigs since an annoying incident back in December and a need to take a break! Coming up in the next few months, apart from the job change (!), I’m back on the road and I’ll be speaking at the following events:

A little bit of Spring and MQTT

I’ve been involved with Spring (the Java framework, not the season…) for a couple of years now – since I joined VMware in 2012 through the former SpringSource organisation in the UK – and I’ve remained “involved” with it through my transition to Pivotal and the framework’s evolution into Spring 4 / Spring.IO under Pivotal’s stewardship.

To be clear, although I’ve been a “Java guy” / hacker through my time at IBM, I have never been a hardcore JEE coder, and my knowledge of Spring itself has always been limited. My good buddy Josh¬†(MISTER Spring, no less) has done his best to encourage me, I briefly played with Spring Shell last year with @pidster, and the brilliant work done by the Spring Boot guys has been helpful in making me look at Spring again (I spoke about Boot – very briefly, as a newcomer to it – at the London Java Community unconference back in November).

Taking all of that on board, then, I’m still an absolute Spring n00b. I recognise a lot of the benefits, I see the amazing work that has gone into Spring 4, and I’m excited by the message and mission of the folks behind the framework. I would say that though… wouldn’t I?:-)

[considering this is my first blog post in a while, I’m taking a while to get past the preamble…]

This week, I chose to flex my coding muscles (!) with a quick diversion into Spring Integration. With a long history in WebSphere Integration back in my IBM days, this was both something of return to my roots, and also a learning experience!

With the new Spring.IO website (written with Spring, and hosted on Pivotal Web Services Cloud Foundry, fact fans!), the Spring team introduced the new Spring Guides – simple and easy-to-consume starter guides to the different technologies available in the Spring family. I knew that the team had added MQTT support via Eclipse Paho, so I thought I’d take a look at how I could modify a Spring Integration flow to take advantage of MQTT.

Incidentally, there’s complete reference documentation on the MQTT support, which is helpful if you’re already familiar with how¬†Spring Integration works.

The resulting simple project is on Github.

Once I’d figured out that it is useful to ensure the latest versions of the various modules are listed as dependencies in the build.gradle file, it wasn’t too hard to adapt the Guide sample. In the example, the docs lead a developer through creating a new flow which searches Twitter and then passes tweets through a transformer (simply flattening the tweet sender and body into a single line of text), into an outbound file adapter.

The bulk of the changes I made were in the integration.xml file. I wanted to replace the file output with the tweets being published to an MQTT topic. To do that, I added the int-mqtt XML namespace information to the file, and configured an outbound-channel-adapter. It was also necessary to add a clientFactory bean configuration for a Paho MQTT connection. You’ll notice that, by default, my example posts results to the test broker at Eclipse (iot.eclipse.org port 1883 – an instance of mosquitto for public testing). Full information on how to build and test the simple example can be found in the project README file.

Thanks to my colleague Gary Russell for helping me to figure out a couple of things, as I was a Spring Integration newcomer!

M2M Community at EclipseCon

Day 2 of EclipseCon 2013 and we’ve already been seeing some strong interest in the M2M community and the kinds of projects we have been working on!

The story so far

There were two ¬†M2M events on day one. The first was the M2M tutorial featuring some real hardware (Raspberry Pi and Arduino), using the Koneki Lua Development Tooling to deploy scripts to the embedded Mihini runtime on the Pi to drive and take input from sensors over Modbus. I cheated a little and mixed in a bit of MQTT using the Paho Lua client, to have my kit publishing temperature and light data to a broker instead:-) It is early days for Eclipse Mihini, but it was a really slick demo and tutorial, and I’m looking forward to playing with this a lot more.

The evening Birds of a Feather session gave a group of us a couple of open hours to hack around with our Pi and Arduino kits. My own efforts were slightly thwarted by a European soldering iron and a 110V power supply, so I wasn’t able to assemble the add-on board I wanted for my Raspberry Pi, but I’ll get back to that in the future. For those looking to explore their new Raspberry Pis and hardware / GPIO interaction from Java, take a look at the pi4j project.

Building a community

One useful “war story” that was told today was in Benjamin Cab√©’s Building the M2M Community talk. The slides will follow, but it was good to hear about the progress of the community and projects involved (yes, ok, I’m a Committer on one of them…) and also to hear about his “Community Manager Toolbox” for tracking and responding to community discussions. I use a very similar set of tools when I engage with the Cloud Foundry and MQTT communities (and others that I’m involved with).

Here’s a summary:

  • Slideshare – monitor who favourites and downloads slides related to your project
  • Twitter – monitor hashtags, engage complainers
  • YouTube – screencasts, demos, build channel to aggregate content
  • StackOverflow – answer questionss (even old ones), find answers that need to be complemented)
  • Github – monitor forks, stars, look for people using your tech (include Gists)
  • Google Alerts
  • IFTTT – e.g. if Eclipse wiki changes get Gtalk notification recipe
  • Google Analytics for your site
  • LinkedIn – monitor groups, post news and relay blog posts

In short: Engage the community. Be very public about what you are working on and where the roadmap is going. Get management buy-in for your projects to persuade them of the value of opening them to the community (use metrics to demonstrate take-up).

Working on standards

The other thing that has been going on today “behind the scenes” at EclipseCon has been the first meeting of the OASIS MQTT Technical Committee. It is exciting to see that Richard Coppen and Raphael Cohn have been elected as co chairs of the group, with seven sponsoring organisations from across the technology and messaging market and 35 publically-visible members – lots of interest in MQTT!