Tag Archives: hursley

Connectivity and Integration podcasts

As well as being WebSphere Messaging Community Lead out of IBM Hursley right now, I’m also part of what we refer to as our “Connectivity and Integration” organisation (middleware… plumbing… the hidden inter-application messaging and adapter stuff, ensuring that systems can talk to one another reliably). Much of what we do in Hursley, and the software that we develop there, is part of the Connectivity space. It’s the software that joins up all the pieces of a Smarter Planet, and it’s an interesting space for a techie like me.

We thought it was about time to talk about some of the features that are in our WebSphere Messaging products – WebSphere MQ, Message Broker, and the family of software that fits around them. So, my colleague Leif Davidsen and I sat down and recorded a series of podcasts. Each episode zeroes in on a specific feature or capability, such as high availability, or telemetry, or security – you get the idea.

As we were talking, Leif and I were trying to keep the discussions bite-sized (about 10 minutes at a time); highlight things that users might not have heard about before; be interesting to administrators and developers as well as to architects; and we tried not to use too much “marketingese” – although I reckon you might spot that in some of the podcast episode titles! 🙂

You can start to subscribe to the Connectivity and Integration podcast series right now in iTunes or add the RSS feed to your favourite podcatcher. There should be some web content and show notes with links and references to follow soon – watch out for those, I’ll tweet about them and update this post when I know more.

NB did you check out my first and second columns for Sphere yet? More to come soon, and I’m hoping to join the GWC Lab Chat series for a future episode as well. Cool stuff.

Advertisements

Inside the Sphere

One of the things I’ve taken on this year is a regular writing assignment for the Global WebSphere Community. If you’re a member of the community (and if you use or are interested in WebSphere products, you probably should join – it’s free to sign up – here’s a link to my profile), depending on your profile preferences you may have just received the first edition of The Sphere Journal, a new online newsletter from GWC.

As the editor, Bruce Lynch, writes:

Welcome to the premier issue of The Sphere Journal Online. We will use this space to bring you opinion, news, and technical information on how to use your current WebSphere products more effectively and help you make more informed decisions about WebSphere products you should be deploying in your organization.

My monthly column is the WebSphere Deep Diving Instructor where I’ll be sharing news from inside the labs about the “hot” areas practitioners might want to explore more deeply, and the areas where I’m hearing the most interest or difficult technical questions from customers and the community. My first column does focus on my key area of messaging, but we’ll certainly broaden out from that over the course of the year.

[somewhat confusingly, there’s also a section called The Message Queue, but that’s a news section rather than being specific to WebSphere MQ!]

Links to places you can find and follow the GWC:

Message Broker goes Hyper with new updates

I’m excited. About a week ago we released the latest update to WebSphere Message Broker, version 7.0.0.2 (also known as fixpack 2 for version 7), as well as the new Hypervisor Edition of the product.

Version 7 and “what’s new”

I’ve mentioned the new drops of WMB in passing during the year, and I’ve spoken about them in detail as I’ve visited customers and conferences during 2010. So far though, I don’t think I’ve written about it at length this year. Considering I’ve written developerWorks articles and Redbooks on the subject in the past, it’s something of an omission that I need to fix! The version 7 release has had four overarching themes: Universal Connectivity for SOADynamic Operational Management; Platforms, Environments, and Performance; and (perhaps most importantly) Simplicity and Productivity.

I said I was excited, and that’s for two reasons, I think. Firstly, as a technical integration developer, I’m constantly interested in the new function being introduced to enhance the capabilities of the product – I’ll list out a few of those in a moment, but the number of new nodes and functions that have been added to enable you as a developer to get at your information, connect to your services and endpoints, and transform your messages, is just fantastic. Secondly, under that theme of Simplicity and Productivity, the product has been hugely streamlined, and with the usability enhancements and patterns support that have been added, it is faster than ever to get going even as the function becomes richer.

There’s too much to talk about that has dropped into the product capabilities since version 7 became available just over a year ago, but to whet your appetite you’ll find that the 2 updates in 2010 included solidDB support, CORBARequest nodes, a DatabaseInput node, FTE nodes for coordinating or responding to file transfers in WMQ FTE, EmailInput and FileRead nodes, a JSON parsing domain and RESTful web service examples, performance profiling, JDEdwards nodes… this is a team that never stops delivering fantastic, high-quality content. The WMB 7.0.0.2 release notes and details are available on the IBM Support pages, you can check out MGK’s summary of the release at MQSeries.net, or you can jump to the What’s New section in the Infocenter to catch up on 12 months of enhancements!

Patterns and Communities

One of the big items that has been delivered in version 7 has been Patterns – the ability to take a predefined operation or template, fill out a few parameters to customise it for your environment, and deploy a working set of message flows. Ant Phillips has just blogged about the enhancements in patterns authoring in the latest release, and the creation of the new Patterns Community which is over at MQSeries.net. If you saw Ant at any of our conferences this year you’ll know what a great speaker he is and how cool the demos of this technology are.

I’m excited about this, as I know it can help to maintain consistency, learn good practices, and speed along development – isn’t it much easier to build something when you have a framework to follow? I know one of the first things I tend to do when learning something is to look for a good example, and then as a good citizen I like to share what I’ve done to help others, when I can. If you’re a Broker developer I hope you’ll be keen to share and learn within that community. I know Ant and the rest of the team will be eager to listen to your feedback, as they have been doing actively for the past couple of years. There is a nice introductory article on how to create your own patterns on developerWorks. Get contributing!

By the way, I love this paragraph from Ant’s post announcing the community, as it echoes and reinforces what I’ve been talking about in my role as WebSphere Messaging Community Lead. I’m sure he won’t mind me borrowing it:

With this in mind you might be interested in a new global pattern community – mqseries.net has added a pattern community where you can find, download and share patterns. We will be putting some very cool example patterns up over the next few weeks to help get it started. Why mqseries.net? Well communities are all about people, and mqseries.net is where the Broker community go to find answers.

Hypervisor Edition

The final thing I want to mention is that WebSphere Message Broker can now be deployed into a virtualised environment from the WebSphere CloudBurst Appliance. This is cool, particularly when you start to see some of the tie-ins with things like patterns and scripting which enable you to customise the broker instances. A video is worth at least another three paragraphs of waffle though, so I’ll hand over to my colleagues…

Enjoy.

MQTT: the Smarter Planet Protocol

I’m at the SHARE conference in Boston this week. Earlier today I gave a talk about one of IBM’s significant software announcements of this year: the forthcoming WebSphere MQ Telemetry feature.

Observant viewers / readers / followers will know that I’ve been at IBM for nearly 10 years now and throughout that time I’ve spent a lot of time working with our WebSphere MQ (yes, aka, MQSeries) family of products. That family includes WebSphere Message Broker, and more recent extensions like the WMQ File Transfer Edition. Now, it has been formally extended to include first-class support for MQ Telemetry Transport, otherwise known as MQTT.

I want to spend a little bit of time talking about this, partly because I haven’t posted all that much here on my blog for a while now, and mostly because I’m hugely excited by this direct and long-overdue convergence of two of our messaging technologies. It plays directly into the Smarter Planet vision that IBM has been talking about over the course of the past couple of years, so it is worth understanding how this all fits. I apologise in advance that this may be a longer blog post than my average! I’ll also warn that I may come across as just a teensy bit of a fanboy… 🙂 as usual, remember that all thoughts expressed here represent my own opinions and understandings, and are not necessarily also those of my employer.

The Smarter Planet vision

So let’s kick off by asking: what on earth is this Smarter Planet concept that IBM has been discussing, and which you’ve likely seen promoted on posters and billboards as you travel around and go about your daily life? Well, it’s a very simple but very exciting idea, founded on three pillars: Instrumented, Interconnected, and Intelligent.

What is that all about then? Well, here it is, broken down, in my own words.

Firstly, there are a heck of a lot of devices around these days… from teeny tiny sensors and RFID tags which may stand alone in a system, through smartphones, GPS location-aware devices, laptops, and embedded systems. These devices typically have enough computing power to at least gather and transmit data, and some of them have enough to respond to requests to modify their behaviour. This is where we recognise that the world is increasingly Instrumented. I won’t bore you with statistics but here’s one of the numbers that tends to blow people away: there are 1 billion transistors in the world for every one of us. Secondly, these devices are nearly all natively “online” to some extent, and most will have an Internet address of their own (even if the connections are not always super-high bandwidth, always-on, or reliable). So, they are becoming Interconnected, either directly to one another across local networks, or indirectly via clouds. This is what you may have heard referred to as The Internet of Things. Finally, to complete the puzzle consider this: what if we could gather all of that data being emitted by these small, medium, or even large devices in real time… route it to where it is best intepreted… and make use of the vast computational resources we have in our enterprises or cities to respond, adjust, and make our environment, well… better? Through Intelligent systems with advanced analytics, we can start to do just that.

Speaking personally, this is what has always excited me about technology: the potential to use it to improve the way that we live, work, collaborate, communicate, and exist. I realise I’m entirely biased, and I’m not sure I could have articulated it at the time that I joined the company… but the sheer breadth of talent, technology and reach that is wrapped up in the IBM brand and company, makes it an exciting place to be as we begin to address these ideas.

The technology foundations

OK, so here we are. We’ve got existing enterprise or municipal systems running on (frankly) a spaghetti of platforms, but which encapsulate fundamental business processes and applications which help to run our day-to-day lives – from the mainframes running critical financial transactions in IMS and CICS, to retail supply chains which ensure goods get to the right places at the right time, social security and medical processes which (in theory!) just make things “work” in our interactions with various authorities, utility companies which are continuously providing gas, water, and other scarce resources… the list is endless, but you name it, over the past 30-40 years there will have been a computer system built to support it.

IBM has been a leader in the enterprise messaging space for a long time. The space I’m specifically talking about here is what is often called message-oriented middleware or MOM. If you’re still looking blank, that’s OK… messaging middleware is, fundamentally, supposed to be invisible to you, so I’m not totally surprised. WebSphere MQ has been the reliable messaging transport used by many of the Fortune 500 companies for ~15 years now. It runs on a ridiculous number of platforms, has a number of language bindings, a stable API which has been backward-compatible the whole time, and it has become the de facto lingua franca and way of gluing disparate applications together. As an added bonus, the wider family of products also includes a high-performance, robust, exceptionally flexible, any-to-any connectivity, transformation and routing engine (Enterprise Service Bus) called WebSphere Message Broker. I don’t have space to talk about WMQ or WMB in detail here right now, but I guess I know some stuff (…!), so, ask me another time 🙂

The piece that has been missing in all of this until recently is the ability to reliably connect the edges, the frontiers of the data network, to the existing systems that already understand what to do in various scenarios. In other words, we could start to take action based on some data, but we generally couldn’t collect that data in real time, and as a result of that our analysis also tended to be flawed and based on outdated information.

What next? Joining the dots

I kind of fibbed, just then. IBM has actually had a technology capable of living out on these little embedded devices for aaaaaages! It’s called MQTT, the MQ Telemetry Transport, and it has had a home over on mqtt.org for a number of years. In fact, if you take a look, you’ll find that a bunch of people have implemented their own language bindings and small footprint message brokers already. It has existed inside a number of IBM products, and it was supported in WebSphere Message Broker up until version 6.1 via what were called the SCADA nodes. The name has been fiddled with a few times, and it hasn’t been a massively prominent part of the portfolio, so I’ve often found that folks haven’t heard of it. It’s very, very cool though.

What’s so special about MQTT? Well, it was specifically designed for constrained environments, with limited processing capabilities, potentially very tiny memory capacities, and fragile, unreliable, high-latency, low-bandwidth networks. As a result, it doesn’t have to have the reliable transactional qualities of service you might find in an enterprise messaging solution (although it can support those, too). The design principles do however bring in some massive advantages: it’s incredibly simple, easy-to-learn, and can be very fast and effective, with many thousands of lightweight clients supported by a single server.

On July 6th, IBM announced that MQTT is being added to WebSphere MQ as a first-class protocol.

[warning: the next couple of paragraphs describe finer details which are
potentially subject to change prior to final release!]

Alongside the existing MQ channels, it will now be possible to define Telemetry channels to enable the connection of one or many MQTT clients (by many read, like, really, LOTS, but wait for the performance SupportPac for full details!). Not only that… here’s what I believe is the really sweet part: MQ and MQTT applications will be able to interoperate, so messages published to a topic by an MQ application will be able to be received and consumed by MQTT clients, and vice versa. Nearly instant interoperability between existing enterprise applications and the edge of the network, if you need it. We’ve got security via SSL and JAAS, and we’ve got some simply beautiful tooling integration with the WebSphere MQ Explorer, including a nice test client.

As the diagram shows, the new feature requires a recent version of WebSphere MQ – I’m currently running 7.0.1.2 or better, but you’ll need to check the final requirements at release time. The new telemetry channels enable simple MQTT clients to be connected directly to a queue manager via a component which is internally called MQXR (for MQ eXtended Reach). The WebSphere MQ Telemetry Daemon for Devices can act as a concentrator, connecting up to a queue manager via telemetry channels if required (think, multiplexing connections at the edge of the enterprise). If you are using one of the other, richer clients that already contain MQTT or an MQTT-based technology like Microbroker (e.g. Lotus Expeditor), they will seamlessly connect up, too. You could even use a tool such as the third-party mosquitto broker to connect into a full MQ network.

My friend James Governor posited not very long ago that “WebSphere MQ won’t ever be a pervasive play“. James is an extremely smart guy and I take his opinions very seriously, but with the extension of WMQ to the web via the HTTP bridge introduced in version 7, and with MQTT joining the product foundation, I’d say that WMQ absolutely has the scope to be pervasive. I’ve been working with the team adding MQTT to WMQ for the past few months, and I have to say that the integration is looking really, really nice. I don’t want to bore with details in this post – I spent an hour talking about them at a conference today! – but I can say that it’s trivial to build a lightweight MQTT client, publish data, have that delivered into WebSphere MQ, transformed from a simple and tiny bytes message into meaningful XML in WebSphere Message Broker, and then routed on to any number of backend systems. There are customers implementing very exciting solutions in healthcare, transport, and energy to name just a few, and they are using MQTT in those solutions today. Plus, our very own lovable propellerhead Andy Stanford-Clark runs his home automation and mouse-slaughtering system on the protocol that he invented (I have a similar, but significantly less impressive and less all-encompassing system that I’ve written about before). Now is the time for these pieces to come together.

Keep watching for more as the release approaches in the next few weeks. MQTT really could be called the Smarter Planet Protocol, and I’m looking forward to find out what kinds of things we can use it for as we collectively create more “smart” solutions.

Update: if you’re interested, I will be giving a similar talk for the Global WebSphere User Group webcast series on August 18th. Additionally, if you are a member of SHARE, my slides for this talk should be available via the schedule planner shortly.

WebSphere Service Registry and Repository Redbooks

One of the products I’ve been becoming increasingly involved with as part of my work at Hursley has been WebSphere Service Registry and Repository. Rather than redefine what the product is here, I’ll take a snippet from the WSRR FAQ:

WebSphere Service Registry and Repository is a system for storing, accessing and managing information, commonly referred as service metadata, used in the selection, invocation, management, governance and reuse of services in a successful SOA. In other words, it is where you store information about services in your systems, or in other organizations’ systems, that you already use, plan to use, or want to be aware of.

The Registry and Repository is becoming increasingly central to many SOA deployments and is strongly integrated with several of IBM’s runtimes (including hooks with my long-term product specialisms, WebSphere MQ and Message Broker).

Version 7 of WSRR was announced at the start of October (more on this later in the week), but in the meantime it’s worth noting that a great set of Redbooks and Redpapers for the current 6.3 release have recently hit the publications website:

Over the past few months I’ve gotten to know many of the IBMers who worked on these books and papers personally, and I have to say that they are the absolute experts on the topics. I know I’ll be reaching for these publications when I need to know my way around specific topic areas.