Pivotal CF – the enterprise platform for software development

My boss and mentor, James Watters, just blogged about the launch of what we’ve been working on since before Pivotal was formed earlier this year – Pivotal One, powered by Pivotal CF (based on Cloud Foundry).

As I wrote back in April

Pivotal is bringing together a number of key technology assets – our Open Source cloud platform (Cloud Foundry), agile development frameworks like Spring, Groovy and Grails, a messaging fabric (RabbitMQ), and big, fast data assets like Pivotal HD.

What we’re announcing today delivers on that promise and our vision – the consumer-grade enterprise, enabling organisations to create new applications with unprecedented speed. The cloud – infrastructure clouds, IaaS like Amazon EC2, VMware vSphere, OpenStack, CloudStack, etc – can be thought of as the new hardware. It’s like buying a beige server box back in the 90s – the IaaS layer gives you a bunch of CPU, network, and storage resources, and for your application to use them, you need a layer in between – an operating system, if you like. We’ve spoken of our ambition for Cloud Foundry as “the Linux of the Cloud”, and it already runs on all of those infrastructures I’ve listed above – in the future, hopefully more.

Why is that important? Why should developers care about this Platform (PaaS) layer? A development team shouldn’t have to go through an 18 month delivery cycle to deliver an app! We’re putting an end to the whole cycle of calling up the infrastructure team, having new servers commissioned, operating systems installed, databases configured etc etc just to get an application deployed and running. When you first push an application to Cloud Foundry, and can then bind data services and scale out with simple individual commands, it really is a liberating experience compared to what traditionally has been required to get your application running. We’re making it quicker and easier to get going – a friction-free, turnkey experience. You should just be able to write your code and make something amazing.

We’re also delivering choice – of runtimes and languages, data services, and also importantly, a choice of “virtual hardware”. When Comic Relief ran in the UK this year, in order to avoid any risk of hardware failure (we all know there’s a risk that Amazon might go down), the applications were deployed on Cloud Foundry running on both Amazon EC2 with geographical redundancy, and on VMware vSphere – no lock-in to any cloud provider, and the developers didn’t have to learn all of the differences of operating different infrastructures, they just pushed their code. We’re happy to know that it was a very successful year for the Comic Relief charity, and that Cloud Foundry helped.

Pivotal One also includes some amazing data technologies – Pivotal HD (a simple to manage Hadoop distribution) and Pivotal AX (analytics for the enterprise). We recognise that as well as building applications, you need to store and analyse the data, so rather than just shipping a Cloud Foundry product, we roll up both the elastic scalable runtime, cutting-edge technologies like Spring.io, and and our big data offerings. That’s different from many of the others in the same market. We’ve been running our own hosted cloud, now available at run.pivotal.io, on AWS for over a year now, so we’ve learned a lot about running systems at scale and Pivotal One can do just that.

Above all, I wanted to say just how excited I am to be part of this amazing team. It is an honour to work with some incredibly talented engineers and leaders. I’m also personally excited that our commercial and our open source ecosystems continue to grow, including large organisations like IBM, SAP, Piston … it’s a long list. We took out an ad in the Wall Street Journal to thank them. I also want to thank our community of individual contributors (the Colins, Matts, Davids, Dr Nics, Yudais… etc etc!) many of whom, coincidentally for me, are in the UK – check out the very cool Github community where some of their projects are shared.

I’m convinced that this Platform is the way forward. It’s going to be an even more exciting year ahead.

A small selection of other coverage, plenty more to read around the web:

Cloud Foundry has gone Pivotal – so what’s new?

A few weeks ago I was privileged to be at the launch of Pivotal – a new organisation formed by VMware, EMC, and with investment from GE. You can read all about our new company at GoPivotal.com.

I am Pivotal
I am Pivotal

What does that mean for me, and for my role on the Cloud Foundry team? What is happening with Cloud Foundry right now? What about the Cloud Foundry community?

Well, as my über-boss James Watters recently wrote – we are a central part of the Pivotal business.

Our mission is to become the most popular platform for new applications and the services used to build them across public and private clouds.

That’s a pretty compelling mission statement, and I’d personally even add that we want to be the “best” platform, as much as “most popular”. One of the main reasons I wanted to spend a couple of weeks at the Pivotal office in San Francisco was really to immerse myself in the team and in the culture of Pivotal Labs, as well as to be at the launch event, and to get a strong handle on what is happening with Cloud Foundry, version 2…

Wait, what? Version 2?

In the middle of last year, the Cloud Foundry team started some major work to improve many of the features offered by the platform. Back then, it was written about on the Cloud Foundry blog. We initially started to refer to “ng” components like the Cloud Controller (“cc-ng”), and that’s what we now mean when we refer to “v2”. At the start of the year we published a roadmap which laid out a lot more detail in terms of what is coming. There’s some really great stuff in there – many bugs squashed; a new, high performance router; support for developers to collaborate on apps, via concepts of organisations and spaces; new containerisation via Warden; custom domains (yes, finally!); and most importantly, support for buildpacks. Buildpacks will bring a major change to our platform, replacing the former concepts of runtimes and frameworks (say, Java with Spring) with the ability to drop in whatever runtime or container you may choose, instantly making the platform more customisable. We’re pleased that the folks over at Heroku have allowed us to inherit the buildpack concept and having played with the new platform, I believe this gives us a really cool and solid way to support apps.

Deploying #cloudfoundry v2 on Amazon
Deploying Cloud Foundry v2 on Amazon

While I was in San Francisco, I used BOSH to deploy my own new Cloud Foundry v2 instance to Amazon EC2 (and also attended the AWS Summit, which was a bonus!). Right now the team is working on migrating our  hosted cloudfoundry.com platform to EC2, and when we officially boot up v2 for the public, it will be running right there. This is not new news – both James, and our CEO Paul Maritz, have repeatedly spoken about AWS.  The point of Cloud Foundry has always been that it is a platform that is Infrastructure-as-a-Service agnostic, even when it was started by VMware, and I’m seeing increasing interest from folks want to run it on OpenStack, AWS, and other infrastructures as well as vSphere (by the way, did you read about how Comic Relief 2013 ran on Cloud Foundry on vSphere and AWS? so cool!). There is no lock-in here – write once, deploy to cloudfoundry.com, to a partner running a compatible Cloud Foundry-based instance, or to your own private cloud on your on infrastructure, as you wish. The Open Source nature of the project is exactly why I jumped on board with the team a little over a year ago.

Talking of the update to cloudfoundry.com: it is also worth mentioning that when the beta period comes to a close we will have pricing plans, a nice web console for user, organisation and application management, and the start of a marketplace for partners to plug-in their own services for developers. I can’t give more details in this post, watch the official channels for news!

I felt very strongly that I wanted to write about version 2. It is a very big step in evolving the Cloud Foundry architecture, and I believe that it is important for the broader  community to understand that it is a significant change. If you are running an app on cloudfoundry.com today, we’ll shortly contact you with information about migration to the new platform, as some changes will be needed to adapt to the fact that runtimes and frameworks are now buildpacks, there will be some changes to the way services work, and you will need our new ‘cf’ gem to deploy to the new service. We have already “paused” new signups on the current platform. If you look at the new documentation, you will find that it now focuses on version 2 – we apologise for any confusion during the transition process.

We’ve been talking with ecosystem partners about version 2 as well. For instance, our friends at Tier 3 recently blogged about Iron Foundry plans, and I had the pleasure of meeting with Stackato folks in person in San Francisco recently. If you are working with your own Cloud Foundry instance privately (we know that many organisations are!) I strongly urge you to talk to us via the vcap-dev mailing list to learn how you can start to take advantage of what the new platform brings.

What else does Pivotal mean for Cloud Foundry? Well – we are more open than ever, and keen to work with the community on pull requests to add features via Github. I’ve just written a  post for the Cloud Foundry blog about how to participate in the Open Source project. In fact, I’ll be talking more about this at the Cloud East conference in Cambridge next Friday May 24. We’re always happy to talk more about how to collaborate.

These are exciting times!

 

Go Pivotal!

I’m in San Francisco today for the launch of a new company – Pivotal.

IMG_0116.jpg

Pivotal is bringing together a number of key technology assets – our Open Source cloud platform (Cloud Foundry), agile development frameworks like Spring, Groovy and Grails, a messaging fabric (RabbitMQ), and big, fast data assets like PivotalHD.

I’ll be live tweeting from the event, where Paul Maritz our CEO will be introducing the company and vision. You can also follow the @gopivotal Twitter ID, and check out the new website.