The era of applications: microservices and containers in the cloud

It’s finally time to turn the page. We’re now right into the era of applications. They now are dictating how the rest of the world of IT should behave. Infrastructure has no longer the spotlight but it’s taking the passenger’s seat and merely delivering what it’s being asked for. What I sort of predicted almost 3 years ago in “Why the developer cloud will be the only one” is now happening.

That’s no good news for someone in this industry, I know. It’s much easier to talk (and sell) dumb CPU, RAM, storage space than it is about continuous integration, delivery, runtimes, inherently resilient services or even things like the CAP theorem. Sorry guys, if you want to remain in this industry, it’s time to step up and start understanding more about what’s happening up there, at the application level. Luckily, we have things that help us do so. Things like thenewstack.io (cheers @alexwilliams and team) who’s doing an awesome job introducing and explaining all these new concepts to the wider audience (including me!).

And to learn more about this phenomenon is also why I am going to attend KubeCon in London next week. For those who don’t know, KubeCon is the conference of Kubernetes, a Google-stewarded project for containers orchestration and scheduling. Some people will say it’s just one out of many right now but for us, at Flexiant, it’s just *the* one, as it possess the right level of abstraction to deliver container-based distributed applications. I’m going there to listen to industry leaders, innovators and just anyone who’s fully understood the new needs and who’s working every day to solve these new problems in new ways.

If you still don’t know what I’m talking about, read on. I’m going to take a step back and tell you a bit more about the drivers that caused applications to take over and why we needed different architectures, such as microservices, to be able to efficiently deliver and maintain that software that’s eating the world. If you’re short of time, you can simply watch the embedded video at the top of the page which should tell roughly the same story.

TL;DR

Software is now pervasive in people’s everyday’s life and it’s handling many of the new business transactions. Application architectures had to evolve in order to be able to cope with the increase in demand. Microservices is the optimal software architecture that combined with cloud infrastructure and containers can successfully fulfil new application requirements.

However, its numerous advantages are counter balanced by an increased complexity, which requires new orchestration tools that are able to join the dots and hold a global vision of how things are going. To achieve this, orchestration needs to happen at a higher level of the stack than what we had been previously seeing in the previous infrastructure era.

The rise of microservices

We can comfortably acknowledge how the way we do business has changed, how we see more and more transaction happening just online and how software is the only mean to achieve them. Software that, by the global nature of the relationships, needs to cope with a large number of users. It also needs to constantly deliver performance, to satisfy its user base as well as new features to keep up with competition, serve new needs and unlock new opportunities. You can easily understand how the traditional way of developing applications could just not power this kind of software. Monolithic applications were just too hard to scale, slow to update and difficult to maintain; on the other hand, the recently hyped PaaS was just too abstracted (compromising on developers’ freedom) and expensive to deliver the required efficiency.

That’s when microservices came in as the new preferred architectural pattern. Of course they had been around for a while even before they were called so but, as Bryan Cantrill (@bcantrill) said “[…] only now that we gave a name to it, it has been able to spread much beyond that initial use case”. That means that whenever we manage to label something in IT, this helps with diffusion, adoption and it serves as a baseline for further innovation. This has happened with cloud computing, and we see this happening again. For once, thank you marketing!

What exactly are microservices? We call microservices a software architecture that breaks down applications into many atomic interdependent components that talk to each other using language-agnostic APIs. A single piece of software gets broken into many smaller components, each of them publishing a API contract. Any other piece of that application can make use of that microservice just by addressing its API and without knowing anything about what’s behind it, including which language it’s written in, or which software libraries it uses. That unlocks a number of benefits, like single components that can be developed by different people, shared, taken from heterogeneous sources and reused a number of times. They can be updated independently, rolled back or grown in number whenever the application as whole needs to handle more workload. All of this without having to tear down the giant, heavy and slow monolith. Sounds great? Thumbs up.

Infrastructure for microservices

Setting up the infrastructure to host such distributed, complex and ever changing software architectures would be a real challenge, if we did not have cloud. In fact, infrastructure-as-a-service is a just no brainer to host microservices. Why? Because it provides commodity infrastructure, because you pay for what you use, because it’s just everywhere near your end users and because it never (well, almost) runs out of capacity. But what makes it so perfect for microservices is its programmability, required whenever a distributed application needs to deploy and re-deploy again and again, while adapting its footprint to its workload requirements at any given moment. We couldn’t have done it with traditional data centre software. Full stop.

When people think of IaaS, they think about virtual machines, virtual disks and network. Let’s call it traditional IaaS. And if you look at it, it’s not really fit for purpose for what we described above. In fact, virtual machines have zero visibility of what’s going on on top of their OS, let alone the interdependencies with other virtual machines hosting other services! So, we’ve seen things like configuration management systems (Chef, Puppet, Ansible, etc) taking over this part and, at the completion of the OS boot, to execute a number of configuration tasks to reach the full application deployment. It sort of worked so far, but at what price? First, virtual machines are slow. They need to be commissioned and then full OS needs to come up before it can execute anything. We’re talking about 20-30 seconds if it’s your lucky day up to several hours if it’s not. Second, virtual machines are heavy. The overhead that the hypervisor carries is just nonsense, as well as all that other multi-process and multi-user functionality that their OS was born to deliver, when in reality they simply need to host a little – micro – service. And configuration management systems? Even slower. Let alone their own weight (I’m looking for example at the full Ruby stack with Chef) they typically rely on external dependencies that, unfortunately, can change and generate different errors every time they are called in.

There was the need for something better. Oh wait, we already had something better! Containers. They existed for a while but they were having a seat in the previous infrastructure-centric IT world that was dominated by the expensive feature-full virtual machines. Guess what, containers understood (yeah, they apparently have a thinking brain!) that they could make the leap into the new application-centric world and shake hands to software developers. That’s how they recently become – rightly so – popular, as I wrote before in “The three reasons why Docker got it right”. Containers are just right to host microservices because (1) they’re micro as well, and they can start in a fraction of a second, (2) they’re further abstracted from the infrastructure and hence have no dependency on the infrastructure provider, (3) they are self-contained and don’t rely on external dependencies that can change but, most importantly, they are immutable. Any change within the container configuration can trigger the re-deployment of a new version of the container itself, which then can be rolled back if the result is not what we were expecting.

New challenges for new opportunities

As it happens every decade or so, the tech world solves big problems by tackling them from the side with disruptive solutions that unlock tremendous new opportunities. The veterans of this industry see these solutions and think “wow, that’s the right way of doing it!”, no question. However, new approaches typically also open up to a number of unprecedented challenges that could not even exist in legacy environments. These new challenges demand new solutions and that’s where the hottest (and most volatile) startup scene is currently playing a game.

Kubernetes, Docker, Mesos and all their eco-systems are right there, trying to overcome challenges that arise from the multitude of microservices that need to operate independently (providing a scalable and always available service) as well as with each other, cooperating to make up the whole application’s business logic. Networking challenges coming from microservices that need to communicate in a timely, predictable and secure way over the network. Monitoring challenges as you need to understand what’s going on when an end user presses a button, if any of the components is suffering from performance and needs to be scaled. And not to forget organisational challenges that come from when you potentially have so many teams working together on so many independent components that involve security, adaptability and access control, to name a few.

In the end, as we can’t stop software from eating more pieces of the world, we simply can try to improve its digestive system. Making software transparent to end users to generate positive emotions and ease transactions, while helping businesses not to miss any opportunity that’s out there are the ultimate goals. Making better software is a just a mean to get there and microservices, cloud and containers are headed in that direction. See you at Kubecon EU!

If cloud can’t wait, will you?

A few days ago I have participated as a panelist in the webinar titled “Cloud Can’t Wait” alongside Michael Coté (@cote), analyst at 451 Research, Jared Stauffer (@jaredstauffer), CEO at Brinkster and Jim Foley, SVP Market Development at Flexiant.

We have debated the cloud opportunity. Sounds old? Maybe. However, surprisingly enough, the majority of IT infrastructure buyers haven’t adopted it yet. Skepticism, natural resistance to change, staff self-preservations and other excuses are amongst the primary reasons for that. If you think about it, this is actually pretty normal when a technology is so much disrupting the status quo.

The title of the webinar “Cloud Can’t Wait” may sound like a way to build the hype but, with regard to cloud, I think we all concur that, by now, the hype is way over. As I’m sure we agree that, indeed, the cloud can’t wait. Those who’ve fully embraced it have demonstrated to have significant advantages over those who haven’t, and these advantages are directly affecting their competitiveness and even their ability to stay in business.

The opportunity is for everyone

We talked about the cloud focusing on the infrastructure side of it. We have deliberately excluded SaaS consumption from the statistics and the debate, as that has a totally different adoption curve and, when put in the same basked, can easily mislead the conclusions. So rule number one, treat SaaS numbers separately.

Michael Coté presented an interesting categorisation of cloud infrastructure services, segmented as follows:

  • Infrastructure-as-a-Service (IaaS): compute, storage and network “raw” infrastructure.
  • Platform-as-a-Service (PaaS): supporting developers and middleware integration they require.
  • Infrastructure-Software-as-a-Service (ISaaS): the applications required to manage IT infrastructure, including backup, archiving, disaster recovery (DR), capacity planning and, more generically, IT management as a service.

Seeing ISaaS as third category was pretty interesting to me as we all knew it existed but we never managed to label it correctly. And as Michael stated later on, expertise in this specific category is what some service providers, mostly those coming from the managed services space, can actually offer as value add on top of raw infrastructure, in order to win business in this space.

So what is this cloud opportunity we are referring to? Again, Michael explained it this way:

“[With a 29% year over year growth rate] there is the opportunity to get involved early and [as a vendor] participating in gathering lots of that cash. Instead, cloud buyers such as developers or enterprises, are not interested in participating in this growth, but in the innovation that comes out of this cloud space, they want to use this innovation and efficiency to really differentiate themselves in their own business”

So the opportunity is there and it is a win-win for everyone.

Why people are buying cloud and who are they?

If you ask yourself why people are buying cloud and what they’re using it for, you maybe won’t find the answers easily. That’s where the work of 451 Research becomes really helpful. As Michael told us, from the conversations they have everyday, it came out that most organisations use the cloud because of “the agility that it brings, the speed you can deploy IT and [afterwards] that you can use IT as a differentiator. [Because cloud] speeds time to market”.

To that, I would add that cloud also speeds the ability to deliver changes which translates into adaptability, essential for any chance of success in our rapidly transforming economy.

Michael continued on this topic:

“Over the past roughly 5 to 10 years much of the focus of IT has been on cost savings, keeping the lights on as cheaply as possible, but things are changing and qualitatively we see this in conversations we have all the time, companies are more interested in using IT to actually do something rather than just saving money, and cloud is perfectly shaped for offering that”

Great. This seems to be now well understood. The days of explaining to organisation that there is more to the cloud than the simple shift from CAPEX to OPEX, are gone.

Who are buying cloud infrastructure services today? My first answer went to:

“Developers. This word returns a lot whenever we talk about cloud. They’ve been the reason of the success of AWS, for sure. That’s because they just ‘get it’, they understand the advantages of the cloud around how they can transform infrastructure into code. For them, spinning a server is just like writing any other line of code for doing anything else. They managed to take advantage of the cloud from the very early days and they contributed to make cloud what it is today under many aspects”

With regard to enterprises, I also added:

“enterprises are [currently] investing in private clouds because that’s the most natural evolution of their traditional IT departments, but eventually, as they get to provide cloud, it’s gonna be extremely easy to get them to consume cloud [services] from third parties. That’s because cloud is more of a mindset than just a technology”

How can you profit from the cloud opportunity?

So you’re a service provider and you want to participate in the cloud opportunity. How do you do that? Michael suggests to use the “best execution venue” approach. That starts, as Michael explains, with understanding the type of workload or applications that you want to address. Then ask yourself what skills, capabilities and what assets do you have that you can leverage to address a specific type of workload? This will tell you what value you can bring on top of raw infrastructure in order to compete and take advantage of this fast-growing multi-billion market.

My comment on this was:

“Eventually service providers should not consider themselves just part of one of these [IaaS, PaaS or ISaaS, Ed.] segments. Eventually I think the segmentation of this type will not there anymore, and there will be another segmentation based more on use cases, where the service provider will specialise on something and will pick a few services to make the perfect portfolio to match a specific use case in a target market”

Yes I’m a big fan of the use case approach. As I’m a big fan of trying to understand what the cloud is exactly being used for. Even if the press tries to push the cloud as heavily commoditised service, you should never stop asking yourself what your customers are doing with it, what applications they’re running and what else you can do to make their life easier.

In any case, either you decide to leverage your existing capabilities or you try to learn what your customers want to do with your cloud, we all agreed around the following statement: it’s still very early days. As Michael again explains, there are still lots of options to get involved, it’s a great time to get involved, and the doors are definitely not closed.

I’d say they’re absolutely wide open. And many have already crossed the doorway. How about you?

You can listen to the full recording of the webinar at this link.

IaaS eats the biggest slice

When I read market research firms saying that SaaS is the most adopted cloud model by the enterprises I can’t but concur due to the ease of use and the simplicity of integration with existing IT assets. Actually, the integration ends up being minimal and entirely in developers’ hands, who can make use of the SaaS service usually comprehensive API, thus completely bypassing their internal IT department.

So what about IaaS and PaaS? Should those who invested heavily in those two cloud models start worrying about their choice? No way. As my provocative title says, I am fairly much convinced that the lower layers of the cloud stack eventually share the whole cloud business, with IaaS eating the biggest slice of it, both directly and indirectly.

I am actually writing this post to give further insight and supporting data to a tweet of mine I wrote some time ago:

Now let’s see what I mean by indirectly.

Layers over layers

In computer science we are used to have layers over layers called “abstraction layers“, each one of them aimed at hiding the complexity of the lower one, while providing some kind of added value and an interface for the immediately upper layer to access resources. With the rise of cloud services, the approach of the community has been the same again: using abstraction layers to handle the increased complexity of IT infrastructures, which now involve thousands of resources to be managed and orchestrated.

As mentioned above, there are three main cloud layers largely accepted by the community: IaaS, PaaS and SaaS. However, many cloud providers don’t fit exclusively in one of them as they tend to enlarge their offering with different services at multiple layers of the stack. Since this creates a little confusion among cloud consumers, I want to take the opportunity to present them one more time from a different perspective, trying to concentrate on what added value each layer brings to the stack.

Ok, I still have to work a bit on my ability to visually represent concepts but I hope the above chart can help making some clarification. First, we have raw resources at the bottom of the stack and if we add some elasticity we obtain an IaaS. This is over-simplified as there are certainly more values brought by any good IaaS layer, however, for the sake of understanding, I’ll limit myself to the most evident one: elasticity, a.k.a. the ability to create, destroy, enlarge and shrink computing resources on demand via an API.

Let’s now go upper, we have an IaaS layer and we decide to add some DevOps tools and operations such as middlewares, auto-scaling, application deployment and code validation mechanisms. While doing that, if the principle of abstraction layers is respected, we don’t need to care anymore about how to handle raw resources, since the IaaS provides us with tools to automate their management. What we obtain is a Platform-as-a-Service, an environment where multiple users can deploy their applications.

Eventually, let’s take some business logic to solve a specific problem (i.e. CRM, ERP, etc) and, provided of course that we have done all the multi-tentancy stuff and that we want it to be consumed as-a-service, we are now working at the SaaS layer. At this stage, we can concentrate on making our software more powerful, adding killing features and conquering our market niche. We don’t need (neither we do want, right?) to take care of all the infrastructure to serve our users nor we want to know what hardware lies underneath, as those would be just a distraction from our core business focus.

Sounds logical doesn’t it? All the layers stack up together so nicely and they look so complementary. Indeed they are. In fact, cloud companies end up buying services from other cloud companies that operate at a lower level of the stack. For further evidence, I have done a small research and I found out that most SaaS companies deploy their software on top of a PaaS provider that, itself, deploys its automation layer on top of one (or more) IaaS providers. What does that mean? That if an enterprise adopts a SaaS cloud service and pays for it, eventually some dollars will end up in some IaaS providers’ pocket. You like it or not.

The infrastructure of PaaS providers

To bring supporting examples, let’s check the most popular PaaS providers infrastructures as they’re most likely obliged to reveal their backends in order to inform their customers on their data center locations.

Most popular PaaS services rely on IaaS providers
PaaS Provider Supported Languages IaaS backends
Heroku Ruby, Java, Python, Node.js AWS
Engine Yard Ruby, PHP, Node.js AWS, Verizon Terremark
AppFog PHP, Java, Node, .NET, Ruby AWS, Rackspace, HP Cloud Services
OpenShift PHP, Java, Node.js, Python, Ruby AWS, Rackspace
Nodejitsu Node.js Joyent
AppHarbor .NET AWS
CloudBees Java AWS, HP Cloud Services

The cloud market is known to be huge and it is mandatory for every player in the IT industry today to take up a position, a vision and a direction within this space. If you’re an investor who wants to participate in the cloud opportunity, it is extremely valuable to understand how different cloud models are currently sharing the market. On the other hand, if you’re an enterprise evaluating the adoption of any cloud service, you should be concerned about who’s running the games up and down the cloud stack, as this will eventually affect you service level, your security and your data integrity.

POST UPDATE on 4/8/2013

I’ve been asked by Jack Clarke (@mappingbabel) of ZDnet on what basis did I single out the above PaaS providers as “most popular”. The answer I gave him is press coverage as well as “on the field”, meaning talking to customers and gathering experiences. It’s a simple personal feeling which is not based on any scientific data. I’m actually a field person and not a researcher. Besides, I don’t think any of those provider is really willing to disclose customer data.

However, it’s noteworthy to mention there are other PaaS services offered by large vendors that are difficult to define in terms of popularity; the press usually refers to the vendor as a whole and since they’re no longer in the startup phase, you can’t even measure the funding amount they’ve received from VCs. Despite the difficult measurability, I owe them a mention in this post for being active players in the PaaS landscape, contributing effectively to the cloud awareness battle.

And one can assume the above theory is respected by the above providers as well, for example with Elastic Beanstalk running on top of EC2 and App Engine running on top of Compute Engine. However, given those services are provided by the same vendor as the PaaS provider, they don’t trigger any economic transaction and thus no real shift in the measurement of the market size.

There is no such thing as the cloud uptime

Yesterday readwrite.com featured an article by Mike Pav titled “Storm Warning: Why 100% Cloud Uptime Is Impossible” and I thought it was such a piece of misinformation that I decided to write this blog post to help clarifying a few things, as I’m really fed up to hear about the unreliability of “the cloud” in general terms.

Titles are usually provocative and I won’t judge its veracity, however there is no such thing as the “Cloud Uptime” because, despite the cloud is considered as a whole, you can imagine that it is made of thousands of components and not all of them go down at once. Therefore, the outage within a cloud service tends to be bigger the more these components are interdependent. I’m going to explain this more in details.

Cloud Outages

The article says “Cloud Outages” are eventually inevitable because doing better than 99.99% availability would cost too much and companies like Netflix (which suffered its cloud provider outage right on Christmas Eve) would still continue using the cloud just because eventually “it does a great job of providing ready-to-use features”. In other words, it says that using the cloud requires a compromise that companies with multi-million businesses are ready to take: losing money from time to time in exchange of the flexibility of the cloud. My dear, I refuse to believe that.

First off, cloud providers do things differently and we can’t generalize. Let’s narrow down to AWS as this is the cloud provider the article mainly refers to. AWS is primarily an IaaS provider with some service components operating at the PaaS layer, such as the ELB (Elastic Load Balancer). In this context, there is no such thing as a “Cloud Outage” but there is the outage of a component of the cloud that your application relies on and that your application has not been instructed to handle in case of failure.

When working at the PaaS layer your freedom is limited. On one hand, you don’t have to worry about how things work underneath because the provider does everything for you but, on the other hand, you also have to rely on it when it comes to availability and SLA. Netflix relied on ELB and their application had no other way to handle its failure than waiting for AWS to fix the problem.

So how should Netflix prevent such things from now on? As others have also said, they should just build their own load balancing service by operating at the IaaS layer. In this case, they would have the freedom and the responsibility to set up multiple LBs in different availability zones or even different data centers, making their application more resilient in case of any infrastructure outage.

The responsibility of a PaaS provider

Later, the article goes through a list of PaaS provider duties in case of an outage. When I read it the second time I figured out that the term PaaS was misused as the author was instead referring to a generic provider offering any kind of services through the cloud.

However, this gives me the chance to say that a real PaaS provider should never ever suffer from any underlying infrastructure outage. The PaaS software should be the very best example of highly available resilient application, architected to exploit most of the isolation/redundancy mechanisms made available by the underlying IaaS. In the end, a PaaS provider employs mostly DevOps who master cloud automation tools and best-practices and who do know how to make an application resilient.

Moreover, a PaaS cloud is not about elasticity or scalability, as the article says, but those two come from the underlying IaaS: it’s the infrastructure that scales, it’s the infrastructure that grows and shrinks fast. Whereas PaaS is all about about automation: automated deployment, auto-scaling, automated failover and recovery on infrastructure failures.

What cloud uptime is about

In conclusion, more than 99.99% is actually possible and there are examples of that. Joyent is one that managed to deliver 99.9999% of uptime in the last 2 years. So how to build more reliable clouds? Simply by architecting an infrastructure with the least possible number of interdependent components. A cloud infrastructure made of distributed and replicated micro-components is capable of delivering scalability and reliability while limiting the impact of an outage, preserving the overall SLA.

Two things to keep in mind for the best uptime of your application in the cloud:

  1. Choose an IaaS provider with an architecture designed to limit the impact of outages. If this sounds too theoretical, then think about EBS (AWS Elastic Block Store) which is a centralized macro-component highly dependent on the network.
  2. Choose to have the freedom to build your own resilient app at the IaaS layer and, if you decide to go PaaS, pick a provider with an refund policy in case of outage that is significative enough for your business.

And in the end, Netflix will keep using the cloud because they learnt from this experience and they know that mastering cloud best-practices can save them from the next (indeed inevitable) infrastructure outage.