Have you ever considered CI/CD as a Service?

Introduction    

Would you like to create a complete, fully configured environment for Continuous Integration / Continuous Delivery with a single request, saving you ton of times in configuring and managing the pipeline? If so keep reading this post, the first one of a series of three where Stefano and I will focus on automation in CloudCenter to align with a DevOps methodology. The entire series is coauthored with Stefano Gioia, a colleague of mine at Cisco. We will be talking about a solution we’ve built on top of Cisco CloudCenter to support CI/CD as a Service.   

To keep things simple, we've decided to split the story into three posts.  
•  In the first post (this one), we will introduce the use case of CI/CD as a Service, describing it in detail to show the business and technical benefits.  
•  The second post will guide you through automating the deployment of a complete CI/CD environment in few minutes, by implementing a service in the catalog exposed by Cisco CloudCenter.  
•  In the third post, we will show how to apply CI/CD to the lifecycle of a sample application.   

A little refresh on DevOps    

Before taking this journey let’s first clarify what we mean by using the term “DevOps.” DevOps is not a technology or a magic wand that will instantly help you unify the development (Dev) and management (Ops) of your applications.   
DevOps encompasses more than just software development.   
It's a philosophy of cooperation between different teams in a company, mostly Development (Dev) and Operations (Ops), with the ultimate goal of being more productive and successful in launching new (or updating existing) services to reflect what your customers want.    
As shown in the picture below, this is how we see DevOps: as a human brain. We know that the right hemisphere of a human brain is said to be creative, conceptual, holistic: the opposite of the left region, which is rational and analytic.  

    
Apparently, two distinct aspects that cannot always work together. But nature finds a way to allow them to cooperate for the benefit of the human body.     
The very same concept can be utilized for your company: your Dev and Ops team has to collaborate to get significant benefits in productivity such as shorter deployment cycles which means increased frequency of software releases and finally better reaction to market and customer needs by quickly deploying new application features.   

What about Continuous Integration / Continuous Delivery / Continuous Deployment?    

Today Continuous Integration, Delivery and Deployment is a common practice in IT software development.    
The central concept is about continuously making small changes to the code, building, testing and delivering more often, more quickly and more efficiently, to able to respond rapidly to changing business contexts.    
The picture below illustrates a sample CI/CD process divided into stages:  

Stages in the CI/CD process

       

• Continuous Integration 

A common practice of frequently integrating and continuously merging code changes from a team of developers into a shared code repository. Quite often, after new code is committed to a repository, the server triggers a “build” and runs some basic test. Once the application is built and all the tests are passed, it’s time to move to the next step: delivery.
  

•  Continuous Delivery  

Simply means delivering the build to a specific target (environment), like Integration Test, Quality Assurance, or Pre-Production.   

•  Continuous Deployment  

Is fundamentally an extension of Delivery (and sometimes it’s included in the Delivery process). It allows you to repeat deployment of your application to production, even many times per day. The production environment could be an on-premises environment or a public cloud. In some advanced scenario, applications can be deployed in a hybrid model, example database on-premise while business logic and front end in a public cloud. This is called a hybrid deployment.    

Usually, when defining a CI/CD pipeline you will need at least the following components:
•  A code repository to host and manage all your source code 
•  A build server to build an application from source code 
•  An integration server/orchestrator to automate the build and run test code 
•  A repository to store all the binaries and items related to the application 
•  Tools for automatic configuration and deployment    

Let’s take a look at the typical challenges of implementing a CI/CD process.  
First of all, having one single CI/CD toolset in your company is not a good option.      

Every LOB uses a different CI/CD toolset

    
As you can see from the picture above, every LOB (line of business) might have different requirements (and sometimes also a developer team inside the same LOB) and most likely they will use different technologies to create a new application/business service. They might even use different code language (Java, nodeJS, .Net, etc.) and be more familiar with a specific tool, e.g. GitHub rather than Subversion.     

How can you accommodate this diversity?    
You probably guessed it right. You can create multiple CI/CD chains and then install multiple tools (perhaps on VM or in a container) depending on the requirements coming from the developers and/or LOB.     
However, how much time you will be spending in configuring every time, for each LOB or DevTeam, a new CI/CD chain? Moreover, what about maintaining and upgrading all the components of the CI/CD toolchain to be compliant with any new security requirements from your security department?     

How long does it take to prepare, configure, deploy and manage multiple CI/CD chains? Sounds like a typical Shadow IT problem, a phenomenon that happens when a Developer Team or LOB users can’t get a fast-enough response from the IT and rush to a public cloud to get what they need. In that case, the solution was to implement automation and self-service to quickly provides the necessary environment to the end users, with the same speed and flexibility as the public cloud.    

Wouldn’t be much easier to adopt the same approach and merely automate the deployment and configuration of the CI/CD chain with a single request generated by a simple HTML form?    
Good news: that’s precisely the purpose of “CI/CD as a Service.”   

Introducing “CI/CD as a Service” 

Let us first clarify an important point: we are not discussing relocating your CI/CD resources and process to the cloud, and then consuming from there. Nor we are discussing the automation task performed by the CI/DP pipeline (push code to the repository, the automatic build of the code, test, and deployment).    

What we are proposing here is to automate the deployment and configuration of the tools that are part of your CI/CD Pipeline. With a single request, you will be able to select, create, deploy and configure the tools that are part of your automated CI/CD pipeline.    

The key things here is that the customer (a LOB as an example) can decide which are the components that will be part of the CI/CD pipeline. One pipeline could be composed by GitLab, Jenkins, Maven, Artifactory while another one could be composed by SNV,Travis,Nexus.    

Your customer will have their own CI/CD chain preconfigured, ready to be used so they can be more productive and focus on what’s matter: create new feature and new applications to sustains your business and competitive advantages.    

Let's now have a look at some of the technical and business benefits you can expect if you embrace CI/CD as a Service.

From a technical point of view, here are some good points:

•  Adaptable: your LOB/Dev Team can cherry-pick the tools they need it, from your catalog 
•  Preconfigured: all the components selected, once deployed, are configured to work immediately for you. 
•  Error-free: as you automated all the steps to deploy and configure the elements, this leaves no room for any human error or misconfiguration 
•  Clean: always have a clean, stable and up to date environment ready to be used 
•  Multi-tenant: serving multiple Line of Business (LOB) in your company: each LOB can have his own environment 
•  Easy Plug: as it’s callable by using REST API, you can easily integrate your IT Service Management system for self-service 
•  Independent Solution: can run on top of any infrastructure/on-prem private cloud    

Don’t stop the business: are you running out of on-premise resources? The solution allows you to quickly deploy the service, temporarily, in a public cloud to avoid blocking the development of your critical project.    

The majority of customers are interested in the CI/CD approach, and they are actively looking for a solution that can be easily implemented and maintained; therefore, we firmly believe that a solution for Cisco will be seen as an enabler for their business strategy.    

In the next post, we will present a solution that decouples the tools utilized in CI/CD pipeline from the deployment targets.  The Use Case is implemented by Cisco CloudCenter (CCC) a fundamental component of the Cisco Multicloud Solution.      

Credits

This post has been authored by Stefano Gioia, a colleague of mine at Cisco.

Just 1 step to deploy your applications in the cloud(s)

As described in my previous post about Terraform, the deployment environment for a new application can be created "on demand" by configuring physical and virtual resources.

Good open source products allow to describe the desired state and to automate the setup of a target infrastructure.

They can also deploy your software application and configure it properly.

But is some use cases this is not enough.

You might want to offer your users - depending on their needs and their skills - a visual catalog in a web portal.

You might want to apply a governance model based on policies, use different clouds as possible targets for the deployment, offer a easy way to manage the life cycle of the deployment (start, stop, scale up/down, terminate) and get reports on usage of the resources.

If this is the case, there are good solutions available.

One of these is Cisco Cloud Center, a powerful tool that offers two main use cases: 

• modeling the deployment of a software stack (creating a template or blueprint for common deployments) and 
• brokering cloud services (different resource pools available from a single catalog).

A easy to consume (and manage) self service catalog

Open Source or commercial products?

In the same project where I used Terraform to deploy Apache on Openstack, I also used Cisco Cloud Center to deploy a portal application on Openstack.

But at the same time, I offered the possibility to target the same deployment to a public cloud (AWS in this particular case) or to the private cloud (choosing between Openstack and vmware in this particular case). No duplication of the effort was needed, because the model you creat is not referred to a specific cloud as a target. It will be matched, when a user orders it, with one of the cloud avaliable for him or for his project.

So I was able to show the difference between a free, open source solution (Terraform) and a commercial product (Cloud Center) in a similar scenario.

The second option addresses different needs of the organization and offers a richer solution.

It’s up to you to evaluate which one fits your requirements better. 

Modeling, policies and multitenancy

One of the differences is that Cloud Center offers a graphical editor to model the topology and the dependencies among all the building blocks of your deployment.

You have a library of services (software applications from a repository, physical and virtual services like load balancers and firewalls).

Services can be dragged and dropped in the editor, then you set their properties and dependencies. 
The architecture of the application you're modeling can be based on a single server or a number of servers with different roles.

If the application architecture has multiple tiers, every tier gets its own attributes and policies: as an example, you can set the minimum and maximum number of instances in a cluster of web servers (or application servers or database servers). 

Autoscaling policies will tell the orchestrator to increase or decrease the number of servers based on metrics like consumption of cpu or memory, inbound/outbound traffic, etc.

Everytime the cluster changes, the orchestrator will modify the configuration of load balancers and firewalls accordingly: no manual intervention is needed.

Models are saved in the catalog and offered to users in a multitenant organization: every tenant is given a portion of resources (target cloud environments) and services (models available in the catalog to deploy applications) that the tenant administrator can offer to his own users and groups... and sub tenants. Every tenant cannot see other tenants' stuff.

A graphical editor to model blueprints for application deployment in hybrid cloud

Dashbooard and Reporting

Every user has a dashboard that shows the consolidated information about all the applications he has deployed (or the other users in the same tenant), and can manage the lifecycle of all the deployments.

Of course the administrator of the system sees the global view including all the assets.
Active VM per cloud and per application are shown in the dashboard, as well as associated costs.

Cloud Center's Dashboard

A powerful reporting features allows to filter deployments and costs by user or group, application, environment and cloud.
Data can also be exported in different formats, to be consumed by humans and other systems.

Unified reporting

Architecture

The architecture of the Cloud Center product is based on two Virtual Machines: the Manager (CCM) and the Orchestrator (CCO).

The Manager is the engine where policies and application models are defined, and where the user portal runs. The Orchestrator lives within each of the target clouds (indeed, there is one CCO in each cloud region), receives commands from the Manager and executes them locally using the API of the cloud platform.

Cisco provides orchestrator images that are specialized for every cloud supported by Cloud Center.  So you have a single place to manage all your cloud resources, and a single model to maintain: you don't need a model, or a workflow, or a script for every target cloud where the syntax of that specific API is used. You create a single model, that is completely decoupled from the target of the deployment: this reduces the amount of work (a single model instead of many) and makes the maintenance of the model easier and more consistent (you don't have to evolve many models for the same application).

 

Cisco Cloud Center architecture

Comparison

Two solutions for the same use case, one for free and one at a cost?
Indeed they address different requirements: as described above, Cloud Center is for enterprise organizations that need to rationalize their usage of cloud resources. It is used by the corporate IT to provide flexibiliy and agility to their developers (within a governance model), to standardize the architecture of their projects based on blueprints (including what products, what versions, what setup configuration they prefer) and to get reports on consumption.

Service providers can use Cloud Center to broker third parties' resources, offering a single catalog to their customers. The hierarchical multi tenant organization and the sophisticated cost models that can be offered make it simple.

I suggest you to consider it if you are using, or plan to use, two or more cloud providers (counting also your private cloud or your virtualized data center). You will see an immediate benefit in terms of compliance and efficiency.

References

http://www.cisco.com/c/en/us/products/cloud-systems-management/cloudcenter/index.html
http://www.cisco.com/c/en/us/solutions/data-center-virtualization/one-enterprise-suite/index.html
http://docs.cliqr.com/display/CCD46/CloudCenter+Documentation+4.6
 

Why don't you try Openstack (without getting your hands dirty)?

Is Openstack ready?

But, more important: are you ready for Openstack? 

 

Openstack is mature (but complex).

Surveys and statistics show that Openstack is mature and provides a number of benefit to a broad spectrum of users, from small to large enterprises and service providers.  

Almost every professional in the IT (including CIOs and CTOs) knows the advantage that Openstack would offer to his organization.

But many are also aware of the complexity of the technology, the need for new operational processes and skills to set up and operate Openstack.

A scalable and reliable production environment is different from a lab where you explore the capabilities of the new platform.

The journey to a mature adoption of Openstack is not easy and you need to invest time and money.

In addition, when you hire people (or train yours), there is a possibility that another company steals them with the offer of a better salary, given the scarcity on the market.

So, many IT organizations - excluding cloud service providers, because that’s exactly their business - started wondering if it’s worth spending time in running the infrastructure, rather than running their business applications.

If you are not a cloud provider, that makes money selling IaaS, why should you dedicate additional effort to installation, monitoring, troubleshooting and release upgrades to ensure reliability and performances to your applications (that’s the only asset you should care of, because your business relies on them)?

Focus on your real business.

Why don’t you delegate all the responsibility to a provider, signing a contract that puts the above tasks and SLA on them?

Doing so, you would be free to use Openstack, getting all the benefit that you expect from it, without the burden of the learning curve and the organization implied by the Openstack adoption.

You would focus on using the infrastructure to develop and run your applications, no longer on running the infrastructure itself.

That is called a managed service.

You own the infrastructure and exploit the value of your Data Center assets (you don’t just drop them to escape to a public cloud).

An expert team (it’s just their business) installs Openstack in your DC and operates it everyday in a HA (high availability) configuration, granting 99.99% uptime.

They take care of all the version upgrades and the compatibility of all the new features released by the community by using a certified configuration.

The user interface (the Horizon console, the Openstack API and command line interface) is available to you so you can deploy virtual server instances, networks, storage at will. You get complete and granular reporting on the health of the system and its performances.

You are the owner, but you don't get your hands dirty with the complex stuff   :-)

You pay them for the service, they grant you the SLA.

Just taste if you like Openstack.

The approach described above can be a strategical decision, because you want to focus on your business applications.

But you could also use this trick to stand up a Openstack environment in very short time, test it (I mean if your organization adapts to it, if your applications run well, if the operational model - IaaS at home, on your infrastructure, no cloud provider lock in - is good for you, if your developers are more productive) for a while, e.g. 3 or 6 months, and finally decide if you want to adopt it. 

At that time you can choose between continuing with the managed service or doing it yourself.

It is a zero risk trial of the technology and of the processes: if you don’t like, you haven’t wasted any time and effort to stand it up so you can happily retreat.

You simply do not renew the service contract and that’s all: you have made a real informed decision about the adoption of Openstack.

Cisco Metapod: Openstack as a managed service.

Cisco has a offer that allows you to do what I described above, that comes from the acquisition of a company whose business was exactly Openstack as a managed service, on your premises.

They had a Openstack distribution of their own, optimized and hardened to provide a smooth and effective service.

Now, thanks to a strong partnership with Red Hat, the team is using the Red Hat Enterprise Linux Openstack distribution (OSP8, based on Liberty).

The essential features of this service are:

- easy start: entry level contract for 90 days

- ready to go live in 2-3 weeks from the engagement

- HA included

- the infrastructure to run Openstack can either be yours or provided by Cisco

- both the Openstack API and the AWS API are exposed by the system

And the infrastructure to run it in production can be as simple as this:

The value you can get from it: a well defined SLA, installation included, maintenance and upgrade included, no cloud provider lock in.

I believe that Cisco Metapod is a very good option to start with Openstack.

You can put your foot in the water to test the temperature, then decide to take a bath if you like it.

References

Openstack users survey 

Cisco Metapod official page

Cisco and Openstack on this blog