Cloud Computing as an extension of SOA

When I started explaining my view of Cloud Computing as an extension of SOA (Service Oriented Architecture) someone didn't take it seriously.
I delivered some TOI sessions to increase the awareness on topics that Cisco was approaching in its transformation into a IT company: software architecture, distributed systems, IT service management. I reused some of the concepts and the slides that I created when I was a SOA evangelist.

The feedback was positive and generated a useful discussion, but I also got few comments like: "this is old stuff, cloud is different" and "don't be nostalgic".
After those days, indeed, I've seen many articles comparing Cloud and SOA.

And it is natural: both the architectures (actually cloud is a consumption model more that a architecture) are based on the concept of Service. To be precise, to offer and consume cloud services you need to build a SOA.

It is easy to understand: to begin with, the consumer of a cloud service wants to delegate the build, the ownership and the operations to a third party, that assumes the responsibility for the SLA.
The service is considered a function that someone else provides to you, and you only care the interface to access it (and the quality and the price). You are interested only in the protocol and the user interface - or the API - plus the URL where you get the service.

The actual implementation is not your business. The service (IaaS, PaaS, SaaS) can run on any platform, in any part of the world, fully automated or manual, implemented in any of the hundreds of programming languages. You just don't care, as long as they respect the SLA.

Definitions

The most known definition of cloud computing is from NIST:
 

While SOA was defined, when I was at BEA Systems (one of the SOA pioneers), in this way:

SOA is an architectural approach that enables the creation of loosely coupled, 

interoperable business services that can be easily shared  

within and between enterprises.

A slightly more technical definition is: "Service-Oriented Architecture is an IT strategy that organizes the discrete functions contained in enterprise applications into interoperable, standards-based services that can be combined and reused quickly to meet business needs.

You can find a discussion of the SOA reference architecture (sorry, it's limited to my italian readers...) here. Also IBM has a good definition of SOA here.

 

SOA concepts that apply to Cloud 

There are some concepts that you find in both the models: each one would deserve a dedicated post, or maybe a book. I will try to give some essential detail in this post.

• The concept of Service: Consumer and Provider’s responsibility
• Distributed systems, where remote API are invoked over standard protocols
• Separation of concerns: interface vs implementation
• Interface and Contract
• Reuse and Loose Coupling
• Service Repository and Service Catalog
• Service Lifecycle
• Service Assurance
• Strategy and Governance

Basic detail 

 

Distributed systems

A distributed system is made of components that are deployed separately, in most cases remotely. Each of them provides a lower level functionality that can be used as a building block for the solution of a business need.
To inter-operate, they need connectivity and a well defined framework for sending and receiving data, managing security, transactions consistency, availability and many other non-functional requirements.

To make the development of such a complex system easier, the software industry has separated the concept of interface from the actual implementation.
The interface of a sw component specifies the functions it implements, the parameters it expects and  returns, their format, the conversation style (sync/async) and the security constraints. It is an artifact that can be produced - and deployed - before the actual implementation is ready: you can generate a stub (or mock) component that always returns fake data, but at least it replies to clients allowing the end to end test of the architecture.

So different developers can split the implementation of the system in components that are built in parallel, based on the definition of the interface that they present to each other. The basic integration test can be executed against a stub, to ensure that the conversation works. This also helps rapid prototyping and agile development.

The separation of the interface from the implementation is fundamental when a distributed system is designed.


A Service = Contract + Interface + Implementation 
The set of the above mentioned artifacts identifies a service.

As I stated, the implementation is not relevant for the consumer of the service - but it must exist, otherwise the service cannot be delivered.

The interface is the only visible part of the service, because the consumer will use that one. Depending on the service, it could be a GUI or the API that a client program invokes.

The most important part is the Contract: the agreement (generally defined in a document) defining who has the right to consume the service, the credentials, the price, the SLA, the constraints (e.g. the response time is granted up to 1000 transactions per second), and more.

A given interface could be offered with two distinct contracts, e.g. with different security requirements. Or different price, or different SLA, ect.
If you do that, a new service is generated (a different triple of contract+interface+implementation):

And of course you can differentiate the interface (e.g. sysnchronous vs asynchronous, that is pretty easy if you use a service bus). Also the addition of a new interface will generate a new service:

Reuse and Loose Coupling 

The effort of building a service in a way that makes it reusable is bigger than just implementing a local component in a software project.
Potential consumers of the service will trust it if it is robust enough, it scales, it is secure, etc.
You need to provide information on what the service does, how to use it, how do you support it.
So a business justification is needed for the additional effort to create a reusable service, both for internal usage (SOA) or as a cloud service.

The integration between service consumers and providers should not create tight dependencies, to allow for innovation and maintenance. Coupling refers to the degree of direct knowledge that one element has of another. The separation of the interface from the implementation plays an important role here, because one could change the implementation without affecting the published interface.
In case of major changes, versioning the interface helps.
See also these definitions of loose coupling on Wikipedia and Techtarget.

Service Repository and Service Catalog

I said that you need to provide information on the service and, eventually, market it. If potential consumers don't know that it exists, they will never use it. They also need descriptive info and technical details.
This is true when you build services for the enterprise architecture, even more if you want to sell them in the cloud. 

An important element of the Service Oriented Architecture was the Service Repository. A central point where all the artifacts produced by projects are exposed for reuse, complemented by the Registry offering a link to the service end points.
Now we have the concept of Service Catalog, managing the entire life cycle of a cloud service: from the inception to the decommissioning, passing through cost models and tenants management.
You can find a definition of a service catalog and its usage in this excellent free book: Defining IT Success Through the Service Catalog

 

Service Lifecycle

When a new service is created, you need to design its provisioning process - that could include fully automated or manual steps, including authorizations - its cost model, the management of the resources allocated for a tenant, the assurance of the quality of the service, the billing and end user reporting, the decommissioning and returning the resource to the shared pool.

It is good to have tools to manage all these phases of the life cycle. A choice of CMS (Cloud Management Systems) is offered by Cisco, that have a solution for a ready to run cloud implementation with pre built services (Cisco Intelligent Automation for Cloud, aka IAC) and the just released Cisco ONE Enterprise Cloud suite, a flexible environment where you can create new services with a very little effort, in a bottom-up approach (from the infrastructure to the catalog).
Both the suites use Cisco Prime Service Catalog (PSC) and the front end. PSC is ranked very high by analysts when they examine the features of service catalogs on market.

 

Service Assurance

Monitoring the infrastructure is essential, if you are a service provider. But it is not enough, because you can't immediately correlate the health status of the infrastructure with the quality of the services that consumers perceive (availability, response time, completeness of the result...).
More sophisticated tools are needed to report the services heath score to the Operations team and to the end users, and to allow troubleshooting.
Root cause analysis is the investigation of the ultimate cause for a service failure that could be due to software, servers, network, storage.
Impact analysis is the notification of the list of services impacted by a fault in the infrastructure, that helps the Operations team to restore the services before consumers complain for a violation of the SLA.

Strategy and Governance

IT governance provides the framework and structure that links IT resources and information to enterprise goals and strategies. Furthermore, IT governance institutionalizes best practices for planning, acquiring, implementing, and monitoring IT performance, to ensure that the enterprise's IT assets support its business objectives.

In recent years, IT governance has become integral to the effective governance of the modern enterprise. Businesses are increasingly dependent on IT to support critical business functions and processes; and to successfully gain competitive advantage, businesses need to manage effectively the complex technology that is pervasive throughout the organization, in order to respond quickly and safely to business needs.

In addition, regulatory environments around the world are increasingly mandating stricter enterprise control over information, driven by increasing reports of information system disasters and electronic fraud. The management of IT-related risk is now widely accepted as a key part of enterprise governance.

It follows that an IT governance strategy, and an appropriate organization for implementing the strategy, must be established with the backing of top management, clarifying who owns the enterprise's IT resources, and, in particular, who has ultimate responsibility for their enterprise-wide integration.

I discussed this topic with reference to SOA (only in italian, again... sorry) in SOA è solo tecnologia? and in

6 errori da non fare in un progetto SOA

 

Enterprise Service Bus

The ESB is a core component in the SOA Reference Architecture. It has the role of a mediation layer between the consumers and the providers of any service, managing the match of available interfaces, the security, the quotas and - in general - the enforcement of the Contract.
The ESB is the backbone of a Enterprise Architecture where new projects benefit from reusing already implemented services.

When you think about cloud, the public interface to available services is offered publicly to consumers. Very often, it consists in a set of API to provision and consume the services. A ESB is not strictly required to expose your implementation as a service, but it can certainly help.
Creating multiple interfaces, as long as new contracts are defined for a service, is just a few clicks activity. There are many ESB available as commercial products, next paragraph shows one example but the same capabilities are commonly available on the market and in the open source.

ESB Core Capabilities (courtesy of Mule Soft - http://www.mulesoft.com/platform/soa/mule-esb-open-source-esb):

• Service Mediation
  Separate business logic from protocols and message formats for rapid, nimble development and long-term flexibility.
• Service Orchestration
  Coordinate and arrange multiple services and expose them as a second-generation composite application.
• Service Creation & Hosting
  Expose app functionality as a service and create an efficient standards-based architecture or host existing services in lightweight containers.
• Message Routing
  Direct messages based on content or predetermined rules and filter, aggregate, or re-sequence as required.
• Data Transformation
  Transform data to and from any format across heterogeneous transport protocols and data types or enhance incomplete messages.
• Event Handling
  Deliver synchronous and asynchronous events, transactions, streaming, routing patterns, and a SEDA architecture.

So are SOA and Cloud identical?

Of course not. They have a lot of common concerns, but while SOA was created to address IT and business needs in a single Enterprise context, Cloud is a wider model that offers commercial services across companies.
There's still the private cloud model, where services are offered internally.
Here we have the same self service consumption model, so the automation of the provisioning is critical as well as the quality of the Service Catalog that you offer to consumers.

The most important lesson from SOA that we can reuse in Cloud is that the human factor is sometimes more impactful than the technology.
Change management is one of the key initiatives that help winning the resistance (both in the IT organization, when a new operational model is adopted, and across consumers that are offered a new way of using applications or implementing new projects). 

A proper documentation of the services is key, and the definition of a go-to-market strategy before you start your journey is fundamental: technology should not be adopted because it's smart or because others are doing the same.
It should always be functional to business requirements and be aligned with the corporate strategy.

Links

http://en.wikipedia.org/wiki/Service-oriented_architecture

http://www.alleywatch.com/2013/03/people-as-a-service/
http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf
http://www-01.ibm.com/software/solutions/soa/what-is-soa.html
http://www.mulesoft.com/platform/soa/mule-esb-open-source-esb

ACI for Simple Minds

Cisco ACI means Application Centric Infrastructure 

Why application designers and developers don't want to speak to network engineers. 

In my previous life I was an enterprise architect and I led design and development of software systems in many projects. When we were in the phase of planning the procurement and the setup of the various environments for the project (dev, test, QA, prod) I was bored by the meetings with the infrastructure guys.
What I needed was a given amount of memory and CPU power, that I could calculate myself, on a single big server or on a number of smaller machines. Then I needed connectivity among the different deployment units in my architecture (a cluster of web servers, a cluster of application servers, a database and some pre-existing systems), and just some services like load balancing.
But those nasty network engineers and the ugly security guys wanted to discuss a long list of requirements and settings: vlans, ip addresses, subnets, firewalls, quality of service, access lists   :-(
I was only interested in application tiers and dependencies, SLA, application performances and compliance and I wanted to discuss that in my language, not in their unfamiliar slang.

How the system engineers see the world: a number of devices with their configuration.

 

How I see the world: a number of servers (or processes) with their role in the application. We can call them End Points.

The communication can be described as a contract.

Provided by some end points, consumed by others. 

And saved as a reusable policy, that could be applied to End Point Groups:

 

 

Eventually, network services like load balancers or firewalls can be added (creating a service graph):

You can easily understand that our meetings were not that easy    ;-)
It was not their fault (and of course it was not... mine): we only saw the world from different angles, or maybe with different glasses.

For the software guys, abstracting the topology of the deployment is essential. For the system guys, devil is in the detail and they need to know exactly what traffic is flowing to engineer the setup accordingly.

Having a set of policies that describe the desired behavior makes the conversation easy: what service is offered by a end point group, what group (or single end point) can consume it, what SLA should be enforced, etc... Contracts could be: access to a web application on http on port 80, access to montoring agents or to log collectors, access to a LDAP server for authentication and so on.
You will see later in the post that ACI allows this conversation.

After long discussions and escalations, the setup of the environment was never as fast as I needed.
It was not the DevOps time yet, but we still tried to roll out many builds of the application for rapid prototyping and quick wins.

Now imagine that you were able to agree on the policy definition.
Having an instant enforcement of those policies on all your network devices without touching them one by one, in a consistent way that prevents human errors and grants compliance by default, and have it done soon would be a miracle... or a magic.

Now we have a network architecture that makes this miracle real: Cisco ACI.
A single software controller (redundant, of course) manages all the network connectivity, security and the network services like load balancers and firewalls.
The network is a hardware fabric, with great performances, scalability, resiliency that I will not discuss here (see the links below for the detail), that extends smoothly into the virtual networks of any vendor or open source solution enforcing the policies for physical servers and VM as end points without any difference.
The controller (named APIC) has a GUI but, most important, a rich set of open API that can be invoked by your scripts, by orchestration tools from Cisco or 3rd parties, by cloud management systems.
You can create the policy from here, and also see the "telemetry" of the network with easy display of the health score of the fabric or individual applications as well.

Use cases for ACI

Fast provisioning

A stateless network like ACI can be provisioned and completely reshaped in seconds by pushing new policies through the controller.
This concept is pretty similar to what the UCS Service Profile made possible in the server industry, introducing the stateless computing.
You can add the complete configuration for a new application to a multitenant shared infrastructure, you can create a new tenant environment, you can create the test environment and the production environment just by cloning the development environment and applying any needed policies to ensure compliance.
Everything is represented as a XML document or a JSON data structure: in any case a small piece of text data that can be saved, versioned and built automatically by a automation script or tool.
Infrastructure as code is one of the pillars of DevOps.

Physical and virtual networking managed the same

When you design End Point Groups and their Contracts, they can be mapped to physical servers (i.e. servers running a single Operating System, like Linux, Unix or Windows servers) or to VM running on any hypervisor.
Traffic from a VM is encapsulated and isolated from other VM's, then the policies are applied to allow it to flow to the destination (physical or virtual).
The spine-leaf architecture of the fabric is extended by a virtual leaf that runs in the hypervisor, under the control of the APIC Controller.

Service Graph

The integration of network services (LB, FW, etc.) from 3rd parties is easy thanks to the Opflex protocol, that allows the extension of the declarative style (vs imperative) of the configuration.
You can add the services to a contract and all the end point that offer and consume that contract will benefit from the insertion without any need for local configuration (e.g. changing the default gateway to the newly inserted firewall).
Many 3rd party vendors added - or will add soon - Opflex agents to their product. The wide ecosystem of ACI becomes richer every day.

Easy deprovisioning

Often, when an existing application is deprovisioned, the network and security configurations created for it are not deleted.
This is due to different reasons: either because it's hard to find them (not all the organizations use a CMDB to track everything) or because the responsible for the operation is afraid of removing firewall rules, ACL, vlans that could potentially be used by a different application, creating a problem as a consequence.

If you have all the policies for a given application defined as attributes of a specific Application Network Profile in APIC, simply removing that ANP will clean all the configurations. You were not applying rules to ports (where other application could be attached) but to end points.

Finally, what advantages can you get from ACI?

Centralized Policy-Defined Automation Management
 • Holistic application-based solution that delivers flexibility and automation for agile IT
 • Automatic fabric deployment and configuration with single point of management
 • Automation of repetitive tasks, reducing configuration errors

Open and Comprehensive End-to-End Security
 • Open APIs, open standards, and open source elements that enable software flexibility for DevOps teams, and firewall and application delivery controller (ADC) ecosystem partner integration
 • Automatic capture of all configuration changes integrated with existing audit and compliance tracking solutions
 • Detailed role-based access control (RBAC) with fine-grained fabric segmentation

Real-Time Visibility and Application Health Score
 • Centralized real-time health monitoring of physical and virtual networks
 • Instant visibility into application performance combined with intelligent placement decisions
 • Faster troubleshooting for day-2 operation

Application Agility
 • Management of application lifecycle from development, to deployment, to decommissioning in minutes
 • Automatic application deployment and faster provisioning based on predefined profiles
 • Continuous and rapid delivery of virtualized and distributed applications

If you liked this post, you may want to read also ACI for (smarter) Simple Minds. You have passed the basic stage now   :-)

Links

Serious product documentation

ACI Marketing page
ACI at a glance
ACI in one page
Application Centric Infrastructure (ACI) Documentation
Learning ACI - Adam's blog

Cartoons (2 min. each)

2015 ACI Video Episode 1: An Impossible Deadline

2015 ACI Video Episode 2: The Needle in the Haystack 

2015 ACI Video Episode 3: Operation Clean Up

DevOps - Tools and Technology

This post is the continuation of the DevOps - Operational model post in this blog.

We have seen how DevOps processes and organization can help the agility of IT, enabling a huge value for the business.

Let’s investigate the tools that smart organizations use to implement DevOps in the real world.

And let’s try to understand how, in addition to code management, the lifecycle of a sw application can be optimized by managing the infrastructure as code.

At the end of the day, we want to apply the following picture to the infrastructure as well.

 

Usually different environments are created to run application, often cloned for each Tenant (customer, project...): development, integration test, QA test, production, Disaster Recovery

The infrastructure must provide similar topology and functions, with different scale and HA requirements.

Those environments are sometimes used for few days, then they are no longer needed and the resources could be reused for next project.

If we were able to generate a new environment "end to end" when it is required, and to release all the resources to a shared pool, this  would help a lot in the optimization of resources usage.

The economy of scale provided by shared infrastructure and resource pools will add to the simplicity and speed of the operations.

The following picture shows the cycle of the builds (for both the sw application and the infrastructure) that optimizes the time and the resources.

From http://www.rightscale.com/blog/cloud-management-best-practices/continuous-integration-and-delivery-cloud-how-rightscale-does-it:

There are a number of tools and solutions that can help automating this process.

Some apply to specific phases, other to the end to end DevOps.

You can find a good list at http://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software

Also collaboration tools help the team(s) to work together for their own and the entire company's benefit: from http://www.collab.net/solutions/devops

The most used DevOps tools, as far as I know from direct experience and investigation, are jenkins, vagrant, puppet, chef.

Here is another possible chain of tools that cover the entire process:

From http://pages.cloudbees.com/rs/cloudbees/images/Continuous-Delivery-Infographic.pdf

Stateless Infrastructure (also known as SDDC)

We understood that the maximum benefit comes from being able to create and destroy environment on demand, allocate resources just when needed (we can also consider Disaster Recovery as a important use case in this scenario, but in that case you should also ensure that data have been replicated before the event).

Infrastructure as code is a core capability of DevOps that allows organizations to manage the scale and the speed with which environments need to be provisioned and configured to enable continuous delivery.

Evolving around the notion of infrastructure as code is the notion of software-defined environments.

Whereas infrastructure as code deals with capturing node definitions and configurations as code, software-defined environments use technologies that define entire systems made up of multiple nodes — not just their configurations, but also their definitions, topologies, roles, relationships, workloads and workload policies, and behavior.

Stateless Computing and Stateless Networking are important innovations that some vendors (Cisco could be considered a leader here) brought to the market in last 5 years.

Policy based configuration and the availability of software controllers for all the components of the architecture allow the separation of the modeling from the physical topology.

Servers

As an example, UCS servers (up to 160 in one management domain, but domains can be joined to share resources and policies) are stateless.

You can imagine each server (either a blade or a rack-mount server) as a dumb piece of iron, before you push its identity, its features (e.g. number, type and configuration of the network interfaces) and its behavior as a piece of configuration.

It is like adding the soul to a body.

Later you can move the same soul to a different body (maybe more powerful, such as from a 2-CPU server to a 4-CPU one). The new machine will be restarted as if it was the same.

This can be useful to recover a faulty server, to do DR but also to repurpose a server farm in few minutes (and eventually restore the previous state the day after).

The state (identity, features and behavior) is defined by a XML document that can be stored, versioned and managed as code in a repository (other than in the embedded UCS Manager).

This abstraction of the server from the actual machine makes the management easier and was the main factor for the incredible success of UCS as a server platform.

Networks

Similarly, in the networking domain, we have had a quantum leap in network management with Cisco ACI (Application Centric Infrastructure).

For those that have not met ACI yet, I have published a “ACI for Dummies” post.

In few words, ACI brings the management of physical and virtual networks together.

It has a very performant and scalable fabric, made of spine and leaf switches, that are managed by a software controller called APIC.

APIC also integrates the virtual switches in the different hypervisors, so that its policy model can be extended to the virtual end points.

A GUI is provided to manage APIC, but essentially you would drive it through the excellent open API offered to orchestration systems and - of course - DevOps tools.

XML (or JSON) artifacts can be stored in a repository as code, and pushing them to APIC will create your new Data Center on the fly.

You can create new Tenants with dedicated resources, or deploy the infrastructure for a new application in such a way that it is isolated (in terms of security, performances and stability) from others, though running on a shared infrastructure.

It would take just the time of a REST call, where you push the new policy to the controller.

And of course you could use the same templates in the different environments: development, integration test, QA test, production, Disaster Recovery

The previous generation of network devices (e.g. the Nexus family) can be managed in a DevOps scenario as well.

They offer API and have puppet agents onboard. And a version of the APIC controller has been created also for networks outside ACI (APIC-EM - https://developer.cisco.com/site/apic-em/discover/overview/).

The Cisco DevNet community prodives a lot of information and samples at https://developer.cisco.com/site/devnet/home/index.gsp

I wrote a short post on Ansible here: http://lucarelandini.blogspot.com/2015/05/a-powerful-devops-tool-ansible.html where a great recorded session from the Openstack Summit is linked.

You might be interested also in my post on DevOps, Docker and Cisco ACI.

 

DevOps - Operational model

This posts is the continuation of the Why DevOps: definition and business benefit post.

As it happens in other areas of the IT, technology is an important factor for success but it is not the most important one.

The human factor is what really makes the difference for successful projects.

So skills, common goals, organization and governance (and a business strategy) will make you win with any tool.

But if you lack them, the best technology in the world will fail to provide a positive outcome.

In this post we’ll see how a lot of companies have adopted DevOps practices, using a variety of products (that we'll examine next time), and they got a important return.

From http://www.slideshare.net/ITRevolution/tuesday-315-stephen-elliot-final-40879014:

Why Project Fail: The Business Management Chasm

Question: Over the past year, what percentage of your current projects have failed to meet your success criteria?

Answer: 19% (n=84)

Question: Why?

Answer:

1. Poor requirements gathering/scope creep: 23%
2. Lack of resources (staff and budget): 21%
3. Changed business priorities: 19%
4. Lack of business stakeholders ownership: 16%
5. Testing delays: 10%
6. User requirements changes: 10%
7. Vendor performance: 1% 

If you sum up points 3 and 4 you get 35%.
You can easily see that if the application lifecycle was leaner and faster, they wouldn't lose their chances for success.
Quick wins are the most important key to lead a project to its final goal: you should deliver a tangible value as early as possible, to keep traction, and be able to react to changes

In this post we’ll see how a lot of companies have adopted DevOps practices, using a variety of products, and they got a important return.

From http://www.businesscomputingworld.co.uk/7-key-pre-requisites-for-a-successful-transition-to-continuous-delivery/ 

Businesses today are moving toward continuous delivery as a methodology and tool to meet the ever-increasing demand to deliver better software faster. Continuous delivery, with its emphasis on keeping software in a release-ready state at all times, can be seen as a natural evolution from continuous integration and agile software development practices. However, the cultural and operational challenges to achieve continuous delivery are even greater.

For most organisations, continuous delivery requires adaptation and extension of existing software release processes. The roles, relationships, and responsibilities of people across the organisation may be impacted. The tools used to deliver, update, and maintain software must support automation and collaboration properly, minimising delays and providing tight feedback cycles across the organisation. While these changes can be a huge challenge to implement for organisations that must live within regulatory and operational constraints, there are many practical steps you can take to make real progress today.

With that in mind, here are 7 key pre-requisites organisations should consider when making a successful transition to Continuous Delivery.

1. Make Sure Development, QA & Operations Teams Have Shared Goals & Communicate

2. Get Continuous Integration Right Before Making The Step To Continuous Delivery

3. Automate & Version Everything

4. Share Tools & Procedures Between Teams

5. Make Your Application Production-Friendly: Make Deployments Non-Events

6. Make Your Infrastructure Project-Friendly: Empower The People & The Teams

7. Make Application Versions Ready To Be Shipped Into Production

Continuous Delivery is not just about a set of tools, ultimately it is also about the people and organisational culture. Technology, people and process all have to be aligned to make Continuous Delivery successful in any organisation, a collaborative approach is fundamental to its success. If organisations are to reap the rewards of a more fluid, automated approach to software development that can also provide them business agility – they need to implement these best practice steps on the path to Continuous Delivery.

From http://blog.devopsguys.com/2013/04/02/are-you-really-doing-devops-8-prerequisites-you-must-consider/

(1) “ Emphasize the performance of the entire system” – a holistic viewpoint from requirements all the way through to Operations
(2) “Creating feedback loops” – to ensure that corrections can continually be made. A TQM philosophy, basically.
(3) “Creating a culture that fosters continual experimentation and understanding that repetition and practice are the pre-requisites to mastery”
These are excellent guidelines at a high level, but we’d like to see a more operational definition. So we’ve made up our own list!
As a starter – we propose that;

1. You must have identified executive sponsors / stake holders who you are actively working with to promote the DevOps approach.
2. You must have developed a clear understanding of your organisation’s “value chain” and how value is created (or destroyed) along that chain.
3. You must have organizationally re-structured your development and operations teams to create an integrated team – otherwise you’re still in Silos.
4. You must have changed your team incentives (e.g. bonus incentives) to reinforce that re-alignment – without shared Goals you’re still in Silos.
5. You must be seeking repeatable standardized processes for all key activities along the value chain (the “pre-requisite to mastery”)
6. You must be leveraging automation where possible – including continuous integration, automated deployments and “infrastructure as code”
7. You must be adopting robust processes to measure key metrics – PuppetLab’s report focuses on improvement in 4 key metrics – Change Frequency, Change Lead Time, Change Failure Rate and MTTR. We suggest Availability, Performance and MTBF should be in there too.
8. You must have identified well-defined feedback mechanisms to create continuous improvement.

Of course, you will need some investment to get there. It can be gradual and the payback from the adoption of DevOps will help next steps:

Two main processes that make DevOps work are Continuous Integration and Continuous Delivery.

From http://en.wikipedia.org/wiki/Continuous_integration

Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.

CI was originally intended to be used in combination with automated unit tests written through the practices of test-driven development. Initially this was conceived of as running all unit tests in the developer's local environment and verifying they all passed before committing to the mainline.

Later elaborations of the concept introduced build servers, which automatically run the unit tests periodically or even after every commit and report the results to the developers.

In addition to automated unit tests, organisations using CI typically use a build server to implement continuous processes of applying quality control in general — small pieces of effort, applied frequently. In addition to running the unit and integration tests, such processes run additional static and dynamic tests, measure and profile performance, extract and format documentation from the source code and facilitate manual QA processes. This continuous application of quality control aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

Best practices

2.1 Maintain a code repository

2.2 Automate the build

2.3 Make the build self-testing

2.4 Everyone commits to the baseline every day

2.5 Every commit (to baseline) should be built

2.6 Keep the build fast

2.7 Test in a clone of the production environment

2.8 Make it easy to get the latest deliverables

2.9 Everyone can see the results of the latest build

2.10 Automate deployment

From http://en.wikipedia.org/wiki/Continuous_delivery

Continuous Delivery (CD) is a design practice used in software development to automate and improve the process of software delivery. Techniques such as automated testing, continuous integration and continuous deployment allow software to be developed to a high standard and easily packaged and deployed to test environments, resulting in the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead. The technique was one of the assumptions of extreme programming but at an enterprise level has developed into a discipline of its own, with job descriptions for roles such as "buildmaster" calling for CD skills as mandatory.

Continuous delivery defines a deployment pipeline as a set of validations through which a piece of software must pass on its way to release. Code is compiled if necessary and then packaged by a build server every time a change is committed to a source control repository, then tested by a number of different techniques (possibly including manual testing) before it can be marked as releasable.

From http://puppetlabs.com/sites/default/files/Characteristics_Successful_DevOps_Team_0.pdf

Characteristics of a Successful DevOps Team

No matter how you’re using DevOps practices — whether your company has a DevOps department or cross-functional teams that share DevOps tools and practices — there are distinct characteristics of DevOps teams that align with high IT performance.

Here’s a checklist that’s food for thought (and fuel for future improvement!).

These points are drawn from the 2014 State of DevOps Report, and from suggestions of DevOps experts like Paul Duvall, Jez Humble and Joanne Molesky. 

Effective DevOps teams don’t think of issues as “someone else’s problem”. 

Developers, IT operations, quality assurance engineers, database admins, and business analysts collaborate, and everyone checks code into the version control system. Everyone is part of the delivery process — and held accountable for it.

We Automate Build, Deployment, and Testing Phases.

With automation, you reduce the chance of human error as you transition code from one phase to the next. Because you’re automating configuration of all environments, you’re minimizing issues caused by writing code in a development environment that is different from the production environment.

Our Culture Reflects Open Communication and Collaboration.

Developers and IT operations attend planning meetings, standups, and release postmortems. Developers share responsibility for writing testable and deployable code, and if code fails in production, the team is kept in the loop, working together to review causes and identify solutions. 

We Have Routine Deployment Processes and Shared Monitoring Practices.

Team members can accurately report how long it’ll take to deploy a new feature, or even a few lines of code, to production. They can identify and remove roadblocks, without a lot of red tape. They understand the key performance and availability metrics to measure, and track them against larger business goals.

We Implement a Continuous Delivery Pipeline.

Continuous delivery, implemented right, lets you release changes continually to production. That lets you testnew features with real customers, facilitating quick feedback about how they’re being used. Continuous delivery helps companies make better business decisions and move more quickly than their competitors.

We Use Version Control For All Production Artifacts.

Version control systems help you track changes and quickly find the source of an error, reducing time to recovery. Everything required to launch a change into the production environment must be checked into version control, including application code, application and system configurations, tests, and deployment scripts.

We Trust Each Other, and Collectively Enable Continous Improvement.

We deliver on our promises to the business, and to our customers. We continually work on developing collaboration, clear communication and trust between team members. We are continually learning and improving as a team. Most important of all: We spend less time fighting fires and more time focusing on great work.

https://puppetlabs.com/blog/top-benefits-continuous-delivery-overview

When it’s well executed, continuous delivery allows an organization to respond more quickly to its market and to customers, both internal and external. It also makes life saner for people in IT operations, software development and quality testing teams. Instead of long periods of development punctuated by looming deadlines, big dramatic releases and panicked remediation of serious bugs, software releases are small, predictable and less dramatic… even boring :-)

Top Benefits of Continuous Delivery

Deliver software with fewer bugs and lower risk.
When you release smaller changes more frequently, you catch errors much earlier in the development process. When you implement automated testing at every stage of development, you don’t pass failed code to the next stage. And it’s easier to roll back smaller changes when you need to.

Release new features to market more frequently — and learn.
Releasing new features early and often — even in a minimally viable state — means you get more frequent feedback, giving you the ability to iterate and learn from your customers. Enlisting customers as development partners gives them a sense of co-ownership and loyalty, and makes them more likely to forgive when you stumble.

Respond to marketing conditions more quickly.
Market conditions change constantly. Whether you’ve just discovered a new product is losing money, or that more customers are visiting your site from smartphones than laptops, it’s much easier to make a fast change if you are already practicing continuous delivery.

Life is saner for everyone: IT operations, software development, QA, product owners and business line owners.

Continuous delivery means the responsibility for software delivery is distributed much more widely, and this shared responsibility and collaboration make life better. Continuous delivery also take a lot of stress out of software release. Releasing smaller changes more often gets everyone used to a regular, predictable pace, leaving room tocome up with ideas and actually enjoy your the work. Best of all, a successful release becomes a shared success, one you can all celebrate together.

In next post, we’ll discuss the most used tools for DevOps and how the infrastructure can be managed “as code”, that means dynamically provisioned creating the needed environment every time you deploy a new version of the code.

Link to the DevOps - Tools and Technology post.

Sources:

http://www.slideshare.net/ITRevolution/tuesday-315-stephen-elliot-final-40879014

http://www.businesscomputingworld.co.uk/7-key-pre-requisites-for-a-successful-transition-to-continuous-delivery/ 

http://blog.devopsguys.com/2013/04/02/are-you-really-doing-devops-8-prerequisites-you-must-consider/

http://puppetlabs.com/2014-devops-report

http://puppetlabs.com/sites/default/files/Characteristics_Successful_DevOps_Team_0.pdf

https://puppetlabs.com/blog/top-benefits-continuous-delivery-overview

Why DevOps: definition and business benefit

Can you just imagine the magic of pushing one button and see your company’s new project materialize in a production environment? The software application code compiled and built, the data center infrastructure (heterogeneous and complex) set up, the application deployed and tested - ensuring compliance - and the business stakeholders given their beloved new campaign ready to use?

I’m working with colleagues and customers to better understand why there’s so much interest around DevOps, what are the business benefit and useful technology.

Here is that state of our reasoning, along with some notes that I collected in my research.

To make the matter easier for people that don’t have experience in managing the software release cycle, I imagined to take a triangle approach: analyse the business drivers that need to be addressed, what is a operational model that could provide the expected results and, finally, what is the enabling technology. With this top-down approach, understanding the concept gradually should be easier for IT professionals that are not expert in the field.

Business Drivers

In every company, Lines Of Business have a dream: have a new solution live in 1 month. It could be a marketing campaign, a new service for their customers, a process to produce new goods.

They think they just need smart developers and the availability of the required infrastructure, that given the spending on IT should not be an obstacle.

Unfortunately, sometimes they feel that IT is not efficient enough. It’s not a matter of technology, but of organization.

Some notes from https://puppetlabs.com/blog/why-every-cfo-should-advocate-devops (Bill Koefoed, the author, is the chief financial officer of Puppet Labs).

IT is the manufacturing of the 21st century. Let’s face it, most products and services these days depend on software, from social media to teleconferencing to household appliances that interact via the internet.
To get ahead of competitors, you have to get your new products and services out fast, test them for customer response, and quickly update to satisfy customer desires. Even as you’re increasing your rate of output, you have to reduce flaws, whether in delivery or the product itself.
That’s why DevOps is so important: The tools, practices and cultural orientation of DevOps enable greater efficiency in IT. Our 2014 State of DevOps report bears this out, both in terms of software throughput and business results. From the standpoint of throughput, we validated last year’s findings that high-performing IT teams (as defined by deployment frequency, lead time for changes and mean time to recover from failure) deploy up to 30 times more frequently than their lower-performing peers, with 50 percent fewer failures. This year, the most provocative finding was the strong connection we found between IT performance and financial performance. Companies with high-performing IT teams got better business results, as they were:

• 3.3 times more likely to have met or exceeded the company’s productivity goals.

• 1.6 times more likely to have exceeded company profitability targets.

In this post I will describe DevOps in gereral terms, with some focus on the Business Benefit.

In my next posts I will approach the Operational Model and investigate the technology that can help. Not only tools for Continuous Integration and Continuous Delivery, but also the concept of “infrastructure as code” that allows a flexible and agile use of the infrastructure resources in the same cycle as the software for the applications.

http://puppetlabs.com/presentations/how-do-we-better-sell-devops

From http://en.wikipedia.org/wiki/DevOps:

DevOps is a software development method that stresses communication, collaboration (information sharing and web service usage), integration, automation and measurement between software developers and Information Technology (IT) professionals. DevOps is a response to the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services and to improve operations performance - quality assurance.

The specific goals of a DevOps approach span the entire delivery pipeline, they include improved deployment frequency, which can lead to faster time to market, lower failure rate of new releases, shortened lead time between fixes, and faster mean time to recovery in the event of a new release crashing or otherwise disabling the current system. Simple processes become increasingly programmable and dynamic, using a DevOps approach, which aims to maximize the predictability, efficiency, security, and maintainability of operational processes. Very often, automation supports this objective.

Development methodologies (such as agile software development) that are adopted in a traditional organization with separate departments for Development, IT Operations and QA, development and deployment activities, previously do not have deep cross-departmental integration with IT support or QA. DevOps promotes a set of processes and methods for thinking about communication and collaboration between departments.

The adoption of DevOps is being driven by factors such as:

1. Use of agile and other development processes and methodologies
2. Demand for an increased rate of production releases from application and business unit stakeholders
3. Wide availability of virtualized and cloud infrastructure from internal and external providers
4. Increased usage of data center automation and configuration management tools

Use Cases

Automating the release of a complete system can provide advantages in the following situations (a partial list - if you have more reference, please add a comment to this post):

1. daily builds in the development environment
2. move the system to the integration environment and to QA
3. regression testing after a patch
4. move a system from testing to production
5. deploy a copy of the system for a new tenant
6. copy a system to Disaster Recovery

Keep in mind that, thanks to the management of "infrastructure as code", you can have the end to end system managed this way... not only the software code.

Adoption in the world
DevOps is ramping in the US, it seems to be a little late in Europe - as many innovations in the IT.
Companies that benefit already from the introduction of this methodology are:
Google, Amazon, Netflix, Facebook, Twitter, Pinterest, Bank of America, Cisco and more...

From https://puppetlabs.com/sites/default/files/CDebook.pdf:

Industry headlines tell us every day that companies rise and fall on moments of infectious delight and irritated disappointment. It's not enough to have a great idea and execute on it once. You have to execute, get feedback, refine, and execute again - and again and again. To keep competitors from grabbing a piece of your market, you need to cycle with ever-increasing speed and agility.

Multiple, independently conducted research studies show that, not only are enterprises already adopting DevOps, they are achieving substantial outcomes.
One such study, conducted by independent research organization IDG, shows that enterprises (measured by having more than $500 million in revenues) are adopting DevOps at an even faster rate than smaller businesses. Another study, conducted by independent research firm Vanson Bourne, found that large enterprises are not only adopting DevOps, but more than 90% have seen or expect to see significant benefits, with quantifiable improvements in delivery speed, development and operations costs, defect detection, ability to innovate, and many more, ranging from 17% to 23%. Then there is additional research from InformationWeek, which also shows high rates of adoption and benefits for large enterprises (measured by having 5,000 or more employees).

In next post, I’ll try to define a operational model:

what teams are involved

what processes do you need

what information do you need 

what roles do you need

what skills do you need 

Link to next post: DevOps - Operational Model.

Sources:

1 - https://puppetlabs.com/blog/why-every-cfo-should-advocate-devops (Bill Koefoed, the author, is the chief financial officer of Puppet Labs.)

2 - http://en.wikipedia.org/wiki/DevOps

3 - https://puppetlabs.com/sites/default/files/CDebook.pdf

4 - http://puppetlabs.com/presentations/how-do-we-better-sell-devops

5 - http://blogs.cisco.com/getyourbuildon/devops4networks

6 - TDD/CI - Test Driven Development/Continuous Integration - Colin McNamara

 

The Elastic Cloud project - Methodology

This posts is the continuation of the post The Elastic Cloud Project - Architecture.

Here I will explain how we worked in the project: the sequence of activities that were required and the basic technologies we adopted.

The concepts are mostly explained by using pictures and screen shots, because an image is often worth 1000 words.

If you are interested in more detail, please add a comment or send me a message: I’ll be glad to provide detailed information.

To begin with, we had to:

• map the data model of the products used to understand what objects should be created, for a Tenant, in all the layers of the architecture
• create sequence diagrams to make the interaction clear to all the members of the team - and to the customer
• understand how the API exposed by Openstack Neutron and from Cisco APIC work, how they are invoked and what results they produce
• implement workflows in the CPO orchestrator to call the APIC controller and reuse the existing services in Cisco IAC
• integrate Hyper-V compute nodes in Openstack Nova
• create a new service in the Service Catalog to order the deployment of our 3 tiers application

Some detail about the activities above:

1 - Map the data model of the products used to understand what objects should be created, for a Tenant, in all the layers of the architecture

know that some of you still don’t know Cisco ACI… I promise that I will post a “ACI for dummies” soon.   :-)

  

This picture shows how concepts in Openstack Neutron map to concepts in Cisco ACI:

2 - Create sequence diagrams to make the interaction clear to all the members of the team

3 - Understand how the API exposed by Openstack Neutron and from Cisco APIC work, how they are invoked and what results they produce

This is a call to the Cisco APIC controller, using XML

This is a call to the Openstack Nova API, using JSON:

to do this, we used a REST client to learn the individual behavior and how the parameters need to be passed

a REST call is essentially a http call (GET or POST) where the body contains XML or JSON documents

some http headers are required to specify the content type and to hold security information (like a token for single sign on, that is returned by the authorization call and you need to resend in all the following calls to be recognized.

So we adopted Google Postman, that is a plugin for the Chrome browser (latest version is also released as a standalone application) to practice with the REST Calls then,after we learned how to manage them, we just copied the same content (plus the headers) into the “http call” tasks in the CPO workflow editor.

The XML or JSON variables that we passed are essentially static documents with some placeholders for current values, i.e. the Tenant name, the Network name, etc. were passed according to the user input.

Of course the XML elements tags are described in the APIC product documentation, you don’t have to reverse engineer their meaning   ;-)

Another way to get the XML ready to use is to export it from the APIC user interface: if you select an object that has been created already (either though the GUI or the API), you can export the corresponding XML definition:

This is how we copied the XML content from the test made in Postman and replaced some elements with placeholders for current values (that are variables in the workflow designer):

This is how the variable appear in the workflow instance viewer, after you have executed the process because a user ordered the service:

4 - Implement workflows in the CPO orchestrator to call the APIC controller and reuse the existing services in Cisco IAC

An example of the services that Cisco IAC provides out of the box.

They are also available through the API exposed by the product, so we created a custom workflow that reused some of the services as building block for our use case implementation.

his is the workflow editor, where we created the orchestration flow:

5 - integrate Hyper-V

At the time of this project, a direct support for Microsoft Hyper-V was not available in Openstack Nova.

But a free library was available from Cloudbase, so we decided to install it on our Hyper-V serverso that the virtual data center (VDC) we had created in Cisco IAC thanks to the integration with Openstack could use also Hyper-V resources to provision the VM.

More detail on the integration can be found here: http://www.cloudbase.it/openstack/

In the current Openstack release (Juno), Hyper-V servers are managed directly.

6 - create a new service in the Service Catalog

Conclusion

This project had a complexity that derived from being the among the first teams in the world to try the integration of so many disparate technologies: Cisco software products for Service Catalog and Orchestration, three hypervisors (ESXi, Hyper-V, and KVM), physical networks (Cisco ACI) and virtual networks in all the hypervisors, Openstack.

I didn't tell you, but also load balancers and firewalls were integrated.

Maybe I will post some detail about the Layer 4 - Layer 7 service chaining in the next weeks.

We had to learn the concepts before learning the products. Actually theinvestigation of the API and their integration was the easiest part... and was also fun for my ancient memory of programmer   :-)

Now, with the current release of the products involved in this project, everything would be much easier.

Their features are more complete (actually the integration of the Neutron API in the management of Virtual Data Centers in ACI was fed back to our engineering during this project).

Skills available on the field are deeper and widespread.

I've already implemented the same use case with alternative architectures twice.

Cisco UCS Director was used once, replacing the IAC orchestration and pre-built services.

And, in another variation, the Openstack API were integrated directly instead of reusing the existing services that manage the Openstack VDC in IAC.

Just to have more fun... ;-)