February 12, 2015

DevOps - Tools and Technology

This post is the continuation of the DevOps - Operational model post in this blog.

We have seen how DevOps processes and organization can help the agility of IT, enabling a huge value for the business.
Let’s investigate the tools that smart organizations use to implement DevOps in the real world.
And let’s try to understand how, in addition to code management, the lifecycle of a sw application can be optimized by managing the infrastructure as code.
At the end of the day, we want to apply the following picture to the infrastructure as well.

 



Usually different environments are created to run application, often cloned for each Tenant (customer, project...): development, integration test, QA test, production, Disaster Recovery
The infrastructure must provide similar topology and functions, with different scale and HA requirements.
Those environments are sometimes used for few days, then they are no longer needed and the resources could be reused for next project.
If we were able to generate a new environment "end to end" when it is required, and to release all the resources to a shared pool, this  would help a lot in the optimization of resources usage.
The economy of scale provided by shared infrastructure and resource pools will add to the simplicity and speed of the operations.

The following picture shows the cycle of the builds (for both the sw application and the infrastructure) that optimizes the time and the resources.




There are a number of tools and solutions that can help automating this process.
Some apply to specific phases, other to the end to end DevOps.
Also collaboration tools help the team(s) to work together for their own and the entire company's benefit: from http://www.collab.net/solutions/devops



The most used DevOps tools, as far as I know from direct experience and investigation, are jenkins, vagrant, puppet, chef.
Here is another possible chain of tools that cover the entire process:


Stateless Infrastructure (also known as SDDC)

We understood that the maximum benefit comes from being able to create and destroy environment on demand, allocate resources just when needed (we can also consider Disaster Recovery as a important use case in this scenario, but in that case you should also ensure that data have been replicated before the event).

Infrastructure as code is a core capability of DevOps that allows organizations to manage the scale and the speed with which environments need to be provisioned and configured to enable continuous delivery.
Evolving around the notion of infrastructure as code is the notion of software-defined environments.
Whereas infrastructure as code deals with capturing node definitions and configurations as code, software-defined environments use technologies that define entire systems made up of multiple nodes — not just their configurations, but also their definitions, topologies, roles, relationships, workloads and workload policies, and behavior.

Stateless Computing and Stateless Networking are important innovations that some vendors (Cisco could be considered a leader here) brought to the market in last 5 years.
Policy based configuration and the availability of software controllers for all the components of the architecture allow the separation of the modeling from the physical topology.

Servers

As an example, UCS servers (up to 160 in one management domain, but domains can be joined to share resources and policies) are stateless.
You can imagine each server (either a blade or a rack-mount server) as a dumb piece of iron, before you push its identity, its features (e.g. number, type and configuration of the network interfaces) and its behavior as a piece of configuration.
It is like adding the soul to a body.
Later you can move the same soul to a different body (maybe more powerful, such as from a 2-CPU server to a 4-CPU one). The new machine will be restarted as if it was the same.
This can be useful to recover a faulty server, to do DR but also to repurpose a server farm in few minutes (and eventually restore the previous state the day after).
The state (identity, features and behavior) is defined by a XML document that can be stored, versioned and managed as code in a repository (other than in the embedded UCS Manager).
This abstraction of the server from the actual machine makes the management easier and was the main factor for the incredible success of UCS as a server platform.

Networks

Similarly, in the networking domain, we have had a quantum leap in network management with Cisco ACI (Application Centric Infrastructure).
For those that have not met ACI yet, I have published a “ACI for Dummies” post.
In few words, ACI brings the management of physical and virtual networks together.
It has a very performant and scalable fabric, made of spine and leaf switches, that are managed by a software controller called APIC.
APIC also integrates the virtual switches in the different hypervisors, so that its policy model can be extended to the virtual end points.
A GUI is provided to manage APIC, but essentially you would drive it through the excellent open API offered to orchestration systems and - of course - DevOps tools.
XML (or JSON) artifacts can be stored in a repository as code, and pushing them to APIC will create your new Data Center on the fly.
You can create new Tenants with dedicated resources, or deploy the infrastructure for a new application in such a way that it is isolated (in terms of security, performances and stability) from others, though running on a shared infrastructure.
It would take just the time of a REST call, where you push the new policy to the controller.
And of course you could use the same templates in the different environments: development, integration test, QA test, production, Disaster Recovery

The previous generation of network devices (e.g. the Nexus family) can be managed in a DevOps scenario as well.
They offer API and have puppet agents onboard. And a version of the APIC controller has been created also for networks outside ACI (APIC-EM - https://developer.cisco.com/site/apic-em/discover/overview/).
The Cisco DevNet community prodives a lot of information and samples at https://developer.cisco.com/site/devnet/home/index.gsp

I wrote a short post on Ansible here: http://lucarelandini.blogspot.com/2015/05/a-powerful-devops-tool-ansible.html where a great recorded session from the Openstack Summit is linked.

You might be interested also in my post on DevOps, Docker and Cisco ACI.

 
Published by No comments:
Tags: , , , , , , , , , , ,

February 2, 2015

DevOps - Operational model

This posts is the continuation of the Why DevOps: definition and business benefit post.

As it happens in other areas of the IT, technology is an important factor for success but it is not the most important one.
The human factor is what really makes the difference for successful projects.
So skills, common goals, organization and governance (and a business strategy) will make you win with any tool.
But if you lack them, the best technology in the world will fail to provide a positive outcome.

In this post we’ll see how a lot of companies have adopted DevOps practices, using a variety of products (that we'll examine next time), and they got a important return.

Why Project Fail: The Business Management Chasm

Question: Over the past year, what percentage of your current projects have failed to meet your success criteria?
Answer: 19% (n=84)
Question: Why?
Answer:
  1. Poor requirements gathering/scope creep: 23%
  2. Lack of resources (staff and budget): 21%
  3. Changed business priorities: 19%
  4. Lack of business stakeholders ownership: 16%
  5. Testing delays: 10%
  6. User requirements changes: 10%
  7. Vendor performance: 1% 
If you sum up points 3 and 4 you get 35%.
You can easily see that if the application lifecycle was leaner and faster, they wouldn't lose their chances for success.
Quick wins are the most important key to lead a project to its final goal: you should deliver a tangible value as early as possible, to keep traction, and be able to react to changes


In this post we’ll see how a lot of companies have adopted DevOps practices, using a variety of products, and they got a important return.

Businesses today are moving toward continuous delivery as a methodology and tool to meet the ever-increasing demand to deliver better software faster. Continuous delivery, with its emphasis on keeping software in a release-ready state at all times, can be seen as a natural evolution from continuous integration and agile software development practices. However, the cultural and operational challenges to achieve continuous delivery are even greater.
For most organisations, continuous delivery requires adaptation and extension of existing software release processes. The roles, relationships, and responsibilities of people across the organisation may be impacted. The tools used to deliver, update, and maintain software must support automation and collaboration properly, minimising delays and providing tight feedback cycles across the organisation. While these changes can be a huge challenge to implement for organisations that must live within regulatory and operational constraints, there are many practical steps you can take to make real progress today.

With that in mind, here are 7 key pre-requisites organisations should consider when making a successful transition to Continuous Delivery.
1. Make Sure Development, QA & Operations Teams Have Shared Goals & Communicate
2. Get Continuous Integration Right Before Making The Step To Continuous Delivery
3. Automate & Version Everything
4. Share Tools & Procedures Between Teams
5. Make Your Application Production-Friendly: Make Deployments Non-Events
6. Make Your Infrastructure Project-Friendly: Empower The People & The Teams
7. Make Application Versions Ready To Be Shipped Into Production

Continuous Delivery is not just about a set of tools, ultimately it is also about the people and organisational culture. Technology, people and process all have to be aligned to make Continuous Delivery successful in any organisation, a collaborative approach is fundamental to its success. If organisations are to reap the rewards of a more fluid, automated approach to software development that can also provide them business agility – they need to implement these best practice steps on the path to Continuous Delivery.


(1) “ Emphasize the performance of the entire system” – a holistic viewpoint from requirements all the way through to Operations
(2) “Creating feedback loops” – to ensure that corrections can continually be made. A TQM philosophy, basically.
(3) “Creating a culture that fosters continual experimentation and understanding that repetition and practice are the pre-requisites to mastery”
These are excellent guidelines at a high level, but we’d like to see a more operational definition. So we’ve made up our own list!
As a starter – we propose that;
  1. You must have identified executive sponsors / stake holders who you are actively working with to promote the DevOps approach.
  2. You must have developed a clear understanding of your organisation’s “value chain” and how value is created (or destroyed) along that chain.
  3. You must have organizationally re-structured your development and operations teams to create an integrated team – otherwise you’re still in Silos.
  4. You must have changed your team incentives (e.g. bonus incentives) to reinforce that re-alignment – without shared Goals you’re still in Silos.
  5. You must be seeking repeatable standardized processes for all key activities along the value chain (the “pre-requisite to mastery”)
  6. You must be leveraging automation where possible – including continuous integration, automated deployments and “infrastructure as code”
  7. You must be adopting robust processes to measure key metrics – PuppetLab’s report focuses on improvement in 4 key metrics – Change Frequency, Change Lead Time, Change Failure Rate and MTTR. We suggest Availability, Performance and MTBF should be in there too.
  8. You must have identified well-defined feedback mechanisms to create continuous improvement.


Of course, you will need some investment to get there. It can be gradual and the payback from the adoption of DevOps will help next steps:



Two main processes that make DevOps work are Continuous Integration and Continuous Delivery.

Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.
CI was originally intended to be used in combination with automated unit tests written through the practices of test-driven development. Initially this was conceived of as running all unit tests in the developer's local environment and verifying they all passed before committing to the mainline.
Later elaborations of the concept introduced build servers, which automatically run the unit tests periodically or even after every commit and report the results to the developers.
In addition to automated unit tests, organisations using CI typically use a build server to implement continuous processes of applying quality control in general — small pieces of effort, applied frequently. In addition to running the unit and integration tests, such processes run additional static and dynamic tests, measure and profile performance, extract and format documentation from the source code and facilitate manual QA processes. This continuous application of quality control aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.



Continuous Delivery (CD) is a design practice used in software development to automate and improve the process of software delivery. Techniques such as automated testing, continuous integration and continuous deployment allow software to be developed to a high standard and easily packaged and deployed to test environments, resulting in the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead. The technique was one of the assumptions of extreme programming but at an enterprise level has developed into a discipline of its own, with job descriptions for roles such as "buildmaster" calling for CD skills as mandatory.



Continuous delivery defines a deployment pipeline as a set of validations through which a piece of software must pass on its way to release. Code is compiled if necessary and then packaged by a build server every time a change is committed to a source control repository, then tested by a number of different techniques (possibly including manual testing) before it can be marked as releasable.


Characteristics of a Successful DevOps Team

No matter how you’re using DevOps practices — whether your company has a DevOps department or cross-functional teams that share DevOps tools and practices — there are distinct characteristics of DevOps teams that align with high IT performance.
Here’s a checklist that’s food for thought (and fuel for future improvement!).
These points are drawn from the 2014 State of DevOps Report, and from suggestions of DevOps experts like Paul Duvall, Jez Humble and Joanne Molesky. 

Effective DevOps teams don’t think of issues as “someone else’s problem”. 

Developers, IT operations, quality assurance engineers, database admins, and business analysts collaborate, and everyone checks code into the version control system. Everyone is part of the delivery process — and held accountable for it.

We Automate Build, Deployment, and Testing Phases.

With automation, you reduce the chance of human error as you transition code from one phase to the next. Because you’re automating configuration of all environments, you’re minimizing issues caused by writing code in a development environment that is different from the production environment.

Our Culture Reflects Open Communication and Collaboration.

Developers and IT operations attend planning meetings, standups, and release postmortems. Developers share responsibility for writing testable and deployable code, and if code fails in production, the team is kept in the loop, working together to review causes and identify solutions. 

We Have Routine Deployment Processes and Shared Monitoring Practices.

Team members can accurately report how long it’ll take to deploy a new feature, or even a few lines of code, to production. They can identify and remove roadblocks, without a lot of red tape. They understand the key performance and availability metrics to measure, and track them against larger business goals.

We Implement a Continuous Delivery Pipeline.

Continuous delivery, implemented right, lets you release changes continually to production. That lets you testnew features with real customers, facilitating quick feedback about how they’re being used. Continuous delivery helps companies make better business decisions and move more quickly than their competitors.

We Use Version Control For All Production Artifacts.

Version control systems help you track changes and quickly find the source of an error, reducing time to recovery. Everything required to launch a change into the production environment must be checked into version control, including application code, application and system configurations, tests, and deployment scripts.

We Trust Each Other, and Collectively Enable Continous Improvement.

We deliver on our promises to the business, and to our customers. We continually work on developing collaboration, clear communication and trust between team members. We are continually learning and improving as a team. Most important of all: We spend less time fighting fires and more time focusing on great work.



When it’s well executed, continuous delivery allows an organization to respond more quickly to its market and to customers, both internal and external. It also makes life saner for people in IT operations, software development and quality testing teams. Instead of long periods of development punctuated by looming deadlines, big dramatic releases and panicked remediation of serious bugs, software releases are small, predictable and less dramatic… even boring :-)

Top Benefits of Continuous Delivery

Deliver software with fewer bugs and lower risk.
When you release smaller changes more frequently, you catch errors much earlier in the development process. When you implement automated testing at every stage of development, you don’t pass failed code to the next stage. And it’s easier to roll back smaller changes when you need to.

Release new features to market more frequently — and learn.
Releasing new features early and often — even in a minimally viable state — means you get more frequent feedback, giving you the ability to iterate and learn from your customers. Enlisting customers as development partners gives them a sense of co-ownership and loyalty, and makes them more likely to forgive when you stumble.

Respond to marketing conditions more quickly.
Market conditions change constantly. Whether you’ve just discovered a new product is losing money, or that more customers are visiting your site from smartphones than laptops, it’s much easier to make a fast change if you are already practicing continuous delivery.

Life is saner for everyone: IT operations, software development, QA, product owners and business line owners.
Continuous delivery means the responsibility for software delivery is distributed much more widely, and this shared responsibility and collaboration make life better. Continuous delivery also take a lot of stress out of software release. Releasing smaller changes more often gets everyone used to a regular, predictable pace, leaving room tocome up with ideas and actually enjoy your the work. Best of all, a successful release becomes a shared success, one you can all celebrate together.


In next post, we’ll discuss the most used tools for DevOps and how the infrastructure can be managed “as code”, that means dynamically provisioned creating the needed environment every time you deploy a new version of the code.
Link to the DevOps - Tools and Technology post.

Sources:

Published by 1 comment:
Tags: , , , , , ,
Subscribe to: Posts (Atom)