The Elastic Cloud Project

The Elastic Cloud was a pilot project that I led as a software architect for a customer.

They wanted to evaluate the complexity and the overal quality of a end to end solution provided by Cisco for Cloud Computing.

So we built a complete infrastructure, including hardware and software, for a public cloud.

The number of use case was limited, and I will tell you about the most curious one.

When I was told about the requirement, I thought it didn’t make sense in the real world… but later I understood the rationale  :-)

The request was to deploy a three tier application, with a single click, provisioning also all the needed infrastructure at the same time: (virtual) servers, network and storage.

So this is an example of software defined datacenter, not only SDN but the entire stack: from hardware to the sw application. 

The strange aspect of this requirement is that every tier of the application should run on a different virtualization platform: web servers on KVM, application servers on Hyper-V, database server on ESXi.

It was a technical demonstration, sure, but it also reflected a real world use case. They have some customers that, for legacy applications, have certification constraints that mandate a specific hypervisor for at least one part of the application. So the deployment cannot be standardized on a single platform.

Our customer wanted to verify that Cisco is able to orchestrate a multi vendor infrastructure and that the advantages of stateless computing (I will explain it in a different post, but essentially it is joining SDN with the quality of a hardware infrastructure) are not compromised by the etherogeneity of the virtualization layer.

They said that no other vendor was able to implement the entire use case on the 3 major hypervisors together.

This is a very high level description of the use case:

We decided to implement the self service process in the portal provided by Cisco Prime Service Catalog.

The orchestration layer was Cisco Process Orchestrator, interfacing with Openstack and vmware vCenter directly.

Openstack had compute nodes running KVM and Hyper-V.

Virtual networks were created, on demand, in the virtual switches in each hypervisor and then joined by the physical network connecting the virtualized servers.

The configuration of the network was done through the APIC software controller, that is part of the Cisco ACI architecture (Application Centric Infrastructure).

 

There is a solution, offered by Cisco, that provides cloud services out of the box.

It is called Intelligent Automation for Cloud (IAC) and it is based on the Prime Service Catalog (PSC) as a front end, the Cisco Process Orchestrator (CPO) and prebuilt services created by the Cisco engineering.

IAC integrates your infrastructure by interfacing with vCenter, vCD, Openstack and other so called “element managers” and has all the tools to manage the resources lifecycle.

We decided to reuse some of the existing services as building blocks, wrapping them in a custom process implementing the logic of the specific use case.

So we created a workflow in the visual editor of CPO that invokes the existing atomic services: create a virtual network, create a VDC (virtual data center), create a virtual server.

Then we added explicit calls to the API of the target systems that were not integrated in IAC: the APIC Controller for the ACI fabric, Neutron as the manager of the networking in Openstack (the 4.0 release of IAC only managed Nova for provisioning VMs in Openstack, now also Neutron is integrated in IAC 4.1).

So the effort in this project was only dedicated to understanding the overall logic of the use case and implementing the needed API calls.

It was not particularly difficult, because the target API are all based on a REST interface (both for APIC and Neutron) so invoking them from CPO was a kids’ play.

We created a process with 3 branches, one for each tier of the application, creating all the needed networks and virtual machines, then “plumbing” all together with the ACI fabric through the API of the APIC controller.

We were forerunners, trying to implement a exotic use case before the Business Units looked at it… now everything is available out of the box    ;-)

So we faced the following issues:

• Lack of reference architecture and some products features.
• Dispersed team (time zones and location) made the coordination difficult.
• Fragmented skills: none of us had the complete knowledge of all the products and technologies, due to the amount of innovation involved. 
• Multitasking: many of us were working part time, engaged on different projects.
• Generous support from individuals and organizations in the company, but limited governance
• Products limitations discovered in progress (and solved with... fantasy)
• Usage of beta code and daily builds for some of the products
• Limited documentation available  

In next post I will tell the entire story and how we were able to demonstrate the main concepts:

• Cisco ACI is one of the best solutions for SDN and is not limited to software overlay only
• Cisco has a end to end solution for cloud, including sw and hw and people that can design the architecture
• Open source solutions can be easily integrated into a commercial architecture, providing additional value
• The contribution Cisco provided to Openstack allows customers to manage our network fabric from Neutron seamless

Link to next post: The Elastic Cloud Project - Architecture