Just one button to provision a production-grade Kubernetes cluster

(this is a guest post, authored by my esteemed colleague Fabio Di Niro)

Do you remember?

I bet all of you who are working or playing with Kubernetes remember perfectly the first time you tried to install it.
And the second.
And the third.
...
And the one that finally worked out.

And if you’re a professional you remember also the long path that brought you to own the expertise on Kubernetes that you need to install and fine-tune production grade clusters.
Or, if you’re not a Kubernetes professional, you probably remember how much time it took for you to find someone able to perform a valid Kubernetes install...and how much it costed.

To save all this time and effort to our customers Cisco released the Cisco Container Platform (CCP), a turnkey solution to easily provision production-grade Kubernetes clusters on-prem or in the cloud in minutes, with few mouse clicks and requiring little to no knowledge of K8s.
All the needed integrations with network, storage, computing and security are done automatically by CCP so that the provisioned K8s clusters are ready to run in production.
Clusters provisioned by CCP are already equipped with finely configured monitoring and logging tools like FluentD, Grafana, ElasticSearch, Kibana.
Through the Container Network Interface (CNI) you can choose whether to leverage Cisco ACI as network infrastructure or Calico (no dependence on the underlying infrastructure).

This is already great, but I thought to create a demo that may push the simplicity of those “few mouse clicks” to its limit, making possible to create a production grade cluster in just one click.

Introducing the Kubernetes dash button.

The concept is fairly simple: build a dash button that, once pressed, creates a production grade Kubernetes cluster ready to use.

Leveraging the rich set of the Cisco Container Platform (CCP) APIs this is even too easy, so I thought to add some more feature on top:

- I wanted to provision the cluster and access it just through the dash button. So, I want CCP to display on the dash button itself the IP address of the master node of the cluster created
- The start and finish of the cluster provisioning process had to be confirmed, so the communications had to be bi-directional with the dash button
- I wanted a fair battery life that would avoid me to recharge the button every day, so I needed to have electronics able to sleep or hibernate
- My lab, where I have the infrastructure and the CCP, is behind a proxy, so I can’t listen for calls inside the lab, I can just initiate communications from the lab. So, I needed a way to change the “push” of the button in a “pull” of the button press information
- I wanted to use the button everywhere I go without worrying about the local Wi-Fi settings

How it works

To satisfy all the above requirements I added a couple of elements in the picture, ending up with the following architecture:

The button is based on an Arduino ESP 32 board, it connects via Wi-Fi to my smartphone and uses its internet connection, this way I can use the button everywhere my phone has data signal. The button leverages a publish-subscribe message service (MQTT) in the cloud to bypass the limitation of the proxy I have behind my lab and reach a couple of scripts that calls the right API in the Cisco Container Platform to trigger the provisioning of a shiny new Kubernetes cluster.
Once the cluster is provisioned the IP address of the master node is returned to the dash button that shows it on its display, at this point it is ready to accept connection and be used.

A 3D printed enclosure completed my project, I took an existing model but then I decided to  leverage the capabilities of CCP to deploy K8s clusters on-prem or in the cloud so I designed the two different enclosures you can see in the picture to have two different dash buttons for the two different deployment target.
All the code and 3D designs have been released and are publicly available at: https://github.com/fdiniro/CCPDashButton

Now, before doing my demo, I can ask to my customers: “How much time and effort takes you to install a production-grade, fully operationalized and secured kubernetes cluster?” and whatever answer I get I know I can answer “I can do it in 2 minutes blindfolded and cuffed”.

You can see the recorded demo here: https://youtu.be/-F-xR0XNPBs

A new community for Cisco Multicloud software users

Today we are launching a new project, that is a local community of people interested in the Cisco software solution for multicloud.

Before you go on reading, this is our next meeting:
Rome and Vimercate, October 23, 2019 - Cisco offices (details below)

Like many open source communities (e.g. meetups on various technologies) our goal is to spread information, share experience and offer access to experts to discuss your own use cases. In our opinion, this could be beneficial for customers, partners and people that are just curious about multicloud and the open source technologies. Of course, the expected result is also to facilitate sales of Cisco technology, products and projects.

That is the value for Cisco, but what's in it for you? 

We think that joining this community (meetings will be in Italian) you could learn the solutions offered by Cisco for multicloud in an informal context (and for free), understand what use cases you can implement and how, how Cisco technologies integrates with the open source technologies like Kubernetes, Docker and others, how to adopt DevOps and, why not, learn the open source stuff regardless the integration with Cisco products. We will also offer hands on labs and activities that matches learning and fun (e.g. teaching Alexa to deploy a database, or create a Kubernetes cluster).
In addition, you get some pizza and beer for free :-)

At the end of the day, it's an easy learning opportunity, a (bidirectional) share of experience, a stage where you can show - if you like - your knowledge and share the experience you've done in your project. Offering help to others or receiving support from peers and from the Cisco experts.

We are starting an experiment

We are starting with a few meetings planned, see the agenda below.
We thought that scheduling outside the office hours will make it easier for you to join, avoiding conflicts and positioning this community as a place where you go for fun. Or, at least, with no relation with your role in your company. As an individual you learn subjects that make your work easier and your resume more interesting, so maybe this is worth dedicating 2-3 hours in your spare time (e.g. every second Thursday of each month, from 5 to 8 pm, pizza included).
We will ask your feedback about the schedule, so that we move to a different time of the day or cadence according to your preference.
And, of course, about the subjects we're going to address in next meetings so that we stay relevant for you.

An additional value is introducing you with the official Cisco DevNet community, that is our developers community.
We are going to leverage a lot of amazing material from DevNet, including documentation, tutorials and sandboxes where you can experiment with no need to install the products and no fear of destroying the environment: it's there for your enjoyment and will be reset after you use it.

Every event will be split in 2-3 sessions:

• one based on presentation/demo of a Cisco product, 
• one based on a subject from the open source world that is not necessarily related to Cisco. Sometimes the integration of open source and Cisco API will be demonstrated.

We will keep each session short and crispy and every event will have a hands-on activity to keep you awake.
When possible we'll add a lab activity that participants can do directly from their laptop.

Next topics

This is a temporary, draft list of the subjects we could offer in the first series of meetings: we can prioritize them based on internal feedback or through a survey with the attendees of first meetings (or sent remotely to the community).

• Amazon Alexa integration
• Devops - CI/CD with CloudCenter - (VM)
• Devops - CI/CD with CloudCenter - (containers)
• The ACI CNI (ltechnical intro)
• The ACI CNI (use cases, operational model)
• Multcloud cost control
• Serverless lab
• Devnet sandboxes
• Devnet Express - programmability
• Meet The Engineer or design clinic - bring your on use cases
• ACI and Terraform
• Managing k8s clusters in cloud and on prem
• Automating the Software-Defined WAN

The next event we have planned is at the Cisco offices in Rome and Vimercate, on October 23, 2019.

Address: 

• Roma - via del Serafico 200 (registration required)
• Vimercate - via Torri Bianche 8 (registration required)
• Padova - via S. Marco 9 (registration required)

Time: 

• from 5 pm to 8 pm (pizza and beer included)

This is the proposed agenda (no technical requirements to attend):

- Why this meetup (10')
- Container track: container 101 (theory, use cases, application architectures)
- Cost control with CloudCenter (30')
- Pizza & beer
- DevOps: Testing methodology (30')

Registration required

To register for the event, please click here.

See you soon!

References

Cisco Multicloud (Italian) - https://www.cisco.com/c/it_it/solutions/cloud/multicloud-portfolio.html 

Cisco Multicloud - https://www.cisco.com/c/en_uk/solutions/cloud/multicloud-portfolio.html

Cisco DevNet - https://developer.cisco.com 

Creating a complete CI/CD environment in few minutes

In a previous post we introduced CI/CD as a Service (CD/CDaaS) and talked about how it can transform and increase the efficiency of your DevOps practice, to serve development teams and Line Of Business (LoB) better.

In this post we will show how to effectively create a CI/CDaaS with Cisco CloudCenter, either for an organization’s internal use or to offer it to your customers, in case you’re an IT service provider.

With our CI/CDaaS we are able to:

1 - Automate the creation of a complete environment for a development project, including:

• a source code repository (we chose SVN, but it could be GitLab or any other), 

• an integration server / orchestrator (Jenkins),

• an artifact repository for the builds (e.g. JFrog Artifactory or Nexus, but it could be any solution including a simple web server or an FTP server)

• a number of workstations (one per developer) ready to be used with all the needed tools: Eclipse, database tools, etc.

• a remote desktop access to the workstations, even if the developers don't have access to the network where the project environment is created

2 - Integrate all the above tools end-to-end, creating a complete chain for CI/CD with no human intervention required.

3 - Get every build - generated automatically after any commit in the source code repository - deployed automatically in one of your targets, selected according to the project phase, e.g. development in an on-premises VMware environment, integration test in on-premises OpenStack, and production in AWS public cloud.

4 - Get acceptance tests - required to promote a deployment to the next phase/environment - executed automatically after each deployment in one of the environments.

Everything we describe is implemented using Cisco CloudCenter as the main orchestration engine, in a multicloud context, meaning you can select any private or public cloud as a target for the deployment).

The next picture represents a project leader ordering the deployment of the CI/CD toolset for two distinct projects, possibly in different clouds. 

Creating a complete chain for CI/CD with no human intervention, for two different tenants, with CloudCenter

 

Step 1: Self-service in the CloudCenter catalog

Accessing the self-service catalog in CloudCenter, the user can order the deployment of a single application or a complex set of components, like our CI/CDaaS service (we call these Application Profiles):

Application Profiles in the self-service catalog

As you can see from the image above, we have created 2 Application Profiles for this demo. The use case is having a development team that releases daily builds of a web application (PetClinic), so we have built:

1. A cloud-agnostic Application Profile to deploy PetClinic, that takes the binary files required for the deployment from a common repository (that will be the Artifactory server created by our CI/CD service described below). The name of this Application Profile (that is a service in the CloudCenter catalog) is “Clinic”. You can select one specific version of the application binaries that you want to deploy from the Artifactory repository (by default you use the last build available).
   
   Please note that we will not focus on the way a generic Application Profile is built and deployed, because it’s covered in the CloudCenter documentation, but PetClinic is functional in our example as developers will generate new builds that need to be deployed.

2. A cloud-agnostic Application Profile to deploy the CI/CD toolset including SVN, Jenkins and Artifactory, installed in their own VM. The name of this service in the CloudCenter catalog is “AdvancedDevOpsEnv”; it creates an operating project environment where the “Clinic” application can be deployed automatically at every new commit made by a developer.
   
   This is the implementation of our CI/CDaaS concept.

 

Step 2: Deploying the CI/CD toolset

When you “order” the AdvancedDevOpsEnv service you have to set the name of the new project and some options for the deployment (which cloud target, the amount of resources to be allocated when VM or containers are created, etc.); the next picture shows the order wizard in CloudCenter, where you make these selections.

CloudCenter order wizard: you can choose where to deploy the CI/CD toolset and the size of each VM

You can monitor the progress of the deployment while the job runs (it takes approximately 15 minutes to complete) as you can see in the image below:

Watching the progress in CloudCenter: last deployment state is "Deployed"

Once the deployment is complete, you can see the details of the new VM that has been created by clicking on the server representation on the left (e.g. Jenkins – see next picture) and expanding the list of running nodes on the right.

Please Note:

• The IP address of the VM is exposed by CloudCenter.
• You can access the VM via SSH or RDP directly from the browser (that is very important because connectivity and security are proxied by CloudCenter, so you don't really need any access to the actual network where the VM is deployed).

Accessing the details of the VM and the log of actions executed by the agent

Step 3: verifying the configuration made by CloudCenter

The Jenkins orchestrator is automatically setup with a connection to the SVN source code repository (see the Jenkins job “repo1”, that is the name provided by the user for the repository when he ordered the deployment) and with Cloud Center (see the Jenkins job “deploy”).

The Jenkins orchestrator and the 2 tasks created by CloudCenter

The only manual configuration we need is to set the login and password used by CloudCenter to access the SVN repository (we were not able to automate this configuration): you just need to click on the Jenkins task repo1 (next picture), then click Configure and enter your credentials (in our example, user001/C1sco123). At this point, Jenkins is fully configured and ready to go.

Jenkins: setting credentials to access the SVN repository

The jFrog Artifactory is also ready to receive binaries from Jenkins, at each new build of the application, because CloudCenter sets the required information when configuring Jenkins and Artifactory. Note that the Artifactory repository name (repo1) is the name used for the SVN repository, provided by the user when he ordered the deployment.

The Artifactory repository for binaries (it will host all the builds generated by Jenkins)

Finally, we configure a new Repository in CloudCenter (there might be many), that needs to point at our new Artifactory repository to deploy new builds of your application automatically as they are released.

Cloud Center: setting up a repository for the artifacts that will be deployed

Here we set the IP address of the Artifactory server:

Setting up the repository on Cloud Center

We have now completed the setup of a CI/CD environment for a new project, just consuming a reusable service that creates it in 15 minutes (CI/CDaaS).

In the next post will show how to use it, following the lifecycle of the PetClinic application from editing the source code to automated deployment.

Credits 

This post is co-authored with a colleague of mine, Stefano Gioia.

A simpler framework for hybrid cloud

Hybrid cloud is one of top mind projects for most IT managers, and there's little content that one can add to be original   ;-)

The hype and the attempt of many vendors (including... Cisco) to provide relevant solutions have populated the space of an incredible number of offers that make it hard to distinguish what works, what's manageable and cost effective, from what is only marketecture.

Recently Cisco decided to invest even more on cloud and, with the advent of a new CTO and some acquisitions, a revision of our approach to hybrid cloud made it easier and more effective. This post is not from official marketing and is not echoing company's direction: it's my attempt to rationalize my understanding of the new framework and to solicit your comments and feedback, so that I can leverage it when I discuss with my customers and partners.
The following picture represents the area where Cisco plays a role, offering hardware and software solutions.
When it comes to the software stack to manage the infrastructure and provide services to the users, we have a mix of Cisco products, open source solutions and integration with 3rd parties. The objective is to offer a set of pre-validated stacks that can match the different needs, granting a deterministic result.

I shared some thoughts with a group of colleagues because we're planning educational activities for our field people: instead of just providing a reference architecture (that would end being a list of products to be forced in every deal) we tried to represent the functions in the system as components of a framework, from which we'll pull the specific architecture for a given project. This, used cum grano salis, should help to be pragmatic and realize quick wins (for both the customers - think of Fast IT initiatives - and of course for Cisco).

As a result, next picture is separating the different functional layers so that they can be explained to sales guys and to customers.
It could also help to manage the possible overlap with alternative solutions that customers may choose – or already have – because every element is replaceable in the picture, based on the open API they expose/consume (as well as any well designed 3rd party product).

It is important to note that the top two layers in the picture are optional, since not all customers need those functions in their system. Based on the level of Governance that they want to have, the existing processes and the way they develop business applications (or use commercial software that only need a resource pool to be deployed), the entry point could be directly at the third layer (Multi-Cloud Management) and ITSM and PaaS would be removed.

So, while we explain all the possibilities as said above, we need to make them feel confident that it’s doable and not overly complex.
In that regard, my motto is that “cloud is not a product (or a set of), it’s a project and it’s complex in nature… regardless the products set you choose”. Generally the cost of hardware and software products is lower than development and consulting services, and customers know it.
If we can claim that a pre-built integration makes the project easier (and we can), I would stress the value of reducing the project risk and delivering outcomes faster rather than a cheaper implementation.

Selling licenses can be (almost) easy, but driving adoption with business outcomes for customers is different. Finally Cisco has built a practice that can deliver IT projects effectively and recruited partners that do the same: customers have different options to choose from.

Now, in the context of a end to end strategy defined with the customer, we can deliver projects based on agile methodologies (e.g. Scrum) and implement the architecture layers with a bottom up approach: from a strong capability to automate the Data Center (and the hybrid cloud) you can create services that are surfaced up to the consumption layers, including a self service catalog.

Software Defined What?

The bottom up approach stresses the value of the API exposed by UCS and ACI (with the further evolution from basic programmability to policy-based management, that I'm not mentioning yet - look out for next post). With the power and the granularity of those API, you can really realize a fully Software Defined Data Center (SDDC): servers and networks can be shaped via software interfaces.
By the way, I take the opportunity here to clarify that Software Defined Data Center does not mean Software Implemented Data Center: you don't necessarily need a software overlay that mimics the behavior of the hardware (living as a separate entity), you need software controllers that drive the shape and the behavior of both physical and virtual resources in the DC as a single system.
Better if they do that based on policies... like the Cisco architecture does  :-)
You will see a post dedicated to policies and application intent soon on this blog.

Competition?

We also recognize that many customers have already an ITSM solution in place, or any other form of governance. So we don't engage in competitive fights, like imposing Cisco Prime Service Catalog vs Service Now, but we rather integrate our solution with the existing components: this is a sort of compromise with a competitor that hurts my pride, but since it's for our customers' benefit... it's a good solution.

Cisco Cloud Center as a broker: the recent acquisition of Cliqr brings a great solution to Cisco to address the multi-cloud management use cases, the most important ones for the majority of customers. In the logical schema above you can see that the hybrid cloud scenario has been qualified better as Multi-Cloud management.
This reflects the fact that having a application deployed partly in your Data Center and partly in the public cloud is still a relevant use case, but many companies are more attracted by other scenarios... like moving from one project stage to next (e.g. Dev-Test-QA-Prod) using different resource pools (on premise or in cloud), or moving their assets from one cloud provider to a different one.

Cloud Brokering and Multi Cloud Management

In the first one (promotion to next stage) it could be useful to leverage resources that are allocated based on business convenience (e.g. cost or flexibility) or compliance (e.g. data sovereignty), so the application and all the needed infrastructure are moved back and forth to the public cloud.
In the second the driver could be a dual provider strategy, or maybe a change in the market conditions that makes one provider more appealing than the current one, or a strategic switch from private cloud to public (or vice versa).


In all these cases, we offer a solution to deploy a software stack (a complete custom application, a development platform, or a commercial software product) as a self service option, where the target can be selected dynamically from a list of available clouds.
You can deploy to your local private cloud, based on vmware or any other virtualization solution, or to a Openstack based cloud, or to any of the public cloud providers if you have an account there.
Any resource pool is a possible destination for the deployment (and the life cycle management, including autoscale or retirement of the application).
The model of the deployment of the application is completely de-coupled from the selection of the target, thanks to the capabilities of the orchestrator that can configure the needed resources in almost any cloud transparently.
It uses the API exposed by the element managers of a multi vendor infrastructure on premise (e.g. vcenter, UCS Manager, the ACI controller, etc.) and those exposed by public clouds like AWS, Azure, etc.

From a logical schema to a real deployment

So we can offer users a different entry point, based on their business needs (they might need a ticketing system, or a self service catalog, a PaaS solution or directly the web portal of the multi cloud manager to model deployments and deliver them).
The customer can have one or more resource pools, allocated wherever he likes (local or in cloud), and let the broker direct the selection of the target based on predefined policies.

The schema in next picture presents different products at every layer: a solution can be based on one of them, or a combination. We have the flexibility to match the specific needs with products from Cisco, from 3rd party vendors or open source.
As an example, MANTL is a new open source project that makes the development of microservices easier if you build cloud native applications.

I will expand the detail of the single products and the open source solutions shown in this picture in my next post.
Stay tuned...

References

http://www.cisco.com/c/en/us/solutions/executive-perspectives/fast_it.html
http://www.cisco.com/web/solutions/trends/futureofit/why-cisco.html
http://MANTL.io
http://Github.com/CiscoCloud/microservices-infrastucture 
http://lucarelandini.blogspot.it/2015/10/devops-docker-and-cisco-aci-part-1.html
http://lucarelandini.blogspot.it/2015/03/aci-for-dummies.html
http://lucarelandini.blogspot.it/2015/09/the-phoenix-project-how-devops-can.html