Cloud automation refers to the practice of leveraging automation technology to automate cloud operations. Cloud automation should be more aptly named automation of the cloud because it is an evolutionary approach to managing cloud operations rather than just the technology that enables it. By automating their cloud operations, organizations can quickly complete previously manually tasks with significantly reduced errors, such as deploying code, backing up data stores, and rolling out configuration changes to environments.
These tasks done infrequently may be manageable, however, automation shows its advantage by enabling IT teams to architect solutions that leverage the speed and accuracy possible only with machines. For example, major cloud service providers, Google, AWS, Azure, IBM, all deploy their versions of cloud provisioning and orchestration software which makes modern cloud computing, with its rapid flexibility and scalability, possible. While cloud computing was initially manually managed—administrators themselves would need to create and manage virtual machines, make back-ups, etc.—the number of users and companies rooting their IT in the public cloud created the demand and necessity for more efficient workflows. CSPs turned to automation, which helped them power the cloud orchestration and provisioning software that is responsible for coordinating and managing cloud resources with cloud environments that serve a multitude of users.
In DevOps, automation plays a key component in their continuous integration/continuous deployment (CI/CD) workflow. By automating aspects of the building, testing, and deployment activities, the gap between development and operations is bridged, and teams can rapidly respond to user demands with releases and updates to address issues, a process that can handle deployments multiple times a day. In combination with other automations, like orchestration software, entire data centers can be rapidly deployed and updated in the fraction of the time that it could be performed manually, and with reduced chance of errors.
Modern cloud computing is made possible through the abstraction advantages of virtualization. But, cloud virtualization entails a complex set of management practices bottlenecked by manual workflows. The logical solution is cloud automation, which programmatically enables rapid scaling and provisioning, system wide configurations, creation and destruction of VMs, and performance monitoring.
Cloud automation serves business goals as well. The costs of operating in the cloud on third-party platforms has become competitive, and for cloud consumers, it has made migrating to the cloud an effective cost strategy. Many of the top tier providers offer cloud automation that increases productivity, promotes cost savings, and reduces risk of errors. These solutions also address managing technology stacks, which grow taller and more complex with every newly integrated system. Often these integrations are challenged by real-time management concerns, a factor that with manual active management is risky, but a problem that automation is ideally suited to solve.
Automation tools streamline cloud computing workflows, relieving burdens, risks, and costs from IT staff. A successful cloud automation strategy will reduce or eliminate the need for many repetitive tasks; it will push redundant decision making to the system which can make and execute decisions faster than relaying them for a human troubleshooting response; it will allow IT staff to focus on innovations and higher level decision making, and automations can extend other domains in the cloud such as performance monitoring and diagnostics, and security.
Cloud automation is ideal for automating routine operations and sets of workflows, including infrastructure provisioning, identify provisioning and management, application deployment, monitoring and remediation, managing hybrid cloud and multi-cloud configurations, and discovering data and classifying it. Among these workflow sets, automation can unburden manual workflows from routine activities such as:
Cloud automation is a general class of IT practices and technology leveraging automated workflows to complete administrative tasks that maintain cloud operations more efficiently than manually workflows. Cloud automation is used in both cloud provisioning and cloud orchestration. Automation uses scripts, schedules, and automation tools to programmatically perform workflows. Whereas orchestration manages the automated workflows, which in orchestrations are called runbooks composed of steps called activities.
Cloud orchestration integrates several systems, e.g. ticketing systems, or container orchestration systems. In the case of a ticketing system, IT staff can respond to a trouble ticket by sending the appropriate information that solves the problem. The automation then refers to a runbook, a predefined set of automated workflows, and supplies any necessary details included in the ticket, and logs its changes in an integrated change management system. In this scenario, the orchestration is mapped to various runbook workflows, which in turn take variables, and follow dependencies, to automatically execute steps.
Automation of the cloud benefits teams by combining the advantages of the cloud and the advantages of automation.
Cloud automation comes in many varieties. Some general automation tools include:
Automation platforms perform automated functionality as well, and will fall under one or more of the above categories. Platforms may be more encapsulated, for example, while developers may write simple scripts to perform simple tasks, platforms aim to bundle essential tools in one package. Each major cloud provider, Google, Microsoft, AWS, and IBM all offer automation services, but below are other popular platforms integrated into many departmental toolboxes.
Organizations use cloud automation for three general uses: managing their infrastructure with DevOps; scaling their production stacks; and sharing and enforcing best practices. To these ends, several leading brands have implemented automated cloud systems to achieve exceptional levels of scale, all while making it look easily doable.
Netflix, thanks to its cloud scaling innovations, many of which have become de facto best practices, purports to push deployments hundreds of times a day. They do this using an open-source automation software, Spinnaker, to push deployments in a multi-cloud environment. It does such a good job that Google, Microsoft, and even Target are using it.
Like Netflix, Amazon also develops many of their automation innovations in-house. Amazon's IT automation confidence rose even further after they moved their operations to the AWS cloud in 2010, and leveraged cloud advantages into expanding operations to tens of thousands of servers, serving millions of users.
Belying the giant's size, Facebook also utilizes open-source code to automatically deploy and scale its servers; the tool they use is Opscode. As well, the configurations of thousands of servers that serve billions of Facebook users are automated by Chef configuration management software, which is used by Google, CapitalOne, IBM, HP, and many more.
A defining characteristic of DataOps is the numerous roles that interact and contribute to the accumulation, processing, and use of a company's data assets. Towards the extreme, companies whose data assets are their main value proposition have the most immediate need to understand the people engaging with proprietary information. These DataOps roles can be generally classified as data consumers, data preparers, and data suppliers.
Data Suppliers — Data suppliers are the end data owners, like database administrators, responsible for data management, processing, and user access control of a company’s DataOps.
Data Preparers — Due to the ever-complicating nature of DataOps, a middle ground of roles is developing between data engineers, data suppliers, and data consumers. Data engineers build the pipelines that refine raw data into new usable, valuable, and monetizable data. Data curators is a developing role that begins with the needs of consumers to optimize accordingly DataOps content to businesses the needed context for enhancing final assets. Another developing role due to heightened requirements around data governance is the data steward which is responsible for developing company data governance policies and ensuring compliance.
Data Consumers — Data Consumers receive the final data output and the largest group that interacts with DataOps assets. Many roles have emerged: data scientists apply data to solve business problems, data citizens are frontline workers in need of real-time information, and data developers need accurate DataOps as they build business applications that use those pipelines.