Cloud Automation with Configuration Management using CHEF Tool

Cloud Automation with Configuration Management using CHEF Tool

Ramandeep Singh, Dr. Ravindra Kumar Purwar

USICT, GGSIPU

Abstract:- Automated, efficient software deployment is essential for todays modern cloud hosting providers. With advances in cloud technology, on demand cloud services offered by public providers are becoming gradually powerful, anchoring the ecosystem of cloud services. Moreover, the DevOps teams are in much bigger focus now since they are responsible for the automation and provisioning of the whole environment along with the client application. This paper focuses upon the automation of customer application right from environment provisioning to application deployment.

According to a white paper by Vision Solutions, 59% of Fortune 500 companies experienced a minimum of 1.6 hours of downtime per week. This means that for a company who has 10,000 employees who on average make a salary of $30 per hour, or $60,000 per year, this downtime can potentially create a loss of $480,000 weekly or nearly 25 million dollars annually, not including the cost of benefits, loss of sales, or negative impact to the reputation of the provider from services being unavailable. Therefore, it is of the utmost importance for a companys servers to have their services installed, configured, and running as quickly as possible and as consistent as possible to help reduce costs. This translates into automated deployment and configuration.

Keywords:- Cloud computing, Cloud Service Models, Cloud delivery models, Configuration Management, automation, Chef, Puppet, Ansible.

1. INTRODUCTION (Cloud and Configuration Management)
   1. 1. Overview

         DevOps is a blend of two terms development and operations. It is considered to be the most effective way to foster collaboration and eliminate the walls of confusion that exist between software developers and operations teams, by way of shared experiences and suggested solutions [2].

         DevOps has been identified as a phenomenon whereby stakeholders of a software development team work together to deliver software continuously, allowing the business to seize emerging and existing market opportunities while reducing the

         time required for inclusion of client feedback. However, little has been done to develop adoption strategies/frameworks for this phenomenon. The absence of such strategies may result in DevOps not being adequately communicated and its impact not fully comprehended in both the practitioner and academic research communities. This study investigates the factors that are hindering the adoption of DevOps and proposes strategies to address them using both literature study and interviews with practitioners actively involved in the DevOps movement[2]. A proposed

         conceptual framework has been developed both to strategize the adoption of DevOps, and to contribute input for future research.

         DevOps is one of the most popular approaches for software delivery nowadays. Even though there is no unified definition of DevOps, there is wide consensus about the set of practices that are part of it[1]. Two of those practices are Infrastructure as Code and Continuous Delivery, which bring new artifacts into the Software Development lifecycle. These new artifacts have direct impact on Software Configuration Management, which is a well-known practice that has been around for decades in the Software Engineering discipline. In particular, these new practices have a direct impact on Version Control. This article describes a Version Control Strategy to manage these new artifacts introduced by DevOps initiatives. The proposed strategy covers the identification of artifacts, versioning tools,version naming conventions and traceability between different artifacts versions. The strategy was validated in three cases of real world projects where it was successfully applied. Each case corresponds to a different kind of organization and in each case a different set of tools where used. Based on these cases, benefits and possible improvements are identified.

         Cloud computing is a model for enabling global, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction [7]. This cloud model is composed of five essential characteristics, three service models, and four deployment models.

      2. What is Cloud Computing?

         Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction [8]. This cloud model is composed of five essential characteristics, three service models, and four deployment models.

         For a service to be considered a cloud service, it must have the following Essential features:

         On-demand self-service Broad network access Resource pooling

         Rapid elasticity and Measured service

      3. Introduction to Cloud Service Models

         Fig 1: Types of Cloud Services Models [7]

      4. Types of Cloud/ Introduction to cloud delivery models Cloud delivery models refer to how a cloud solution is used by an organization, where the data is located, and who operates the cloud solution [2]. Cloud computing supports multiple delivery models that can deliver the capabilities needed in a cloud solution.

         The cloud delivery models are as follows:

         • Public cloud
         • Private cloud
         • Hybrid cloud
         • Community cloud

   Table 1: Types of Cloud Delivery Models [8] (1.4.1) Public clouds

   A public cloud is one in which the cloud infrastructure is made available to the general public or a large industry group over the Internet. The infrastructure is not owned by the user, but by an organization that provides cloud services. Services can be provided either at no cost, as a subscription, or as a pay-as-you-go model.

   Examples of public clouds include IBM SoftLayer, Amazon Elastic Compute Cloud (EC2), Google AppEngine, and Microsoft Azure App Service [8].

   (1.4.2) Private clouds

   A private cloud refers to a cloud solution where the infrastructure is provisioned for the limited use of a single organization. The organization often acts as a cloud service provider to internal business units that obtain all the benefits of a cloud without having to provision their own infrastructure. By consolidating and centralizing services into a cloud, the organization benefits from centralized service management and economies of scale.

   A private cloud provides an organization with some advantages over a public cloud. The organization gains greater control over the resources that make up the cloud. In

   addition, private clouds are ideal when the type of work being done is not practical for a public cloud because of network latency, security, or regulatory concerns.

   A private cloud can be owned, managed, and operated by the organization, a third party, or a combination. The private cloud infrastructure is usually provisioned on the organizations premises, but it can also be hosted in a data centre that is owned by a third party.

   (1.4.3) Community clouds

   A community cloud shares the cloud infrastructure across several organizations in support of a specific community that has common concerns (for example, mission, security requirements, policy,and compliance considerations). The primary goal of a community cloud is to have participating organizations realize the benefits of a public cloud, such as shared infrastructure costs and a pay-as-you-go billing structure, with the added level of privacy, security, and policy compliance that is usually associated with a private cloud [8].

   (1.4.4) Hybrid clouds

   A hybrid cloud is a combination of various cloud types (public, private, and community). Each cloud in the hybrid mix remains a unique entity, but is bound to the mix by technology that enables data and application portability. The hybrid approach allows a business to take advantage of the scalability and cost-effectiveness of off-premise third- party resources without revealing applications and data beyond the corporate intranet [8]. A well-constructed hybrid cloud can service secure, mission-critical processes, such as receiving customer payments (a private cloud service), and secondary processes such as employee payroll processing (a public cloud service).

   The challenge for a hybrid cloud is the difficulty in effectively creating and governing such a solution. Services from various sources must be obtained and provisioned as though they originated from a single location, and interactions between on-premises and off-premise components make the employment even more complicated.

   Table 2: Few Cloud Providers[9]

2. CLOUD AND CONFIGURATION MANAGEMENT
   1. How cloud is related to Configuration Management? Cloud and cloud services have become more popular in recent years. In the cloud hosting industry, companies have found that costs can be reduced by improving up-time in servers and creating a scalable server based on load.

      According to a white paper by Vision Solutions, 59% of Fortune 500 companies experienced a minimum of 1.6 hours of downtime per week . This means that for a company who has 10,000 employees who on average make a salary of $30 per hour , or $60,000 per year, this downtime can potentially create a loss of $480,000 weekly or nearly 25 million dollars annually, not including the cost of benefits, loss of sales, or negative impact to the reputation of the provider from services being unavailable. Consequently, it is of the utmost importance for a companys servers to have their services installed, configured, and running as quickly as possible and as consistent as possible to help reduce costs. This translates into automated deployment and configuration.

   2. What is Configuration Management?

      Configuration management (CM) is a systems engineering process for establishing and maintaining consistency of a product’s performance, efficient, and physical attributes with its requirements, design, and operational information throughout its life [2].

      Configuration management can be used to maintain OS configuration files.

      Various tools for Configuration Management include Ansible, Bcfg2, CFEngine, Chef, Otter, Puppet, SaltStack etc [2].

3. AUTOMATION TOOLS
   1. 1. Chef

         This is a system and cloud infrastructure automation tool for installing applications and software to bare metal, virtual machine, and container-based clouds [1]. The configuration is written in Ruby DSL, and it uses the concepts of organizations, environments, cookbooks, recipes and resources, all driven by supplied or derived attributes. The tool has a set of control parts that work together to provide its functionality. Chef Workstation is used to control the deployment of configurations from the Chef Server to Chef managed nodes. Nodes are bootstrapped with agents and pull configurations from the server. The core is developed in Erlang and is designed to provide scale to tens of thousands of servers [1]. It is developed around an infrastructure-as-a- code model with version control integral to the workstation. Directives run top to bottom, and the cookbooks, does not matter how many times are running, generate the same result.

         Fig 3: Architecture of Chef [1]

         There are three main components:

         • Chef Server is the most important part because it stores the whole configuration data

           for all nodes. It has also the role to administrate the access rights.

         • Chef Workstation is the place where the cookbooks, recipes and all of the configuration parts are created (to be deployed to Chef Nodes through Chef Server).
         • Chef Nodes are the places were the cookbooks, recipes and all of the configuration parts are stored.

           Advantages

         • Large community of cookbooks and development tools.
         • Ability to handle physical, virtual, and containers deployments.
         • Provides hosted services.
         • Excellent at managing operating system.

           Disadvantages

         • Complex to set up the entire stuff because it requires good understanding of Ruby.
         • Huge amount of documentation.
         • It requires an agent to be installed and pulled configuration
4. IMPLEMENTATION
   1. 1. Creating/Copying a file on multiple servers

         A number of times, we have a requirement to copy the exact same file on hundreds of servers. Its practical to create a file on a couple of servers but creating a file with exact same content on hundreds of servers is cumbersome and waste of manpower. Hence, with tools like CHEF, we can create a file using the defined resources and have the file deployed on any number of servers.

         In the figures below, we create a file by writing a small script in the cookbook. Also, for user creation, use the resource user to add users, update existing users, remove users, and to lock/unlock user passwords. Moreover, Use the package resource to manage packages. This resource is the base resource for several other resources used for package management on specific platforms. While it is possible to use each of these specific resources, it is recommended to use the package resource as often as possible. All we need to do is to upload the cookbook to the server which is linked to the clients (on which configuration changes have to be deployed).

         Fig 4: Creating and uploading a cookbook for file creation

         Executing a cookbook on client and Display the data of file

         Fig 5: Executing a cookbook on client for file creation

      2. Creating a User on multiple servers

         Fig 6: Creating and uploading a cookbook for user creation

         Fig 7: Executing a cookbook on client for user configuration

      3. Installation of software packages on multiple servers All of us have installed a number of software packages in our daily life. In organizational servers, there are a few packages which have to be present/install as basic packages on almost all the servers. Such packages if installed individually on each server would take a lot of time. Hence, configuration tools like CHEF reduces the time and effort to a considerable amount.

         Lets install nmap package which is used for port scanning purpose.

         Fig 8: Installing a software package-nmap

         Fig 9: Executing a cookbook to install nmap package

         The package nmap is installed as we can see above. By using the mentioned command, we can use nmap for port scanning.

         nmap -v 192.168.127.129 -p 22, where -v means verbose and -p means port to be scanned.

         Nmap can also provide further information on targets, including reverse DNS names, operating system guesses, device types, and MAC addresses.

         Hence, this is the beauty of automation and configuration tools like CHEF, they make our daily tasks much more easier and less time consuming, reduces the manpower and cost also.

      4. Creating and configuring a web server/web page

         Chef can be used to install/configure a web server/web page also.You can host your own website using this.In this we are configuring Apache web server. The primary Apache configurationfile is /etc/httpd/conf/httpd.conf.

         Fig 10: Installing/configure apche web Server

         Create index.html file

         The index.html file is the default file a web server will serve up when you access the website. In the /var/www/html directory, create a file with the name index.html. Add the content. PFB thindex.html

         Fig 11:Executing a cookbook to install/configure apache

         Once mentioned below pre-requisites done, then check apache service:

         • Package httpd should be installed
         • Index.html should be there in /var/www/html
         • Apache service should be running

   Fig 12: Checking apache service status

   As apache is running we can see above, so lastly open any browser, and hit the servers ip then you can or able to see your own customizable website/webpage.You can customize your index.html as per your requirement and then can automate this process by using CHEF tool.

   This is how you can create your own customizable website/webpage easily.

   Fig 13: Customizable website/webpage using CHEF TOOL

5. CONCLUSION

   The manual installation, updation and configuration of software packages requires a lot of manpower and time. We have compared the manual installation with automation tools in order to provide a accurate cloud deployment. The solution can be easily scaled by cloning the cookbook/ playbook or module for the Compute node and by increment assigning of IP addresses. The automation tools decrease the time required to complete the tasks mentioned from a few hours to few minutes.

   Fig 14: Performance Evaluation of Automation tools v/s doing it manually [1]

6. ACKNOWLEDGEMENT

   I would like to express my special thanks of gratitude to my mentor Dr. Ravindra Kumar Purwar, Associate Professor of USICT, who gave me an opportunity to do a research and review on the topic Cloud Automation with Configuration Management, which helped me in doing my work and prepared myself with the vast knowledge of the related area. I am thankful to him.

7. REFERENCES

1. Nicolás Paez, Versioning Strategy for DevOps Implementations, Department of Science and Technology Universidad Nacional de Tres de Febrero, IEEE 2018, pp. 1-6.
2. Morgan B. Kamuto, Josef J. Langerman, Factors Inhibiting the Adoption of DevOps in Large Organisations: South African Context, 2017 2nd IEEE International Conference On Recent Trends InElectronics Information & Communication Technology, pp. 48-51
3. Eduard Luchian, Cosmin Filip, Andrei Bogdan Rus, Iustin- Alexandru Ivanciu, Virgil Dobrota, Automation of the Infrastructure and Services for an OpenStack Deployment Using Chef Tool, IEEE International Conference on Cloud Engineering IC2E, 2017, pp. 295-302.
4. James O. Benson, John J. Prevost, and Paul Rad, Survey of Automated Software Deployment for Computational and Engineering Research, IEEE Transactions, 2016.
5. Dmitry Duplyakin and Robert Ricci, Introducing Configuration Management Capabilities into CloudLab Experiments, IEEE INFOCOM International Workshop on Computer and Networking Experimental Research Using Testbeds, 2016, pp. 453-458
6. M. Boschetti and P. Ruiu, A Cloud automation platform for flexibility in applications and resources provisioning, 9th International Conference on Complex, Intelligent, and Software Intensive Systems, 2015, pp. 204-208
7. Gregory Katsaros, Alexander Lenk, Michael Menzel, Jannis Rake, Ryan Skipp, Jacob Eberhardt, Cloud application portability with TOSCA, Chef and Openstack, IEEE International Conference on Cloud Engineering, 2014, pp. 295-302.
8. Nishant Kumar Singh, Sanjeev Thakur, Himanshu Chaurasiya and Himanshu Nagdev, Automated Provisioning of Application in IAAS Cloud using Ansible Conguration Management,1st International Conference on Next Generation Computing Technologies (NGCT-2015) Dehradun, India, 4-5 September 2015, pp. 81-85.
9. Johannes Hintsch, Carsten G¨orling, and Klaus Turowski, Modularization of Software as a Service Products: A Case Study of the Conguration Management Tool Puppet, Third International Conference on Enterprise Systems, 2015, pp. 184-191
10. http://www.globaldots.com/cloud-computing-types-of-cloud/
11. http://www.redbooks.ibm.com/redpapers/pdfs/redp4873.pdf
12. https://CiscoDevNet/devnet-1008-private-public-or-hybrid-cloud- which-cloud-should-i-choose