CentOS 7 – Open source configuration management tools

How to create virtual machines with VirtualBox

In this section, we are going to enumerate the world’s most famous open source configuration management tools, going through the description of each tool, the major focus, and where it can serve well.

Chef

Chef is a configuration management tool that helps us configure and maintain servers that provide a variety of services. Chef uses something called recipes, made to describe the desired management (configuration, installation, change) for the server’s applications and utilities. It is widely used for infrastructure with cloud-computing-based platforms (such as Rackspace, Internap, Amazon, Google, OpenStack, SoftLayer, and Microsoft) because of its capacity to automatically provision and configure new machines on the go. The scalability of Chef allows it to be used for small and large infrastructures on demand.

Chef can be executed as a server or a client. The clients are executed on the servers/machines that we want to manage using our server. And the Chef server agent is installed in one centralized server, on which it manages the recipes depending on what the administrator has defined for each node (web server, cluster machine, storage server, and so on). Then the client executes the recipes sent to it by the server. By default, the client updates its recipe from the server every 30 minutes to see whether there is any change that needs to be done.

At the beginning of the project, Chef used to manage only Linux machines, but now it is capable of managing Windows machines as well. The Chef server is now supported by multiple Linux platforms (RHEL/CentOS, FreeBSD, OSX, Solaris, and so on).

This is what it looks like, architecture-wise, after having Chef installed. The first images shows the interaction between chef and a variety of nodes that can manage.

Source: http://www.yet.org

The following screenshot show the interaction between chef and the Amazon web services to be on use for advanced configuration

Source: https://www.chef.io/

Note

For more detailed information on how to set up and master Chef, please have a look at some of Packt’s other books on Chef at: https://www.packtpub.com/all/?search=chef

SaltStack (Salt)

SaltStack is a Python-based open source configuration management tool. It is also called a remote engine. In a way, it is a new approach to infrastructure management. It is easy to run, scalable, can handle thousands of nodes, and can establish very fast communication between its nodes (servers, terminals, and so on).

It is well known due to its dynamic communication inside the infrastructures it mages, where it benefit amazingly from that to organize, execute, and establish configuration management through its nodes, as well as much more.

It is available for a variety of operating systems (CentOS, Debian, RHEL, Ubuntu, and even Windows).

Salt is designed to be very extensible, wherein it has the capacity to use module management so as to be moldable to a variety of applications. Salt offers dynamic modules to manage the remote execution and administration of its components. They are categorized as follows: execution modules, state modules, grains, renderer modules, returners, and runners.

SaltStack has a paid product called SaltStack Enterprise. It is designed for big infrastructures in IT organizations, system administration, site reliability engineering, and development configuration and management for modern data center infrastructures, applications, and code. It is briefly depicted in the following figure, which shows how it covers a variety of fields to help the establishment, management, and maintenance of the following services shown in the two following figures:

Source: http://www.saltstack.com

This figure shows the interaction between the three major services (ITOps, DevOps, CloudOps) and the capacity to use SaltStack to manage those all together.

Source: http://www.saltstack.com

SaltStack services operate on several steps as shown in the following figure:

Source: http://saltstack.com

Note

For more detailed information on how to set up and master SaltStack, please have a look at some of Packt’s other books on SaltStack at: https://www.packtpub.com/all/?search=saltstack.

Ansible

Ansible is a free and simple solution for configuration management. It is well known for automated configuration management, whereby it relies on SSH to access nodes and configure them through Python-based solutions. It uses a minimal amount of dependencies and environment tools (such as client agents), and is secure. It is also very reliable and does not require very advanced learning lessons to start using it.

Its configuration requires the editing of some human-readable configuration files to ensure that everyone can understand them, if an update or reconfiguration is ever needed. It requires only the SSH password or an SSH key in order to access the nodes to be managed, without the need for the installation of an agent. So, there will definitely be no need to worry about not having the agent running due to an upgrade, patch, or daemon malfunction.

Ansible supports modules as well. Basically, most of them run on JSON and it has standard output support for all kinds of programming languages. It also includes more than 200 modules, which can be used to manage and automate services.

It is well supported by the RHEL community (Red Hat, Fedora, CentOS, and Scientific Linux). It is easily downloadable through the EPEL repository. It is also supported by other Linux distributions of the Debian family, and other Unix-like systems such as OS X, BSD, and recently by MS Windows.

Source: https://sysadmincasts.com

Note

For more detailed information on how to set up and master Ansible, please have a look at some of Packt’s other books on Ansible at: https://www.packtpub.com/all/?search=ansible

Puppet

Puppet is an open source configuration management tool designed to manage and configure machines within the same infrastructure, to simplify various system administration tasks where the user declares the machines’ states and resources using different ways (Ruby DSL, Puppet declaration language, and others). It is published under the GPL and Apache licenses. It is mainly used to manage and configure Unix-based machines and Microsoft systems. To establish Puppet, as best practice, we require a Puppet server agent to do the role of a configuration management server and for the other nodes, the ones the server is going to manage, we require the client version. Puppet inspects each node’s configuration, and then identifies the changes between the old and the new configurations specified at its configuration file, if there are any. Then it starts making the required changes to adjust the node to the specified state. This task is useful; not only for establishing a new configuration, but also for regaining it if anything goes wrong with the machines.

Basically, Puppet’s major advantages are that it automates every step of the software delivery process, from the provisioning of physical and virtual machines to organization and reporting. Puppet is widely used by many organizations because it has the following features:

  • It is designed to prevent duplication during the process of problem resolution
  • It gives the capacity to make rapid and repeatable configurations automatically
  • It simplifies the management of both physical and virtual devices
  • It makes the task of system administration much easier

Puppet offers two services: one as an open source and free solution, and the other as a paid solution for enterprises. Both of them serve to help system administrators automate their infrastructure configuration management. Puppet Enterprise has much more to offer for big enterprises.

Source: http://www.slideshare.net/

Puppet is not just a simple configuration management tool. It is a streamlined node installation, management, and repair solution that simplifies the management of a grid of nodes.

As the following figure shows, to use Puppet to establish a new configuration, there are four important steps to be followed: Define, Simulate, Enforce, and Report. Herein, the system administrator first defines what kind of configuration needs to be set. Then, Puppet tests the deployment without harming the infrastructure and makes sure that it will work ok. After that, Puppet starts comparing what is new to what is already being used, and starts enforcing what needs to be done to meet the user’s demands. Finally, it reports the status of each node on its dashboard.

Source: http://www.slideshare.net

Installing and configuring Puppet

For our Puppet installation, we are going to install the free open source version of Puppet.

We have chosen Puppet to be our primary configuration management solution because Puppet is a more declarative solution. This has made it more attractive for damage repair than any other solution.

There are two possible architectures for Puppet: the Agent/Master architecture or the standalone architecture. The Agent/Master architecture is based on one or many Puppet master servers managing servers/nodes that has a Puppet client agent running as a background service. In the standalone architecture, the client nodes run Puppet as a master and slave agent at the same time. Which is used usually to schedule management tasks or jobs to that machine it is installed on.

In this chapter, we are going to use the Client/Master architecture, where we are going to use two CentOS 7 servers; one will be used as the master Puppet server and the other as the client that the Puppet agent will run on.

Before starting the installation, we need to make sure that some preinstallation tasks are done. We have already decided on the architecture, so we have a better idea of which software packages we are going to install.

Then we should define which machine will act as a Puppet server agent. We should always start by installing the server and then move on to the clients.

The following figure presents the interaction between the Puppet master and client when trying to establish a change requested by the manager from the Puppet master interface:

Source: http://elatov.github.io/2014/08/setting-up-puppet-master-on-centos-7/

Our infrastructure will consist of the following:

  • Puppet master server:
    • OS: CentOS 7
    • IP address: 10.0.0.10
    • Hostname: masterpuppet.packt.co.uk
  • First client server:
    • OS: CentOS 7
    • IP address: 10.0.0.11
    • Hostname: webserver1.packt.co.uk
  • Second client server:
    • OS: CentOS 7
    • IP address: 10.0.0.12
    • Hostname: webserver2.packt.co.uk

Usually, we should check which Linux version and family we are using for the clients and the servers of the infrastructure. In our case, we have already defined those to be CentOS 7 for all the machines of our environment.

After that, the Puppet architecture will be managed through the network, so we should make sure that all the nodes (servers and clients) are well configured to run the Puppet traffic. And we need to open the default Puppet access port 8140 for incoming connections at the server so that the client can access it:


$ sudo firewall-cmd --permanent --zone=public --add-port=8140/tcp
$ sudo firewall-cmd --reload

Finally, for the network, we should have all the nodes configured with a unique hostname, and each one of the machines needs to have in its /etc/hosts file all the other machines defined with their IP addresses:


$ sudo nano /etc/hosts
10.0.0.10  masterpuppet.packt.co.uk
10.0.0.11  webserver1.packt.co.uk
10.0.0.12  webserver2.packt.co.uk

To ensure that all the confirmations are synchronized between the nodes, we need to establish a network time-server (using NTP) connection to have all nodes synchronized with the same time:

We can now start the installation of our Puppet server:

  1. The first step is to enable the Puppet open source repository that offers Puppet, PuppetDB, and other packages. We will be using RPM to add the Puppetlabs file to the system:
    
    $ sudo rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
    
  2. Then we install the Puppet server:
    
    $ sudo yum install puppet-server
    
  3. We need to accept all that the installation manager may ask, for example, the RPM-GPG-Key verification, and whether we want to install the package or not.
  4. Then we need to configure Puppet to understand that this node is the master Puppet server:
    
    $ sudo nano /etc/puppet/puppet.conf
    
  5. Next, we need to add these two lines under the [main] section:
    
        dns_alt_names = masterpuppet,masterpuppet.packt.co.uk
        certname = masterpuppet.packt.co.uk
    
  6. Since this is our Puppet server, we need to create its certificate. We need to start the Puppet server manually, for long enough until it creates the required certificate. After that, we need to stop it because we are still running the configuration:
    
    $ sudo puppet master --verbose --no-daemonize
    
  7. After getting the following message, we need to hit Ctrl + C to stop the service:
    
    Notice: Starting Puppet master version 3.8.4
    

At this point, we can start Puppet. But we would like to integrate the Puppet service with Apache to make it one service that can serve both by installing a special module. To do this, we need to install some useful packages:


$ sudo yum install mod_passenger curl curl-devel make automake httpd httpd-devel mod_ssl ruby-devel rubygems gcc gcc-c++ pygpgme zlib-devel openssl-devel

Then we need to install Passenger, which is an open source tool that we will use to install some Apache plugins to help with the development of the Puppet web server. For the installation task, we will use GEM, a Ruby tool for installing Ruby-based packages:


$ sudo gem install rack passenger

From that point onward, we can install the Passenger Apache module. For CentOS 7, we need to locate the executable file. It doesn’t get its binary set in the BIN environment during the installation. So, we need to execute it from where it is installed:


$ sudo locate passenger-install-apache2-module	
$ sudo /usr/local/share/gems/gems/passenger-5.0.21/bin/passenger-install-apache2-module

After that, we follow the instructions provided by the last command as follows.

First, we need to write 1 to install the Apache 2 module:


1. The Apache 2 module will be installed for you.

Then we just type Enter whenever the program asks. We can ignore the warnings and hope for no errors. Otherwise, we need to make sure that we have installed all the required tools earlier. We know that it is done when we see the error and warning summary, and we get this message:


Phusion Passenger is a registered trademark of Hongli Lai & Ninh Bui.

Next, we create the two directories where Puppet is going to generate its public and temporary files, and set ownership to its configuration file:


$ sudo mkdir -p /usr/share/puppet/rack/puppetmasterd/public /usr/share/puppet/rack/puppetmasterd/tmp
$ sudo cp /usr/share/puppet/ext/rack/config.ru /usr/share/puppet/rack/puppetmasterd/
$ sudo chown puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru

To finish, we create the Puppet virtual host file at the Apache2 configuration directory, as follows:


$ sudo nano /etc/httpd/conf.d/puppetmaster.conf

We fill it with the following configuration options, changing the red text depending on what we have configured on a different environment:


# CentOS 7:
# Set those as the Gems and Passenger version installed on your system
LoadModule passenger_module  /usr/local/share/gems/gems/passenger-5.0.21/buildout/apache2/mod_passenger.so
PassengerRoot /usr/local/share/gems/gems/passenger-5.0.21/

PassengerRuby /usr/bin/ruby
# And the passenger performance tuning settings:
PassengerHighPerformance On
PassengerUseGlobalQueue On
# Passenger Max Pool should be configured as 3/2 the number of CPU cores of the master machine:
PassengerMaxPoolSize 4
# Limit number of request before recycling
PassengerMaxRequests 1000
# Limit time for process to set as idle
PassengerPoolIdleTime 600
Listen 8140
<VirtualHost *:8140>
    SSLEngine On
    # Only allow high security cryptography. Alter if needed for compatibility.
    SSLProtocol             All –SSLv3
    SSLCipherSuite          HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP
    SSLCertificateFile	/var/lib/puppet/ssl/certs/unixmen-centos7.arnhem.chello.nl.pem
    SSLCertificateKeyFile	/var/lib/puppet/ssl/private_keys/unixmen-centos7.arnhem.chello.nl.pem
    SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
    SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
    SSLVerifyClient         optional
    SSLVerifyDepth          1
    SSLOptions              +StdEnvVars +ExportCertData
    # These request headers are used to pass the client certificate
    # authentication information on to the puppet master process
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e
    #RackAutoDetect On
    DocumentRoot /usr/share/puppet/rack/puppetmasterd/public/
    <Directory /usr/share/puppet/rack/puppetmasterd/>
        Options None
        AllowOverride None
        Order Allow,Deny
        Allow from All
    </Directory>
</VirtualHost>

We set SELinux to permissive to let the new modules use the resources:


$ sudo nano /etc/selinux/config

Next, we change the SELINUX option to look like this:


SELINUX=permissive

Then we need to restart the Apache service and add it to the system startup services, while disabling the Puppet service:


$ sudo systemctl restart httpd.service
$ sudo systemctl enable httpd.service
$ sudo systemctl disable puppetmaster.service

With this step done, we can say that we are done configuring the server. We can now proceed to setting up and configuring the clients.

Source: http://www.aosabook.org

As we said earlier, we need to make sure that the /etc/hosts file is well configured and has all the nodes (whether client or server) on it. Plus, we need to make sure that the NTP service is running and loading the time for the same time-server as our server. Usually, we can just go with the default set by the system and that works fine:


$ sudo yum install ntp ntpdate
$ sudo systemctl start ntpd
$ sudo systemctl enable ntpd

As with the server, we do the same thing to configure the Puppet repository for the clients:


$ s
udo rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm

Then we install the Puppet client agent:


$ sudo yum install puppet

Next, just before starting the service, we need to add the Puppet master server id to the client agent configuration file. So, we go ahead and edit the Puppet configuration file. We add the server address under the agent option:


$ sudo nano /etc/puppet/puppet.conf
[agnet]
server = masterpuppet.packt.co.uk
report = true
pluginsync = true

Then we save the file, start the service, and enable it for the system startup services:


$ sudo systemctl enable puppet.service
$ sudo systemctl start puppet.service

We need to do the same for all client servers.

After that, we go back to our Puppet server and check whether the certificate has been sent:


$ sudo puppet cert list

We should see something like this:


"webserver1.packt.co.uk" (SHA256) 04:55:E0:82:4E:AD:7F:3C:9C:C7:52:79:98:ED:5E:A6:B0:B5:85:08:DC:81:E5:2D:A6:32:ED:8B:5C:F4:3B:DA

This means that we have a client asking the server to approve a certificate request. We can sign each server certificate using this command:


$ sudo puppet cert sign webserver1.packt.co.uk

We can now go back to the client to check whether the certificate is successfully set using the following command:


$ sudo puppet agent -t

We know that it is well configured if we get the following output:

At this point, our puppet Master and Client infrastructure is working fine. Let’s try and create a configuration from the Puppet server and send it to the client to apply it.

We need to create a manifest file called site.pp in the /etc/puppet/manifest folder, as follows:


$ sudo nano /etc/puppet/manifests/site.pp

Then we test the service by making the client nodes ( webserver1 and webserver2) and install some tools. We need to add the following code to the last opened file for editing:

node 'webserver1', 'webserver2 {
              package { 'php' :
              ensure => installed,
                       }
}

To verify that Puppet is running well we can always go to the client and type in the following:


$ sudo puppet agent --test

By default, the Puppet agent will check the Puppet server every 30 minutes to see whether there is any change that has happened so that it can apply it. Therefore, during the test, we can always force a manual update, as we did earlier. The update time can be changed, by changing it in the Puppet configuration file under the [main] section, as follows:


$ sudo nano /etc/puppet/puppet.conf
runinterval = 900 # To make change every 15 minutes (60 x time in minutes)

Then we can change it to have a service installed and ensure that it is running:

node 'webserver1', 'webserver2 {
              
              package { 'httpd' : 
              ensure => installed, 
                       } ->  # this to ensure that the order before it is executed before going to the order after it
              service {'httpd': 
              ensure => running, 
              enable => true, 
                       }
}

We can also install a program that is located only at the EPEL repository by forcing the installation of the EPEL repository before the program:

node 'webserver1', 'webserver2 {        
package { 'epel-release' : 
              ensure => installed, 
                       }  ->
              package { 'iperf' : 
              ensure => installed, 
                       }
}

Then we can perform some user manipulation where we can create a user on all the client servers. First, we need to create it at the same machine to get its password with the hashing. Then we edit the site.pp file to give some information about the user:

node 'webserver1' {
              
                 user { "usertest":  
                 ensure => "present",
                 shell => '/bin/bash', 
                 home => '/home/usertest', 
                 managehome => true, 
                 password => '############################', }
}

Before finishing the chapter, we would like to explore what the Puppet Labs offer as extra modules, which we can use without the need to create our own, to perform simple, daily tasks. To search for the modules, we use the following command:


$ sudo puppet module search ftp

Then, to install the desired one, we use the install option:


$ sudo puppet module install puppetlabs-apache

Then, to check what is installed, we go to the /etc/puppet/module folder, and there we will have all the modules with their configuration files. We can always change them depending on our needs.

Source: http://static.virtualizationpractice.com

Note

We believe that we have presented some decent information about how to use Puppet for CentOS 7 but still Puppet has a lot to offer. To explore more about it, please check some of Packt’s other books on mastering Puppet at: https://www.packtpub.com/all/?search=puppet

Comments are closed.