Endring av Icinga2 for å bruke hoved-/klientmodell på CentOS 6 eller CentOS 7

Icinga2 er et kraftig overvåkingssystem, og når det brukes i en master-klient-modell, kan det erstatte behovet for NRPE-baserte overvåkingskontroller. Master-klient-modellen er sikrere av ulike årsaker . Det gir også mulighet for mer distribuert overvåking ved å tillate at sjekker kan endres på én server og spres til andre. Det gir også ett sentralisert sted for endringer.

For eksempel: La oss si at du har 4 forskjellige belastningsbalanserte webservere. Hvis du vet at du kommer til å ha en belastningstopp (kanskje et salg eller noe du forventer mye aktivitet for), kan det være lurt å øke nivåene der høy CPU-bruk varsler systemadministratorene. Hvis du bruker NRPE-sjekker, må du gå inn på hver server og endre /etc/nrpe.d/common_commands.cfgdokumentet på hver av dem. I master-klient-modellen kan du ganske enkelt endre vertsdefinisjonene i /etc/icinga2/repository.d/hosts/mappen, og deretter laste inn Icinga på nytt. Sjekkene vil forplante seg til hver server og vil bli plukket opp automatisk. Du kan også legge til sjekker veldig enkelt ved å modifisere /etc/icinga2/repository.d/services/filene, og de vil automatisk forplante seg og bli gjenkjent, så lenge plugin-modulene finnes på klientserverne.

Forutsetninger

• En serverforekomst med Icinga2 og Icingaweb2 allerede installert og konfigurert ( instruksjoner ) navngitt icinga2master.example.com.
• En fersk Vultr CentOS 6 eller 7 serverforekomst du ønsker å overvåke heter icinga2client.example.com.
• Sudo tilgang.

Noe av denne veiledningen vil speile installasjonsinstruksjonene som er oppført ovenfor, men dette må gjøres på klientmaskinen ikke servermaskinen.

Trinn 1: Oppdater systemet

Legg til "ICINGA"-repoen til klientsystemet for å installere Icinga-pakkene.

sudo cat > /etc/yum.repos.d/ICINGA-release.repo << 'EOF'
[icinga-stable-release]
name=ICINGA (stable release for epel)
baseurl=http://packages.icinga.org/epel/$releasever/release/
enabled=1
gpgcheck=1
gpgkey=http://packages.icinga.org/icinga.key
EOF

Trinn 2: Installer nødvendige pakker

Installer de nødvendige pakkene.

# CentOS 6
sudo yum -y install icinga2 mysql-server mysql icinga2-ido-mysql nagios-plugins-disk nagios-plugins-load nagios-plugins-procs nagios-plugins-ssh nagios-plugins-users nagios-plugins-swap nagios-plugins-ping icingacli

# CentOS 7
sudo yum -y install icinga2 mariadb-server mysql icinga2-ido-mysql nagios-plugins-disk nagios-plugins-load nagios-plugins-procs nagios-plugins-ssh nagios-plugins-users nagios-plugins-swap nagios-plugins-ping icingacli

CentOS 7 bruker MariaDB i stedet for MySQL, så de nødvendige pakkene er litt forskjellige. Start tjenestene, og sett dem til å starte ved oppstart.

sudo chkconfig icinga2 on
sudo service icinga2 start

Sett MySQL-tjenesten til å starte ved oppstart og start den nå slik at vi kan sikre den.

# CentOS 6
sudo chkconfig mysqld on
sudo service mysqld start

#CentOS 7
sudo systemctl enable mariadb.service
sudo systemctl start mariadb.service

Sikre MySQL/MariaDB-installasjonen. Både CentOS 6 og 7 bruker samme kommando.

sudo mysql_secure_installation

Under prosessen, svar på spørsmålene som vist:

Enter current password for root (enter for none): Enter
Set root password? [Y/n]: Y
New password: $newRootPassword
Re-enter new password: $newRootPassword
Remove anonymous users? [Y/n]: Y
Disallow root login remotely? [Y/n]: Y
Remove test database and access to it? [Y/n]: Y
Reload privilege tables now? [Y/n]: Y

Legg merke til $newRootPasswordsiden du trenger det flere ganger, og det er vondt å tilbakestille.

Trinn 3: Sett opp databasen

Du må opprette icingabrukernavnet og importere skjemaet til databasen. Dette gjøres på samme måte på både CentOS 6 og 7.

sudo mysql -u root -p
>CREATE DATABASE icinga;
>GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icinga.* TO 'icinga'@'localhost' IDENTIFIED BY '$someIcingaPassword'; 
>exit

Deretter importerer du skjemaet til databasen du nettopp har opprettet.

sudo mysql -u root -p icinga < /usr/share/icinga2-ido-mysql/schema/mysql.sql

Du må skrive inn $newRootPasswordMySQL/MariaDB-passordet du brukte under mysql_secure_installationkommandoen ovenfor, ikke det $someIcingaPassworddu nettopp har opprettet - det brukes senere.

Aktiver ido(database)-modulen.

sudo icinga2 feature enable ido-mysql

Du må redigere /etc/icinga2/features-enabled/ido-mysql.conffilen for å legge til databaseinformasjonen. Du kan gjøre det med følgende kommando.

sudo vi /etc/icinga2/features-enabled/ido-mysql.conf

I den filen finner du linjene som er kommentert ut (start med //).

//user = "icinga"
//password = "icinga"
//host = "localhost"
//database = "icinga"

...og rediger dem med brukeren/passordet du nettopp har opprettet.

user = "icinga"
password = "$someIcingaPassword"
host = "localhost"
database = "icinga"

Pass på at du fjerner kommentarene til linjene ved å fjerne de to første skråstrekene.

Deretter starter du tjenesten på nytt for å hente endringene.

sudo service icinga2 restart

Du må legge til en forbindelse mellom klientserveren og Icinga2-masteren som allerede er konfigurert. Du må gjøre det ved å bruke enten Firewalld på CentOS 7, eller IPTables i enten CentOS 6 eller 7.

# CentOS 6
sudo vi /etc/sysconfig/iptables
...
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5665 -j ACCEPT
...
#Icinga2 Master
-A OUTPUT -p tcp -m state --state NEW -m tcp --dport 5665 -j ACCEPT

sudo iptables restart

# CentOS 7    
sudo firewall-cmd --add-port=5665/tcp --permanent
sudo firewall-cmd --reload

Rediger nå vertsfilen for å legge til masteren som et lokalt domenenavn.

sudo vi /etc/hosts
...
$masterIP  icinga2master icinga2master.example.com

Trinn 4: Initialiser masteren

Dette trinnet må gjøres på Icinga2-masteren. Du må initialisere den som en master, og deretter generere en sertifikatsigneringsforespørsel som klienten din vil bruke til å kommunisere.

sudo icinga2 node wizard

This will run the node wizard, which will ask several questions. The screen will look like the following, and the answers you will give are listed.

Welcome to the Icinga 2 Setup Wizard!

We'll guide you through all required configuration details.

Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]: n
Starting the Master setup routine...
Please specify the common name (CN) [icinga2-master1.localdomain]: icinga2master.example.com
Checking for existing certificates for common name 'icinga2master.example.com'...
Certificates not yet generated. Running 'api setup' now.
information/cli: Generating new CA.
information/base: Writing private key to '/var/lib/icinga2/ca/ca.key'.
information/base: Writing X509 certificate to '/var/lib/icinga2/ca/ca.crt'.
information/cli: Generating new CSR in '/etc/icinga2/pki/icinga2master.example.com.csr'.
information/base: Writing private key to '/etc/icinga2/pki/icinga2master.example.com.key'.
information/base: Writing certificate signing request to '/etc/icinga2/pki/icinga2master.example.com.csr'.
information/cli: Signing CSR with CA and writing certificate to '/etc/icinga2/pki/icinga2master.example.com.crt'.
information/cli: Copying CA certificate to '/etc/icinga2/pki/ca.crt'.
Generating master configuration for Icinga 2.
information/cli: Adding new ApiUser 'root' in '/etc/icinga2/conf.d/api-users.conf'.
information/cli: Enabling the 'api' feature.
Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
information/cli: Dumping config items to file '/etc/icinga2/zones.conf'.
information/cli: Created backup file '/etc/icinga2/zones.conf.orig'.
Please specify the API bind host/port (optional):
Bind Host []: <hit enter>
Bind Port []: <hit enter>
information/cli: Created backup file '/etc/icinga2/features-available/api.conf.orig'.
information/cli: Updating constants.conf.
information/cli: Created backup file '/etc/icinga2/constants.conf.orig'.
information/cli: Updating constants file '/etc/icinga2/constants.conf'.
information/cli: Updating constants file '/etc/icinga2/constants.conf'.
information/cli: Updating constants file '/etc/icinga2/constants.conf'.
Done.

Now restart your Icinga 2 daemon to finish the installation!

Then, restart Icinga2 to pick up the changes.

sudo service icinga2 restart

Now that it's a master, you can use it to generate the certificate signing request string.

sudo icinga2 pki ticket --cn icinga2client.example.com

You will get some string $pkiString. Copy that string, as you'll need it on the client.

Step 5: Initialize the client

Run the node wizard to generate the certificates and connect to the master.

sudo icinga2 node wizard

It will come up with a similar page to the above. This page is shown below with answers included.

Welcome to the Icinga 2 Setup Wizard!

We'll guide you through all required configuration details.



Please specify if this is a satellite setup ('n' installs a master setup) [Y/n]: Y
Starting the Node setup routine...
Please specify the common name (CN) [icinga2client.example.com]:
Please specify the master endpoint(s) this node should connect to:
Master Common Name (CN from your master setup): icinga2master.example.com
Do you want to establish a connection to the master from this node? [Y/n]: Y
Please fill out the master connection information:
Master endpoint host (Your master's IP address or FQDN): icinga2master.example.com
Master endpoint port [5665]: 5665
Add more master endpoints? [y/N]: N
Please specify the master connection for CSR auto-signing (defaults to master endpoint host):
Host [icinga2master.example.com]: icinga2master.example.com
Port [5665]: 5665
information/cli: Created backup file '/etc/icinga2/pki/icinga2client.example.com.key.orig'.
information/cli: Created backup file '/etc/icinga2/pki/icinga2client.example.com.crt.orig'.
information/base: Writing private key to '/etc/icinga2/pki/icinga2client.example.com.key'.
information/base: Writing X509 certificate to '/etc/icinga2/pki/icinga2client.example.com.crt'.
information/cli: Fetching public certificate from master (icinga2master.example.com, 5665):

Certificate information:

 Subject:     CN = icinga2master.example.com
 Issuer:      CN = Icinga CA
 Valid From:  Jan 10 21:08:37 2017 GMT
 Valid Until: Jan  7 21:08:37 2032 GMT
 Fingerprint: FE 72 AB F3 18 A5 12 E0 0C 5D 94 8B 96 C4 57 3B 00 5C E0 04

Is this information correct? [y/N]: Y
information/cli: Received trusted master certificate.

Please specify the request ticket generated on your Icinga 2 master.
 (Hint: # icinga2 pki ticket --cn 'icinga2client.example.com'): $pkiString

information/cli: Requesting certificate with ticket '$pkiString'.

warning/cli: Backup file '/etc/icinga2/pki/icinga2client.example.com.crt.orig' already exists. Skipping backup.
information/cli: Writing signed certificate to file '/etc/icinga2/pki/icinga2client.example.com.crt'.
information/cli: Writing CA certificate to file '/etc/icinga2/pki/ca.crt'.
Please specify the API bind host/port (optional):
Bind Host []: <enter key>
Bind Port []: <enter key>
Accept config from master? [y/N]: Y
Accept commands from master? [y/N]: Y
information/cli: Disabling the Notification feature.
Disabling feature notification. Make sure to restart Icinga 2 for these changes to take effect.
information/cli: Enabling the Apilistener feature.
warning/cli: Feature 'api' already enabled.
information/cli: Created backup file '/etc/icinga2/features-available/api.conf.orig'.
information/cli: Generating local zones.conf.
information/cli: Dumping config items to file '/etc/icinga2/zones.conf'.
information/cli: Created backup file '/etc/icinga2/zones.conf.orig'.
information/cli: Updating constants.conf.
information/cli: Created backup file '/etc/icinga2/constants.conf.orig'.
information/cli: Updating constants file '/etc/icinga2/constants.conf'.
information/cli: Updating constants file '/etc/icinga2/constants.conf'.
Done.

Now restart your Icinga 2 daemon to finish the installation!

Go ahead and restart Icinga2 to pick up the changes.

sudo icinga2 restart

This may already be done, but you need to ensure that the command and API features are enabled.

sudo icinga2 feature enable command
sudo icinga2 feature enable api

If they weren't enabled, it will say "feature enabled", so you'll need to restart the Icinga2 daemon. But first, you need to change ownership of the files in the /etc/icinga2 directory, as some of them have the wrong permissions after running the node wizard. They should all have their owner and group set to icinga. Change the permissions by using the following command:

sudo chown -R icinga:icinga /etc/icinga2/

Now, you can restart the Icinga2 daemon.

sudo service icinga2 restart

Step 6: Configure the client

The client configuration requires editing a few different files.

• /etc/icinga2/zones.conf: Contains the endpoints (servers) and zones that contain them. This is how the servers are partitioned. Make sure the client names match the server names.
• /etc/icinga2/icinga2.conf: You are accepting commands and configs from the master, so you need to do a minor edit on this file to avoid duplicate command definitions.

First, we're going to modify the zones.conf file to define the server and the zone it's in, as well as creating a "global" zone for command definitions that are universal to ALL servers, instead of unique to each (such as disk checks).

sudo vi /etc/icinga2/zones.conf

object Zone clientZone { 
        endpoints = [ "icinga2master.example.com" ]
}

object Zone "global-templates" { # Add global templates zone
        global = true
}

object Endpoint NodeName {
        host = "$icinga2clientIP" 
}

object Zone ZoneName {
        endpoints = [ NodeName ]
        parent = clientZone
} 

It is important to know, there can only be 2 nodes per zone at the moment. You can make as many zones as you wish, but if there are more than 2 nodes per zone there is a chance that the Icinga2 service will not work properly. This is not a problem on the client, unless you already have 2 nodes in the zone clientZone, in which case you will need to make another zone name.

Now, make the modification in the /etc/icinga2/icinga2.conf file. You need to comment out the line include_recursive "conf.d".

sudo vi /etc/icinga2/icinga2.conf
...
//include_recursive "conf.d"
...

Verify config to ensure that everything is formatted correctly.

sudo icinga2 daemon -C

If you get permissions problems, it's likely because the item is owned by root:root. Chown to icinga:icinga to resolve.

sudo chown -R icinga:icinga /etc/icinga2/

There are other issues with the configs, generally the information given from the output of the sudo icinga2 daemon -C command will be verbose enough to track them down. Addressing every potential issue that arises is outside the scope of this guide.

If the verification succeeds, go ahead and reload the service:

sudo service icinga2 restart

You'll need to give the icinga user sudo permissions so that the commands can be run. These sudo permissions will be restricted to the specific plugin directory. This can just be put at the end of the file.

sudo vi /etc/sudoers
...
Defaults:icinga !requiretty
icinga          ALL=(ALL) NOPASSWD: /usr/lib64/nagios/plugins/

Now the client configuration is officially finished. You shouldn't need to make any more modifications, any service checks will configured on the master and will propagate down to the client. You will need to install any additional plug-ins you might need, however. Plug-ins can also be written using python, perl, BASH, PHP, and other scripted languages.

Step 7: Add client to master config

You will need to get back onto the master to configure the client so that it can be connected. There are a few steps that will need to be taken:

1. Add host config to repository.d.
2. Add host to zone as endpoint.
3. Configure any checks for the client. This is slightly out of scope of this guide, I'll provide only a basic explanation of the steps involved.

The host config will be entered in the /etc/icinga2/repository.d/hosts/icinga2client.example.com.conf file.

sudo vi /etc/icinga2/repository.d/hosts/icinga2client.example.com.conf
object Host "icinga2client.example.com" {
  import "satellite-host"
  address = "$icinga2clientIP"
  vars.os = "Linux"
}
sudo chown icinga:icinga /etc/icinga2/repository.d/hosts/icinga2client.example.com.conf

Now the host exists in the client config, but it needs to be put into a zone. This zone will be in the /etc/icinga2/repository.d/zones folder, with the same name as the zone name to make it easier to keep track.

sudo vi /etc/icinga2/repository.d/zones/clientZone.conf
object Zone "clientZone" {
        endpoints = [ "icinga2client.example.com" ]
        parent = "icinga2master.example.com"
}

Next, you need to add the global-templates zone.

sudo vi /etc/icinga2/repository.d/zones/global-templates.conf
object Zone "global-templates" {
        global = true
}

Finally, add the client endpoint so that the "clientZone" endpoints array is populated when it runs.

sudo vi /etc/icinga2/repository.d/endpoints/icinga2client.example.com.conf

#Change values to match the host you're adding
object Endpoint "icinga2client.example.com" {
        host = "$icinga2clientIP"
}

In order to make sure there are no permissions issues, go ahead and change the owner/group again.

sudo chown -R icinga:icinga /etc/icinga2

Verify configuration changes to make sure there are no more issues.

sudo icinga2 daemon -C

If verification succeeds, go ahead with the reload.

sudo service icinga2 restart

If you visit your Icinga2 website, you will see the server become accessible.

Step 8: Custom commands and checks

You may encounter errors about "command not found" or "check not defined". This is where the global-templates zone comes into play. You will need to copy the configuration in /etc/icinga2/conf.d/ files and put them into /etc/icinga2/zones.d/global-templates/ files. For example, to create a "check_software" command, you would need to do do the following.

sudo mkdir /etc/icinga2/zones.d/global-templates
sudo vi /etc/icinga2/zones.d/global-templates/commands.conf
object CheckCommand "check_software" {
  import "plugin-check-command"
  command = [ "/usr/bin/sudo",  PluginDir + "/software_service_check.sh" ]
}

The software_service_check.sh file will need to exist in the /usr/lib64/nagios/plugins folder on the client.

Next, add the service check into the /etc/icinga2/repository.d/hosts/services.conf file.

sudo vi /etc/icinga2/repository.d/hosts/services/services.conf
apply Service "Check Software Service" {
  import "mail-service"
  check_command = "check_software" # This check command is the same name as the one you defined in global-templates/commands.conf

  assign where host.vars.client_endpoint # This will apply to every client server. If you need it to be more restrictive, you should look into custom variables
}

You'll need to change the permissions again.

sudo chown -R icinga:icinga /etc/icinga2

Then, re-verify configuration.

sudo icinga2 daemon -C

Finally, if it succeeds, reload:

sudo service icinga2 restart

Congratulations! You are now using Icinga2 in a master-client model and can now avoid having to use the insecure NRPE checks!