Automated Nagios monitoring with Puppet exported resources

NagiosOne of the coolest things Puppet can do is create exported resources. In plain words it means that you can include a manifest on all your nodes which then gets customised and applied for each node thanks to the node facts.
A popular usage of exported resources is automating Nagios monitoring. Instead of manually creating a Nagios configuration with the basic checks such as load, disk usage etc… then duplicating it for each server and changing the hostname, an exported resource can do it all for you by including a single manifest.

PuppetDB configuration

Exported resources requires storing facts for the Puppet nodes so we need PuppetDB installed on the PuppetMaster.

Create /etc/puppet/puppetdb.conf

Add to /etc/puppet/puppet.conf under [main]

Create /etc/puppet/routes.yaml

Start PuppetDB and restart the PuppetMaster:

Nagios Server Configuration

Create a Nagios module with the following manifests:

nagios/manifests/init.pp

nagios/manifests/install.pp

Note that at the time of writing, Nagios 4 isn’t available as a package for Ubuntu 12.04 LTS so we’re assuming you built your own one which installs it under /etc/nagios4

nagios/manifests/service.pp

There is a bug in Puppet when the exported resources are created, they do not have the correct permissions to allow the nagios user on the server to read them so declare a fix-permissions exec resource.

nagios/manifests/import.pp

The exported resource operator <<||>> (not to be confused with the spaceship <||> operator) is the resource which will realize all @@ virtual exported resources. How it works is described further down.

Nagios NRPE nodes

For all nodes which you want to monitor with nagios, we need the Nagios NRPE server installed

nagios/manifests/nrpe.pp

Then the export manifest where the @@nagios virtual resources are declared

nagios/manifests/export.pp

To include Nagios on all nodes to monitor, just add to the default node on manifests/nodes.pp

How it works

When you declare an exported virtual resource on the node, after the puppet agent run, the exported configurations are stored into PuppetDB. Then when you run the puppet agent on the Nagios server, it collects all the nodes exported resources from PuppetDB and subsequently creates the Nagios .cfg files. Beware that if you have a lot of nodes that use exported resources, it can create a long catalogue compilation time, so consider extending the run interval of the puppet agent.

Therefore all nodes must trigger a puppet run before the server can import the configurations.

Note that if you remove nagios::export class from a node, it will not remove the exported configurations in PuppetDB, the resources will still be created on export. You need to keep the virtual exported resources and set ensure to absent. Or if you’re completely removing the node, on the PuppetMaster you can deactivate it with

Exported resources can have many other uses such as managing sshkey know_hosts resources or for dynamically adding/removing load balancer members to Apache for example.

14 thoughts on “Automated Nagios monitoring with Puppet exported resources

    • Hi Thomas,

      I did not have a nagios::server class. The nagios::install takes care of installing it, if you have packages. You may need to fine tune it, depending on how you’re installing it.

      • I am getting this error as well:
        Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid relationship: Service[nagios] { require => Class[Nagios::Server] }, because Class[Nagios::Server] doesn’t seem to be in the catalog

        I do not see a class nagios::server defined anywhere.
        -Dave

  1. Hello,
    Very good article 😉 . I’ve one question, the template create for the nodes based on export.pp include standar checks (cpu, ping, ssh, etc) but ¿is possible add other checks for a specific node (example nfs) but this checks no apply all nodes? .

    Regards.

  2. Hi,
    Nice writeup, I’m not sure if its the same in Nagios 4 but in 3 I had to make sure that nrpe was allowed to be contacted by the nagios server. I did this by adding the following into the nrpe config file
    file_line { “allowed_hosts”:
    line => “allowed_hosts = 127.0.0.1,192.168.1.29”,
    path => “/etc/nagios/nrpe.cfg”,
    match => “allowed_hosts”,
    ensure => present,
    notify => Service[“nrpe”],
    }
    where 192.168.1.29 is my nagios server.

  3. Can I ask how would I work if the client is not Linux and rather a hardware device like a vpn Router? In that case we can’t use NRPE but snmp for it.

  4. Hi Tom,

    I had a query with regards to using hostgroups. You used hostgroups in @@nagios_host but that doesn’t create a hostgroups.cfg. It gives me this error.
    Error: Could not find any hostgroup matching ‘Mgmt Servers’ (config file ‘/usr/local/nagios/etc/objects/dev-local_hosts.cfg’, starting on line 66)

    My target.pp
    ============
    class nagios::target {
    @@nagios_host { $fqdn:
    ensure => present,
    target => “/usr/local/nagios/etc/objects/${hostname}.cfg”,
    alias => $hostname,
    address => $ipaddress,
    hostgroups => “Mgmt Servers”,
    use => “linux-server”,
    }

    monitor.pp
    ===========
    Nagios_host <> {
    target => “/usr/local/nagios/etc/objects/dev-local_hosts.cfg”,
    }
    Nagios_service <> {
    target => “/usr/local/nagios/etc/objects/dev-local_service.cfg”,
    }
    Can you tell me what am i missing??

    Regards,
    Farooq Ahmed

Leave a Reply

Your email address will not be published. Required fields are marked *