Puppet notes:

Intro:

Commands:

  • facter

  • puppet resource ${resource} [ ${inst} ] # displays resource in dsl

  • puppet cert list (master) # lists puppet certs

  • puppet cert sign (master) # signs puppet certs

  • puppet describe ${resource} # describes a specific resource

  • puppet parser validate ${module}/init.pp # Validates syntax of a module

  • validate_erb ${template} # Validates erb syntax

  • puppet lint # style checker

  • puppet describe –list # displays all resource types

  • puppet describe [-s ] ${type} # displays specifics of ${type}

  • puppet module list –tree # lists modules on host.

  • puppet module search ${search_term} # seaches forge for mods.

Things to do:

  • Experiment with puppet enterprise in kvm lab

  • Experiment with other consoles in kvm lab

  • Reengineer home puppet env, possibly using ll from 2 above.

  • Get a ubuntu kvm working so I can test alternate code.

Lessons learned:

  • Be explicit in dsl code.

  • Do not rely on source ordering.

Definitions/acronyms:

DSL:

Domain specific language

idempotence:

Multiple runs of an operation will not change the results.

Day 1:

  • CERN: 50,000+ nodes.

  • puppet forge: 3000+ modules

  • Legacy automation:

    • manually configure - what we do

    • golden images: Andy

    • packaging: Sue

  • Orchestration tool: mcollective

  • puppet master service

    • all comms ssl encrypted.

    • Does not run on aix, solaris, or windows. (lol)

    • As of puppet 3.7, is a JVM

    • More than a few hundred nodes, split install

      • compilation master is very cpu intensive.

      • Multiple masters and a master of masters

    • puppet installation:

      • 100 gb in /opt or

      • use an answers file to put the db/logs in a different directory

      • directions in docs.puppetlabs.com* puppet installation:

  • puppet agent

    • options

    • config example:

      • env vars in [main] evaluated first

      • later definitions override.

      • important:

        • basemodulepath

        • environmentpath

  • VCS process:

    • update local working diretory

    • edit code, make changes,

    • validate style check

    • test code locally

    • update puppet manifest repo

    • test on development nodes

    • elevate to production

  • facter:

    • Dirs:

      • opensource: /etc/facter/facts.d

      • puppetlabs: /etc/puppetlabs/facter/facts.d

      • Format:

        #!/bin/bash

echo "env=PROD"

  • Git:

    • need a write up for work flow

      • git pull

      • edit

      • add/commit

      • git push

      • Test of development infrastructure

  • Lab 4.3: Lots and lots of pointy clicky shit on the lab regarding node classification and overriding environment defaults. Work through it again to ensure you understand everything.

  • Puppet Resources

    • Resources and types are synonymous

    • basic building blocks

    • stacked to create classes.

    • definition:

      • Format:

        ${type} { '${title}':
    ${attribute}   => ${value}
}

      • Rules:

        • type/titles must be unique for a node.

        • titles, even single words, should be wrapped in quotes

        • colon at the end of the resource name

    • puppet describe user

    • Resource references:

      • require/notify, subscribe,before).

      • Referenced type is capitalized (File/Package/Service, etc)

  • Classes are collection of resources; modules are collections of modules

  • Modules

    • Predefined structure. should be followed.

    • Modules should do one discrete task and only that one discrete task.

    • modules location are defined in basemodulepath in puppet.conf

    • module is a directory under the basemodulepath

      • subdir: manifests

      • subdir: moduleso

    • Top level init file is init.pp in ${basemodulepath}/${module}/manifests

    • subclass ssh::server will be in ${base}/${mod}/manifests/server.pp

    • example 1:

      • apache class defined in ${base}/apache/manifests/init.pp

      • apache::mod class defined in ${base}/apache/manifests/mod.pp

      • apache::mod:php class defined in ${base}/apache/manifests/mod/php.pp

  • Autoloading rules:

    • First segment in name idenfies the module

    • Final segment in a name identifies the filename

    • Any intermediate segments are evaluated as subdirectories

    • Modules default class is located in manifests/init.pp

  • Define/declare a class:

    • Defined: listed, written out in a module somewhere.

    • Declaring: included in some module.

  • Testing:

    • Create a tests directory under ${basemodulepath}. Tests is a standard, not a requirement. nothing in puppet is looking for tests subdir

    • init.pp contains include ${module_to_be_tested}

    • puppet apply --noop ${base}/tests/init.pp

  • Node classification:

    • site.pp:

      • Format:

        node '${cert_name}' {
    include ${mod}
    ..
    include ${mod}
}

      • cert_name is what the puppet master signed. not necessarily the hostname but guaranteed unique.

      • perl style regex can be also be used.

      • Suggested to have more specific node definitions at the top of site.pp and more general node definitions further down.

      • as the very last line, node default {…} statement.

        node default {
    notify { "${::fqdn} has no node definition": }
}

      • Do not use both site.pp and an external node classifier.

Day 2:

  • Create hosts class. Cheateated and used info from my home env.

    [dkoleary ~/puppetcode ]# cat modules/hosts/manifests/init.pp
class hosts {
    host { 'testing':
      ensure       => present,
      ip           => '127.0.0.1',
      host_aliases => 'testing.puppetlabs.vm',
    }
}

  • Style guide: need to review

  • meta resource types:

    • notify: been there, done that.

    • resources: sets default parameters

    • schedule: not to be confused with schedule resource type

    • etc

  • schedule usage:

    • format:

      # The schedule resource type
schedule { 'daily maintenance window':
  period => daily,
  range  => '20:00-22:00',
}

exec { '/usr/bin/apt-get update':
  # The schedule metaparameter
  schedule => 'daily maintenance window',
}

    • No guarantee that it’ll run. Nate says don’t do this.

  • Namevar: The name of the item being discussed.

    • Changes based on th eresource.

      • package => name

      • file => path

      • etc

  • File resource:

    • source:

      • Structure:

        puppet://{server hostname (optional)}/{mount point}/\
    {remainder of path}

      • Examples:

        • puppet:///

        • puppet:///modules/apache/index.html

      • URI host is the target that generated the catalog.

        • puppet apply = localhost

        • puppet agent = master server

      • Relative vs absolute:

        • If mount point is modules, url is relative to modules directory.

        • Can also specify absolute path for files outside of vcs:

          puppet:///etc/p-labs/puppet/templates/warning.erb

        • Looks like we have to set up alt file locations in a config file. See https://docs.puppetlabs.com/guides/file_serving.html for details.

    • namespace:

      hosts
├── files
│   └── hosts
├── manifests
│   └── init.pp
└── tests
    └── hosts.pp

  • Resource dependencies:

    • Relationship metaparameters:

      • require: referenced resource must be applied first. NOTE: Referenced resource must be the capitalized title

      • before: referenced resource must be applied first.

      • subscribe: listens for changes to referenced resource. If changes noticed, calling resource is refreshed.

        service { 'sshd':
  ensure    => running,
  enable    => true,
  subscribe => File['/etc/ssh/sshd_config'],
}

      • notify: If current resource changes, will cause a refresh of referenced resource:

        file { '/etc/ssh/sshd_config':
  ensure => present,
  source => 'puppet:///modules/ssh/sshd_config',
  notify => Service['sshd'],
}

      • Suggested ordering:

        • Package

        • File: require => Package

        • service: subscribe => File

    • Implicit dependencies:

      • Users/groups

      • Files/directories

      • File ownership

  • Typical puppet development cycle:

    • developer

      • pulls modules

      • Writes modules

      • pushes modules

    • VCS centralized - need some expansion on here. How is code reviewed

    • puppet master:

      • pulls codes

      • compiles catalogs & runs.

  • Language constructs:

    • Variables:

      • Declaration: $httpd_dir = ‘/etc/httpd/conf’

      • Reference: file { “${httpd_dir}”: }

      • single quotes do not evaluate variables

      • Can bracket everything, but must be double quoted?

      • Variables are immutable: cannot be reaasigned.

      • Scope:

        • params file: like what I use

        • $apache::params::httpd_dir

      • Global facts: (facter) ${::hostname}

    • Resource defaults:

      • Resource specified with a capital:

        File {
    owner => 'root',
    group => 'root',
    mode  => 0644,
}

      • like owner/group/mode will be set for all applicable resources

      • modes of 0644 on a directory will automatically set to 0755 only if the read bit is set (600 will become 700, etc)

    • Arrays:

      • eg: $somearray = [ ‘one’, ‘two’, ‘three’ ];

      • This example:

        file { ['/opt/app','/opt/app/logs', '/opt/app/logs/crash']:
    ensure = directory,
    owner => root,
    [[snip]]

        will createa three distinct resourc types which if ralized, will result in the entire tree being created.

    • Conditionals:

      • selectors:

        package { 'ssh':
    ensure => present,
    name   => $::operatingsystem ? {
    'Ubuntu' => 'ssh',
    'Redhat' => 'openssh',
    default  => 'openssh',
    },
}

        • Puppet version of a trinary op:

          • Syntax:

            $pkg = $::operatingsystem ? {
    'redhat' => 'ssh',
    'dbian'  => 'openssh,
    default  => 'openssh,
}

          • ${::operatingsystem} ? syntax requires a default entry.

        • case insensitive.

        • $var = $::operatingsystem ? { [[snip]] another syntax:

          $sshpkgname = $::operatingsystem ? {
  'ubuntu' => 'ssh',
  default  => 'openssh',
}

package { 'ssh':
  ensure => present,
  name   => $sshpkgname,
}

    • Conditionals:

      • case statements

      • if/elsif/else

        • False:

          • undef

          • ‘’

          • false

        • All eles is true.

    • Functions:

      • functions run on master

      • Only run during catalog compilation.

      • statement functions: take actions w/o returning a value.

        • node {..}

        • include

        • realize

      • rvalue functions: return a value

        • content => template(‘apache/vhost.erb’);

        • generate: returns the resulf of a shell command

      • Complete list of what looks like rvalue functions only is availabe at the function reference list

  • Template:

    • Lab provided was pretty straight forward.

    • Separation of concerns: Keeping code generation and file presentation separate.

    • Basic syntax:

      • Normal vars: ${text} <%= @ruby_vriable %>

      • Arrays:

        <% @array.each do |val| -%>
    array has an item with a value of <%= val %> # note no '@'
<% end -%>

      • ‘-%>’ ending will prevent a new line from being printed.

      • Conditionals:

        • if:

          <% if @kernel != 'Linux' %>
    This is a <%= @kernel %> system.
<% end %>

        • Existance:

          <% if @vlan then -%>
    <%= @vlan %> exists
<% end -%>

    • All variables smashed down into the same name space. If using multiple classes, this could cause issues. This may not be true. http://docs.puppetlabs.com/guides/templating.html seems to paint a different picture re variable scope particularly with the scope.lookupvar function.

Day 3:

I missed all of day three because of a production issue at work. An issue with a piece of crap software that realistically should have been gone more than FIVE FUCKNG YEARS AGO! This time, I’m fucking bitter!

So, I’m trying to do day three on my own. fucking lovely.

  • String comparisons in puppet are case insensitive. Need to test/verify and/or play with that.

  • Defined resource types:

    • modularize chunks of configuration. Example given is apache virtual hosts.

    • Example: File location: ${mod}/apache/manifests/vhost.pp

      define apache::vhost (
    $docroot,
    $port = '80',
    $priority = '10',
    $options = 'Indexes MultiViews',
    $vhost_name = $title,
    $servername = $title,
    $logdir = '/var/log/httpd',
) {
  file { "/etc/httpd/conf.d/${title}.conf":
    ensure => file,
    owner => 'apache',
    group => 'apache',
    mode => '0644',
    content => template('apache/vhost.conf.erb'),
  }
}

    • ${title} is the title provided when the resource is realized. Book says To maintain uniqueness, always derive the titles of resources included within your defined resource type from $title.

    • Leveraging a template based on the above definition: File location: ${mod}/apache/templates/vhost.conf.erb

      NameVirtualHost <%= @vhost_name %>:<%= @port %>
<VirtualHost <%= @vhost_name %>:<%= @port %>>
  ServerName <%= @servername %>
  DocumentRoot <%= @docroot %>
  <Directory <%= @docroot %>>
    Options <%= @options %>
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>
  ErrorLog <%= @logdir %>/<%= @title %>_error.log
  LogLevel warn
  CustomLog <%= @logdir %>/<%= @title %>_access.log combined
  ServerSignature Off
</VirtualHost>

  • Parameterized classes:

    • Basically, similar in model to function arguments. Not clear on the declaration vs instantiation, though.

    • Working my way through the lab, semi-clearing things up:

      • Class definition is still in the normal namespace. IE: init.pp contains the full class definition.

      • Sub class can use similar syntax to override parameters that are instantiated when the class is. Kind of cirumvents the whole ‘thou canst reassign variables in a class’ thing.

      • Got it. Subtle difference.

        • class definitinos are class ${title} {...}

        • Passing parameters is class { ${title: ... }

  • Inheritance:

    • Only use with a parameters subclass.

    • Reason to inherit vs include is to be able to override class parameters. Syntax is as follows:

      class apache (
  $httpd_user    = $apache::params::httpd_user,
  $httpd_group   = $apache::params::httpd_group,
  $httpd_pkg     = $apache::params::httpd_pkg,
  $httpd_svc     = $apache::params::httpd_svc,
  $httpd_conf    = $apache::params::httpd_conf,
  $httpd_confdir = $apache::params::httpd_confdir,
  $httpd_docroot = $apache::params::httpd_docroot,
) inherits apache::params {
[[rest-o-class-def-snipped]]

    • Class ‘inherits’ reasonable defaults from $apache::params yet can override them using class resource declaratios as described in previous section.

  • Hiera

    • Site specific info is not maintained in the manifests.

    • yaml file syntax:

      ---
:backends:
- yaml

:yaml:
:datadir: '/etc/puppetlabs/puppet/environments/production/hieradata'

:hierarchy:
- "%{clientcert}"
- "%{datacenter}"
- defaults

    • Params:

      • backend-> yaml. self explanatory

      • yaml -> datadir: specifies directories under which search should be conducted

      • hiearchy: specifies the order in which resolution should occur, most specific to least specific. Files will be under :datadir:

      • hiera searches are automatic as of puppet 3.x

        class ntp (
  $time_server, # auto uses hiera('ntp::time_server') default
  $crypto = false, # auto uses hiera('ntp::crypto', false) default
) {
  file { '/etc/ntp.conf'
    content => template('ntp/ntp.conf.erb')
  }
}

        is the same as:

        class ntp (
  $time_server = hiera('ntp::time_server'),
  $crypto = hiera('ntp::crypto', false),
) {
  file { '/etc/ntp.conf'
    content => template('ntp/ntp.conf.erb')
  }
}