/===================== Puppet notes - take 2: ======================
Overview:¶
Going back over the pro puppet book. Now that we’re closing in on installing puppet at work, it’d behoove me to be the best puppet engineer I can… and the best at MPI.
To-do:¶
Go over the url for bootstapping with puppet.
Review params subclass inheritance examples
Read puppet style guide: https://docs.puppetlabs.com/guides/style_guide.html
Examine vagrant (http://www.vagrantup.com) for interaction with kvm.
Lessons learned:¶
DSL = Domain Specific Language
puppet won’t start without a sites.pp file.
puppet key’s not working with the spacewalk. need to run the following command on all clients:
rpm --import https://yum.puppetlabs.com/RPM-GPG-KEY-puppetlabs
Date needs to be in sync across the puppet environment.
Directories to back up:
/var/lib/puppet/ssl
Definition vs instantiation:
Definitions go in module files (vhosts.pp)
instantiations go in node files:
node '$host' { include base include apache apache:vhost {'www.oci.com': port... docroot... ssl ... etc } apache:vhost {'www.nuther.com': ... } apache:vhost {'www.nuther1.com': ... } }
notify vs notic:
notify is a puppet resource:
notice {"Hello, world": }Class is determined by location in the log and/or display msg
notice is a function:
``notice(“Hello, world”)
Class is determined by output:
# puppet apply -e 'notice("Hello, world")' Notice: Scope(Class[main]): Hello, world
Passing an array to a type (like notify) produces one resource of that type for every element in the array.
# puppet apply -e 'notice {["one","two", "three"]:}' puppet apply -e 'notify {["one","two", "three"]:}' Notice: Compiled catalog for node1.olearycomputers.com in environment production in 0.03 seconds Notice: one Notice: /Stage[main]/Main/Notify[one]/message: defined 'message' as 'one' Notice: three Notice: /Stage[main]/Main/Notify[three]/message: defined 'message' as 'three' Notice: two Notice: /Stage[main]/Main/Notify[two]/message: defined 'message' as 'two' Notice: Finished catalog run in 0.07 seconds
Valuable commands:¶
puppet help
- puppet config print …
Prints configuration settings. For instance:
# puppet config print ssldir /var/lib/puppet/ssl
- puppet module generate ${name}
Generates an involved puppet module after asking a bunch of questions
- puppet cert list [–all]
lists outstanding certs. If –all is supplied, signed certs are prepended with a ‘+’
- puppet parser validate
Validates the syntax of the module.
# puppet parser validate manifests/init.pp # echo $? 0
- erb -x -T ‘-’ main.cf.erb | ruby -c
Checks syntax of a template:
# erb -x -T '-' main.cf.erb | ruby -c Syntax OK
- puppet apply -e ‘if “Puppet” == “puppet” { notify {“true!?”:}}’
Demo of testing puppet syntax
Interesting URLs:¶
- http://docs.puppetlabs.com/references/stable/type.html:
Full list of puppet manageable types
- http://projects.puppetlabs.com/projects/1/wiki/Bootstrapping_With_Puppet
Bootstrapping with puppet
- http://docs.puppetlabs.com/guides/faq.html#Common+Misconceptions
FAQ
- http://docs.puppetlabs.com/puppet/latest/reference/lang_node_definitions.html#inheritance
Puppet documentation on inheritence and why you shouldn’t…
- http://docs.puppetlabs.com/puppet/latest/reference/lang_summary.html#conditionals
Puppet language basics: conditioanls specifically.
- http://docs.puppetlabs.com/guides/style_guide.html
Puppet style guide
- https://github.com/openstack-infra/system-config/blob/master/manifests/site.pp
openstack puppet infrastructure: example of using parameterized classes and a glue class.
03/31/15:¶
I have 2 kvm guests, one’s ubuntu. I’m probably going to blast the centos one because it’s already using puppet. I want to get this thing running via the puppet master this time.
Declarative vs imperative language.
Layers
Resource abstracton layer - facter
Transactional layer - the puppet engine
Creates graph showing resources, relationships. Used to calculate action order.
Mixed releases of puppet:
puppet master must be newest release of the batch
Older versions don’t support everything.
Stopping at the installation. It’s getting late.
04/01/15:
Installation:
prereqs:
ruby
ruby-libs
ruby-shadow
epel repo
Master:
puppet
puppet-server
facter
Clients:
puppet
facter
W/my setup, all available packages are on the spacewalk server.
Ubuntu:
apt-get install puppet facter [puppetmaster]
Firewall config: 8140 needs to be open.
The /etc/puppet/ssl dir doesn’t exist. Starting to notice a few things aren’t right with this book and the ver 3.X ver:
# puppet config print ssldir /var/lib/puppet/ssl # ll /var/lib/puppet/ssl total 36 drwxrwx--x. 8 puppet puppet 4096 Apr 1 20:58 ./ drwxr-x---. 11 puppet puppet 4096 Apr 1 20:58 ../ drwxr-xr-x. 5 puppet puppet 4096 Apr 1 20:58 ca/ drwxr-xr-x. 2 puppet puppet 4096 Apr 1 20:58 certificate_requests/ drwxr-xr-x. 2 puppet puppet 4096 Apr 1 20:58 certs/ -rw-r--r--. 1 puppet puppet 975 Apr 1 20:58 crl.pem drwxr-x---. 2 puppet puppet 4096 Apr 1 20:58 private/ drwxr-x---. 2 puppet puppet 4096 Apr 1 20:58 private_keys/ drwxr-xr-x. 2 puppet puppet 4096 Apr 1 20:58 public_keys/
04/07/15:
Need to do better on the studying…
Getting the hosts ready
First connection, as advertised:
# puppet agent –server=pm.olearycomputers.com –no-daemonize –verbose Info: Creating a new SSL key for node1.olearycomputers.com Info: Caching certificate for ca Info: csr_attributes file loading from /etc/puppet/csr_attributes.yaml Info: Creating a new SSL certificate request for node1.olearycomputers.com Info: Certificate Request fingerprint (SHA256): CC:A2:AE:22:4A:32:54:4F:F2:D1:C9:0D:40:E9:A9:09:E0:47:F5:D2:17:38:4D:71:04:4D:F6:20:99:67:51:87 Info: Caching certificate for ca
puppet master:
# h pm # puppet cert --list "node1.olearycomputers.com" (SHA256) CC:A2:AE:22:4A:32:54:4F:F2:D1:C9:0D:40:E9:A9:09:E0:47:F5:D2:17:38:4D:71:04:4D:F6:20:99:67:51:87 # puppet cert --sign node1.olearycomputers.com Notice: Signed certificate request for node1.olearycomputers.com Notice: Removing file Puppet::SSL::CertificateRequest node1.olearycomputers.com at '/var/lib/puppet/ssl/ca/requests/node1.olearycomputers.com.pem'
On target:
[[after signing]] Info: Caching certificate for node1.olearycomputers.com Notice: Starting Puppet client version 3.7.4 Info: Caching certificate_revocation_list for ca Info: Retrieving pluginfacts Info: Retrieving plugin Info: Caching catalog for node1.olearycomputers.com Info: Applying configuration version '1428452868' Info: Creating state file /var/lib/puppet/state/state.yaml Notice: Finished catalog run in 0.04 seconds
Interesting, on puppetmaster, can import nodes via:
import 'nodes/*' import 'classes/*'
Fuck me, that took longer than it should have. My first puppet module and I fucked it up… and took almost an hour to find the fucker. What’s wrong with this picture?
class sudo { package { sudo: ensure => present, } if $operatingsystem == 'Ubuntu' { package { "sudo-ldap": ensure => present, require => Package["sudo"], } } file { "/etc/sudoers": owner => 'root', group => 'root', mode => 0440, source => "puppet:///modules/sudo/etc/sudoers", require => Package["sudo"], } }When run on node1, I was getting:
# puppet agent --server=pm.olearycomputers.com --no-daemonize --verbose --onetime Info: Retrieving pluginfacts Info: Retrieving plugin Info: Caching catalog for node1.olearycomputers.com Warning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/lib/ruby/site_ruby/1.8/puppet/type/package.rb:430:in `default') Info: Applying configuration version '1428454886' Error: /Stage[main]/Sudo/File[/etc/sudoers]: Could not evaluate: Could not retrieve information from environment production source(s) puppet://pm.olearycomputers.com/modules/sudo/etc/sudoers
WTF, over? Finally, as I’m typing up a question on server fault, I finally figured it out…
source => "puppet:///modules/sudo/etc/sudoers",
It’s not in etc you slack-jawed numb nut!
source => "puppet:///modules/sudo/sudoers",
works just fine… fucktard!
Checked the book again. Noticed they had etc in the file resource; but, they actually made modules/sudo/files/etc and copied sudoers into it. Not a bad idea, actually.
Good place to stop; heading into chapter 2.
04/08/15:¶
Chapter 2:¶
node definitions:
node ‘${node1}’, ‘${node2}’, ‘${node3}’ { … }
node /^webd+.example.com$/ {..} # perl regex
external node classifier
node inheritance:
Book gives example. From class, though:
Only use with a parameters subclass.
Reason to inherit vs include is to be able to override class parameters. Syntax is as follows:
class apache ( $httpd_user = $apache::params::httpd_user, $httpd_group = $apache::params::httpd_group, $httpd_pkg = $apache::params::httpd_pkg, $httpd_svc = $apache::params::httpd_svc, $httpd_conf = $apache::params::httpd_conf, $httpd_confdir = $apache::params::httpd_confdir, $httpd_docroot = $apache::params::httpd_docroot, ) inherits apache::params { [[rest-o-class-def-snipped]]Class ‘inherits’ reasonable defaults from $apache::params yet can override them using class resource declaratios as described in previous section.
Variable scoping. Bet that’s going to bite me in the ass someday. For the most part, fairly comfortable with the concept.
Got up to the params class. all four systems are part of the puppet env now.
# puppet cert list --all + "node1.olearycomputers.com" (SHA256) 88:9B:03:37:E9:05:D4:8F:63:4B:F2:0D:3F:60:5B:B3:9D:D5:3E:7B:2C:1F:0C:D4:2A:B6:97:DC:97:9A:45:07 + "pm.olearycomputers.com" (SHA256) F3:8B:D6:CF:8B:58:B7:3E:66:76:79:D3:99:F0:03:D3:45:C9:87:2F:30:57:FB:B1:39:C0:48:6C:9A:0D:F9:0D (alt names: "DNS:pm.olearycomputers.com", "DNS:puppet", "DNS:puppet.olearycomputers.com") + "ubuntu.olearycomputers.com" (SHA256) C9:4B:60:C6:A1:3E:8F:20:2A:51:45:50:17:B2:D3:05:60:AC:52:10:12:A7:86:AA:04:37:2C:25:CC:4B:33:F1 + "xymon.olearycomputers.com" (SHA256) DD:1E:66:48:11:81:7A:2C:B2:A4:08:62:80:BE:9A:E3:C9:05:91:09:10:05:AC:39:D0:B3:A1:8E:27:F1:42:DC
Tired, time to go to bed.
04/12/15:¶
Been noticing many differences betwee the puppet I installed and the one the book was talking about. Looked on amazon and found an update to the book. Bit pricy, but I bought it. Starting over. Lessons learned, et al from above will remain.
Bit different first chapter. More heavily into the configuration.
Update puppet.conf to include
server=puppet.oci.comin the main section.Shows how to update iptables for puppet:
iptables -A INPUT -p tcp -m state --state NEW --dport 8140 -j ACCEPT
or, better yet, restrict to IPs that are in use only:
iptables -A INPUT -p tcp -m state --state NEW \ --dport 8140 -s 192.168.0.0/24 -j ACCEPT
That
server=pm.oci.comhas to be everywhere… just tried runningpuppet agent --teston node1 and it choked with all sorts of ugly errors. Once server line was added to /etc/puppet/puppet.conf, everything worked well.Book’s for ver 3.2; I’m running 3.7. Seems there’s been some changes since the book was written. They’re talking about having to install puppet agent on the puppet master. I didn’t do that on pm and it’s already there.
DSL notes:
Parameterized classes:
Simple version:
node /node1/ { include ::sudo {
Parameterized version:
node /node2 { class { '::sudo': users => [ 'tom', 'jerry' ], } }selector - puppet trinary, but more. Case insensitive:
$packge_name = $::osfamily ? { 'RedHat' => "openssh-server", 'Debian' => "openssh-server", 'Solaris' => "openssh", }Selector requirements:
All selectors and case statements must have a default value. Note the example above does not.
Defaults should call fail() if a default behavior is questionable.
defautl: {} is acceptable
Scope:
::${class}indiccates a top level class. Possibly similar in paradigm to adding {} around var names.Facter vars are top scope.
Available scopes:
top - facter, or site.pp vars
node: vars in a node definition
parent: vars from an explicity inherited class
local: sub class vars
Misc:
==operator is case insensitive. To do case sensitive searches, need a perl syntax.$osfamily =~ /Debian$/$::serveris the hostname of the puppet master.Seemngly backed away from the
import nodes.ppin site.pp.
Node classification:
straight:
node [...}multiple nodes:
node 'node1.oci.com', 'node2.oci.com', 'node3.oci.com' {...}``regex:
node /^node[1-3]\.oci\.com$/ {...}ENC: discussed in chapter 5
Default node: particularly useful when auto-signing certs.
Stopping at page 44 right before Creating a module to manage ssh
04/14/15:¶
WTF with this syntax? it’s not explained anywhere but passes the
puppet parser validateApparently, some type of short hand forincludeclass ssh { class { '::ssh::package': } -> class { '::ssh::config': } -> class { '::ssh::service': } -> Class['ssh'] }Notes on DSL stuff in the section above
Short night; stopping right before params class on page 48. woohoo, 4 whole pages.
05/24/15:¶
Getting ready to hop back into puppet. Been gone for a bit getting the cmdb ready at work. That’s mostly done so now I can concentrate on this. I briefly contemplated rebuilding everything, but decided against it. I haven’t gone so far as to make that necessary. I ame taking a step back and starting chapter 2 completely over again. I’ll start in on that later tonight or tomorrow.
05/25/15:¶
Variable scoping:
Puppet DLS is a declarative language therefore order doesn’t matter. As such, vars can’t be reassigned once set as there would be no predictable value for the var.
Scopes:
top
Anything defined in facter, sites.pp or imported manifests
Explicity accessed by prepending ‘::’ to var.
node
vars created within the brackets of a node definition
No explicit accessing.
local
Single class or defined type
No explicit accessing.
Has access to node scope vars unless overridden
parent
Class scope that is specifically inherited through the inherits keyword.
NOTE: This is not node inheritance. Class inheritance is possibly a good thing.
Class inheritance vs include statement:
Examples:
include (old style):
class ssh::params { case $::osfamily { 'Debian': { $sshd_package = 'ssh' } 'RedHat': { $sshd_package = 'openssh-server' } default: {fail(" Login class does not work on osfamily: ${:: osfamily}")} } } class ssh { include ssh::params package { $::ssh::params::sshd_package: ensure = > installed, } } include sshparameterized:
class ssh::params { case $::osfamily { 'Debian': { $ sshd_package = 'ssh' } 'RedHat': { $ sshd_package = 'openssh-server' } default: {fail(" Login class does not work on osfamily: ${::osfamily}")} } } class ssh inherits ssh::params { package { $::ssh::params::sshd_package: ensure = > installed, } } include ssh
On the surface, definitions don’t seem that much different; however, paraemterized classes allow variables to be overridden.
Class definiont ‘->’ WTF??
Example:
class ssh { class { '::ssh::package': } -> class { '::ssh::config': } -> class { '::ssh::service': } -> Class ['ssh'] }Based on this line in the text, I believe this is an alternate form of the include function; however, I can’t seem to find anything to confirm or refute that:
In listing 2-13, we created a funcitnoal structure by dividing the components of the serve we're managing info functional domains: packages to be installed, files to be configured, and service to be executed. We created a class called ssh (which we need to ensure that the module is valid) and used the include function to all all the classes to the module.
Finally found something. It’s puppet’s chaining syntax. It’s setting up an order of the classes in ::ssh, first package, then config, followed by service. WTF is that for???
Templates:
<%= … %> Ruby variable
Facter facts listed with ‘@’ rather than ‘$’
Directories:
W/o a source parameter, a
recurse => truestatement can be used to force a user/group ownership on all files there in….
Parameterized classes:
need to review the notes from class.
that being said, syntax very similar to a class definition inside of another class or node definition is another form of
includeinit.pp:
class mysql ( $user = 'mysql', $group = 'mysql', $service_enabled = true, $service_running = true ) { class { 'mysql::install': user => $user, group => $group, } class { 'mysql::config': user => $user, group => $group, } class { 'mysql::service': ensure => $service_running, enabled => $service_enabled, } }Node definitions using the same class with different parameters:
node 'prod.example.com' { include base include ssh class { 'mysql': user => 'mysql', service_running => true, service_enabled => true, } } node 'qa.example.com' { include base include ssh class { 'mysql': user => 'qamysql', service_running => true, service_enabled => true, } } node 'dev.example.com' { include base include ssh class { 'mysql': user => 'devmysql', service_running => false, service_enabled => false, } }
Audit mode:
Can use puppet as a rich man’s HIDS.
file { $file: audit => [ owner, group, mode ], }
Stopping at the apache stage.
05/30/15:¶
Definitions:
Similar to classes but can be specified and evaluated multiple times.
Definitions go in module files. Using the example below, the definition would be in vhost.pp.
Instantiation goes in the nodes.pp file
Definition syntax:
define apache::vhost ( $docroot, $port, $priority, $ssh=true, $serveralises = '', $template='apache/vhost.conf.erb', ) { include apapche file { "/etc/apache2/sites-enabled/${priority}-${name}": content => template($template), owner => 'root', group => 'group', mode => '0640', require => Class['apache::install'], notify => Class['apache::service'], } }Instantiation syntax:
apache::vhost { 'www.example.com': port => '80', docroot => '/var/www/www.example.com', ssl => false, priority => '10', serveraliase => 'home.example.com', }
Chapter 3: Developing and deploying puppet:¶
05/31/15: Started with some studying as documented below, then went on to rebuild the environment. My puppet git govno was starting to be quite different then the one in the book.
Some interesting tidbits about running pupppet apply, differences between notice and notify, and how arrays are handled. All ended up in the lessons learned section.
Typical layout for /etc/puppet on puppet master - starting at /etc/puppet:
# tree environments/ environments/ ├── dev │ ├── manifests │ │ └── site.pp │ └── modules │ ├── apache │ ├── collectd │ ├── concat │ ├── nrpe │ ├── postfix │ ├── stdlib │ └── sudo ├── prod │ ├── manifests │ │ └── site.pp │ └── modules │ ├── apache │ ├── collectd │ ├── concat │ ├── nrpe │ ├── postfix │ ├── stdlib │ └── sudo └── staging ├── manifests │ └── site.pp └── modules ├── apache ├── collectd ├── concat ├── nrpe ├── postfix ├── stdlib └── sudoBook goes into some depth on vagrant, a method of automating vm builds for vmware or virtualbox. might work with kvm as well. need to look into this some other time.
Environments:
module maintenance models:
Each module is its own repository
Single repository under modules
Tools for external modules: Covered in chapter 8 (this “we’ll cover that later” thing is getting old)
puppet-librarian
r10k
Configuration:
Add support for environments in pm:/etc/puppet.conf: