Pulp notes:¶

Title:: Pulp patching too notes
Author:: Douglas O’Leary <dkoleary@olearycomputers.com>
Description:: Information that’s different from the admittedly very good installation docs.
Date created:: 12/13/16
Date updated:: 12/21/17: updated for rhel7
Disclaimer:: Standard: Use the information that follows at your own risk. If you screw up a system, don’t blame it on me…

Overview:¶

This is ver 2 of the docs. First one went into the installlation in some depth and basically rewrote the very good docs available at http://docs.pulpproject.org/user-guide/installation/f23-.html. This one’s not going to do that. I’ll follow the docs and make notes about where I had to deviate from them and why. I’ll leave the pulp people to keep their docs up to date because they do a whole lot better job of it than I do.

urls:¶

Pulp rhel installation url:: http://docs.pulpproject.org/user-guide/installation/f23-.html
rhel6 qpid repo - required for consumer install:: https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo
rhel6 qpid repo - a different one. maybe better luck?: https://copr-be.cloud.fedoraproject.org/results/@qpid/qpid/epel-6-x86_64
rhel6 pulp repo:: https://repos.fedorapeople.org/repos/pulp/pulp/rhel6-pulp.repo
rhel7 pulp repo:: https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo
epel for el7 repo:: http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
General Users guide:: http://docs.pulpproject.org/user-guide/index.html
RPM users guide:: http://docs.pulpproject.org/plugins/pulp_rpm/user-guide/index.html
rhel-pulp.repo:: https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo
copr:: https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo Seems to work for both el6 and 7.
Admin client docs:: http://docs.pulpproject.org/user-guide/admin-client/index.html

Commands:¶

pulp-admin –map:: Displays groups of valid command line arguments for pulp-admin. 354 lines long…
pulp-admin ${group} [ ${subcommadn} ] -help:: Displays help for a specific subcategory of pulp-admin commands.
pulp-admin [login -u ${u} [ -p ${p} ]] | [ logout ]:: Creates or deletes a pulp session. Required to execute pulp-admin commands but can be avoided by use of ~/.pulp/admin.conf
pulp-admin auth user create –login dkoleary –password ‘${pwd}’: Creates a user. Docs suggest not using --password ${pwd} option.
# pulp-admin auth role user [add|reomve] –role-id super-users –login dkoleary: Adds/removes dkoleary to super-users role

Lessons learned:¶

repoview cannot handle utf8 code in package metadata which results in repo publishing failures and, eventually, failed patching work. Details are available at https://pulp.plan.io/issues/2346 with an as yet globally unavailable patch to repoview available at https://pulp.plan.io/attachments/320. I’ve tested the patch and it seems to work with no identifiable side effects yet. Edit the /usr/bin/repoview file (after backing it up) adding the lines in the patch.
Required firewall ports:
- 80 (http)
- 443 (https)
- 5671
- 5672
(el6 only) Need to install the copr repo in order to install the pulp server otherwise, you’ll run into dependency issues:
```
cd /etc/yum.repos.d
wget https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo
```
Ensure host uses fqdn for hostname before installing pulp. ssl certs will become confused if not done.
Consumer script installation errors as defined below errors out if puppet is not installed. Either install it or remove pulp-puppet-handlers from the cli.
Disable ssl: not a good idea, but for testing, set verify_ssl: False in the following:
- /etc/pulp/admin/admin.conf
- /etc/pulp/consumer/consumer.conf
- /etc/pulp/repo_auth.conf
- /etc/pulp/nodes.conf # note: didn’t exist in my installation.
Default user/pwd: admin/admin Can be changed in /etc/pulp/server.conf in the [server] section:
```
[server]
default_login: Bog
default_password: 5WHc69wHDWDaMSq
```
Creation of ~/.pulp/admin.conf with syntax below allows execution of pulp-admin commands w/o havinng to execute pulp-admin login:
```
# cat ~/.pulp/admin.conf
[auth]
username: admin
password: admin
```
Consumer registration must happen from consumer side.
To restart pulp services:
- service httpd restart
- service pulp_workers restart
- service pulp_celerybeat restart
- service pulp_resource_manager restart

If task is stuck in waiting and shows ‘unstarted’ as the ‘start time’, pulp-admin -vv repo task details --task-id=${long_hex}, restart services, particularly pulp_workers:

# service pulp_workers restart
celery init v10.0.
Using config script: /etc/default/pulp_workers
celery multi v3.1.11 (Cipater)
> reserved_resource_worker-0@pulp.olearycomputers.com: DOWN
> reserved_resource_worker-1@pulp.olearycomputers.com: DOWN
> reserved_resource_worker-2@pulp.olearycomputers.com: DOWN
> reserved_resource_worker-3@pulp.olearycomputers.com: DOWN
celery multi v3.1.11 (Cipater)
> Starting nodes...
        > reserved_resource_worker-0@pulp.olearycomputers.com: OK
        > reserved_resource_worker-1@pulp.olearycomputers.com: OK
        > reserved_resource_worker-2@pulp.olearycomputers.com: OK
        > reserved_resource_worker-3@pulp.olearycomputers.com: OK

Process:¶

Install server
Install admin client on server
Install consumer client on server.
install consumer client on additional clients.

Software Installation:¶

Once again, following the directions at http://docs.pulpproject.org/user-guide/installation/f23-.html for rhel7.

Install repos on both clients and server:
- rhel-pulp.repo (OS version specific)
- epel (yum) yum install epel-release
- qpid (el6)
- Repos:
  - rhel7: https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo
  - epel: http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Create filesystems for /var/lib/mysql and /var/lib/mongodb:

# pvcreate /dev/vdb
  Physical volume "/dev/vdb" successfully created.
# vgextend vg00 /dev/vdb
  Volume group "vg00" successfully extended
# lvcreate -L 20g -n mongdb vg00
  Logical volume "mongdb" created.
# lvcreate -L 20g -n pulp vg00
  Logical volume "pulp" created.
# mkfs.xfs /dev/vg00/mongdb
# mkfs.xfs /dev/vg00/pulp
# vi /etc/fstab
# mkdir -p -m 755 /var/lib/{mongodb,pulp}
# mount /var/lib/mongodb
# mount /var/lib/pulp

Install software per directions:
- mongodb. Starting mongod will take some time.
- qpid
- pulp server
- pulp extras
Services that must be enabled/started. Note: follow the directions. There are tasks between enabling/starting some of these processes.
- mongodb
- qpidd
- httpd
- pulp_workers
- pulp_celerybeat
- pulp_resource_manager
Install admin client. Doesn’t have to be server or consumer.
- Install required repos if not already done:
  - epel (yum)
  - rhel-pulp.repo (wget)
- Install packages:
```
yum install pulp-admin-client \
    pulp-rpm-admin-extensions \
    pulp-puppet-admin-extensions \
    pulp-docker-admin-extensions
```
- Update hostnane in /etc/pulp/admin/admin.conf:
```
[server]
host: pulp.olearycomputers.com
```
Install pulp consumer on all servers that will be consumers:
- Install required repos if not already done:
  - rhel-pulp.repo (OS version specific)
  - epel (yum) yum install epel-release
  - qpid (el6)
- Install packages:
```
yum install pulp-consumer-client \
    pulp-rpm-consumer-extensions \
    pulp-puppet-consumer-extensions \
    pulp-agent pulp-rpm-handlers \
    pulp-rpm-yumplugins \
    pulp-puppet-handlers \
    python-gofer-qpid
```
- Update hostname in /etc/pulp/consumer/consumer.conf
- Enable/start the service:
  
  chkconfig goferd on service goferd start
In my little lab environment, disable ssl checking by adding verify_ssl: False to server section of /etc/pulp/admin/admin.conf

Create an administrative account and enable it:

# pulp-admin login -u ${root} -p ${root_pwd}
# pulp-admin auth user create  --login dkoleary --password 'no-me-pwd'
# pulp-admin auth role user add --role-id super-users --login dkoleary
# mkdir -p -m 700 ~/.pulp
# vi ~/.pulp/admin.conf
# cat ~/.pulp/admin.conf
# cat admin.conf
[auth]
username: dkoleary
password: no-me-pwd

[server]
host: pulp.olearycomputers.com
verify_ssl: False
# pulp-admin logout
# chmod 600 ~/.pulp/admin.conf
# pulp-admin auth user list

Open required firewall ports on the pulp server
Create and sync repos.

Admin Client:¶

LL on ~/.pulp/admin.conf
Users:
- Create users, roles, etc. Examples in command section.
- Permissions. Create, read, update, delete, execute. Can be set on individual repos. Potentially useful.
- Roles:
  - Default role: super-users
  - Rather than provide permissions to specific accounts, create roles which have permissions set, then add/delete users as needed.
  - Some users need to be in super-users to ensure maintainability:

# pulp-admin auth role user add --role-id super-users --login dkoleary
User [dkoleary] successfully added to role [super-users]

  *   Groups:  This will be the interesting one at work.  Some interesting
      possiblilities just based on the reading::

          pulp-admin rpm consumer group package update \
              --name tzdata --consumer-group dev
          pulp-admin rpm consumer group update \
              --consumer-group qa

Repos:
- rpm repo create doesn’t link a url with the pulp repo. Still need to find out how to do that.
- I created centos6_base and centos6_updates for my testing.
- I can group them together

Creating repo mirrors:¶

Using epel as the example:

Create the pulp repo:

pulp-admin rpm repo create \
--description 'Extra Packages for Enterprise Linux 7 - x86_64' \
--display-name 'Extra Packages for Enterprise Linux 7 - x86_64' \
--feed http://download.fedoraproject.org/pub/epel/7/x86_64 \
--relative-url=epel_el7_x86_64 \
--repo-id=epel_el7_x86_64 \
--remove-missing=true \
--retain-old-count=2 \
--repoview=true

Run the initial sync:

# pulp-admin rpm repo sync run --repo-id epel_el7_x86_64

Publish the repo:

# pulp-admin rpm repo publish run --repo-id epel_el7_x86_64

Set a schedule for resyncing:

# Sched=$(date -d "2AM tomorrow" +"%FT%TZ-600/P1DT")
# echo ${Sched}
2017-12-22T02:00:00-600/P1D
# pulp-admin rpm repo sync schedules create \
--schedule ${Sched} --repo-id epel_el7_x86_64

Follow the same process for any other repo. Watch space utilization in /var/lib/pulp. Epel alone took upwards of 13 gigs. A good place to start for CentOS 7:

#----------------------------------
name='CentOS-7 (x86_64) - Base'
baseurl=http://mirror.centos.org/centos/7/os/x86_64/
#----------------------------------
name='CentOS-7 (x86_64) - Updates'
baseurl=http://mirror.centos.org/centos/7/updates/x86_64/
#----------------------------------
name='CentOS-7 (x86_64) - Extras'
baseurl=http://mirror.centos.org/centos/7/extras/x86_64/
#----------------------------------
name='CentOS-7 (x86_64) - Plus'
baseurl=http://mirror.centos.org/centos/7/centosplus/x86_64/

and for CentOS 6:

#----------------------------------
name='CentOS-6 (x86_64) - Base'
baseurl=http://mirror.centos.org/centos/6/os/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Updates'
baseurl=http://mirror.centos.org/centos/6/updates/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Extras'
baseurl=http://mirror.centos.org/centos/6/extras/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Plus'
baseurl=http://mirror.centos.org/centos/6/centosplus/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Contrib'
baseurl=http://mirror.centos.org/centos/6/contrib/x86_64/

Repositories:¶

Add –feed=${url} to link a pulp repo to an external one.
Process:
- Create the repo
- Sync the repo
- register consumers
- Bind repos: