=========== Pulp notes: =========== :Title: Pulp patching too notes :Author: Douglas O'Leary :Description: Information that's different from the admittedly very good installation docs. :Date created: 12/13/16 :Date updated: 12/21/17: updated for rhel7 :Disclaimer: Standard: Use the information that follows at your own risk. If you screw up a system, don't blame it on me... .. contents:: Overview: ========= This is ver 2 of the docs. First one went into the installlation in some depth and basically rewrote the very good docs available at http://docs.pulpproject.org/user-guide/installation/f23-.html. This one's not going to do that. I'll follow the docs and make notes about where I had to deviate from them and why. I'll leave the pulp people to keep their docs up to date because they do a whole lot better job of it than I do. urls: ===== Pulp rhel installation url: http://docs.pulpproject.org/user-guide/installation/f23-.html rhel6 qpid repo - required for consumer install: https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo rhel6 qpid repo - a different one. maybe better luck? https://copr-be.cloud.fedoraproject.org/results/@qpid/qpid/epel-6-x86_64 rhel6 pulp repo: https://repos.fedorapeople.org/repos/pulp/pulp/rhel6-pulp.repo rhel7 pulp repo: https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo epel for el7 repo: http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm General Users guide: http://docs.pulpproject.org/user-guide/index.html RPM users guide: http://docs.pulpproject.org/plugins/pulp_rpm/user-guide/index.html rhel-pulp.repo: https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo copr: https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo Seems to work for both el6 and 7. Admin client docs: http://docs.pulpproject.org/user-guide/admin-client/index.html Commands: ========= pulp-admin --map: Displays groups of valid command line arguments for pulp-admin. 354 lines long... pulp-admin ${group} [ ${subcommadn} ] -help: Displays help for a specific subcategory of pulp-admin commands. pulp-admin [login -u ${u} [ -p ${p} ]] | [ logout ]: Creates or deletes a pulp session. Required to execute pulp-admin commands but can be avoided by use of ~/.pulp/admin.conf pulp-admin auth user create --login dkoleary --password '${pwd}' Creates a user. Docs suggest not using ``--password ${pwd}`` option. # pulp-admin auth role user [add|reomve] --role-id super-users --login dkoleary Adds/removes dkoleary to super-users role Lessons learned: ================ * repoview cannot handle utf8 code in package metadata which results in repo publishing failures and, eventually, failed patching work. Details are available at https://pulp.plan.io/issues/2346 with an as yet globally unavailable patch to repoview available at https://pulp.plan.io/attachments/320. I've tested the patch and it seems to work with no identifiable side effects yet. Edit the /usr/bin/repoview file (after backing it up) adding the lines in the patch. * Required firewall ports: * 80 (http) * 443 (https) * 5671 * 5672 * (el6 only) Need to install the copr repo in order to install the pulp server otherwise, you'll run into dependency issues:: cd /etc/yum.repos.d wget https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo * Ensure host uses fqdn for hostname before installing pulp. ssl certs will become confused if not done. * Consumer script installation errors as defined below errors out if puppet is not installed. Either install it or remove pulp-puppet-handlers from the cli. * Disable ssl: not a good idea, but for testing, set ``verify_ssl: False`` in the following: * /etc/pulp/admin/admin.conf * /etc/pulp/consumer/consumer.conf * /etc/pulp/repo_auth.conf * /etc/pulp/nodes.conf # note: didn't exist in my installation. * Default user/pwd: admin/admin Can be changed in /etc/pulp/server.conf in the [server] section:: [server] default_login: Bog default_password: 5WHc69wHDWDaMSq * Creation of ~/.pulp/admin.conf with syntax below allows execution of pulp-admin commands w/o havinng to execute ``pulp-admin login``:: # cat ~/.pulp/admin.conf [auth] username: admin password: admin * Consumer registration must happen from consumer side. * To restart pulp services: * service httpd restart * service pulp_workers restart * service pulp_celerybeat restart * service pulp_resource_manager restart * If task is stuck in waiting and shows 'unstarted' as the 'start time', ``pulp-admin -vv repo task details --task-id=${long_hex}``, restart services, particularly pulp_workers:: # service pulp_workers restart celery init v10.0. Using config script: /etc/default/pulp_workers celery multi v3.1.11 (Cipater) > reserved_resource_worker-0@pulp.olearycomputers.com: DOWN > reserved_resource_worker-1@pulp.olearycomputers.com: DOWN > reserved_resource_worker-2@pulp.olearycomputers.com: DOWN > reserved_resource_worker-3@pulp.olearycomputers.com: DOWN celery multi v3.1.11 (Cipater) > Starting nodes... > reserved_resource_worker-0@pulp.olearycomputers.com: OK > reserved_resource_worker-1@pulp.olearycomputers.com: OK > reserved_resource_worker-2@pulp.olearycomputers.com: OK > reserved_resource_worker-3@pulp.olearycomputers.com: OK Process: ======== 1. Install server 2. Install admin client on server 3. Install consumer client on server. 4. install consumer client on additional clients. Software Installation: ====================== Once again, following the directions at http://docs.pulpproject.org/user-guide/installation/f23-.html for rhel7. 1. Install repos on both clients and server: * rhel-pulp.repo (OS version specific) * epel (yum) ``yum install epel-release`` * qpid (el6) * Repos: * rhel7: https://repos.fedorapeople.org/repos/pulp/pulp/rhel-pulp.repo * epel: http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm 2. Create filesystems for /var/lib/mysql and /var/lib/mongodb:: # pvcreate /dev/vdb Physical volume "/dev/vdb" successfully created. # vgextend vg00 /dev/vdb Volume group "vg00" successfully extended # lvcreate -L 20g -n mongdb vg00 Logical volume "mongdb" created. # lvcreate -L 20g -n pulp vg00 Logical volume "pulp" created. # mkfs.xfs /dev/vg00/mongdb # mkfs.xfs /dev/vg00/pulp # vi /etc/fstab # mkdir -p -m 755 /var/lib/{mongodb,pulp} # mount /var/lib/mongodb # mount /var/lib/pulp 3. Install software per directions: * mongodb. Starting mongod will take some time. * qpid * pulp server * pulp extras 4. Services that must be enabled/started. Note: follow the directions. There are tasks between enabling/starting some of these processes. * mongodb * qpidd * httpd * pulp_workers * pulp_celerybeat * pulp_resource_manager 5. Install admin client. Doesn't have to be server or consumer. * Install required repos if not already done: * epel (yum) * rhel-pulp.repo (wget) * Install packages:: yum install pulp-admin-client \ pulp-rpm-admin-extensions \ pulp-puppet-admin-extensions \ pulp-docker-admin-extensions * Update hostnane in /etc/pulp/admin/admin.conf:: [server] host: pulp.olearycomputers.com 6. Install pulp consumer on all servers that will be consumers: * Install required repos if not already done: * rhel-pulp.repo (OS version specific) * epel (yum) ``yum install epel-release`` * qpid (el6) * Install packages:: yum install pulp-consumer-client \ pulp-rpm-consumer-extensions \ pulp-puppet-consumer-extensions \ pulp-agent pulp-rpm-handlers \ pulp-rpm-yumplugins \ pulp-puppet-handlers \ python-gofer-qpid * Update hostname in /etc/pulp/consumer/consumer.conf * Enable/start the service: chkconfig goferd on service goferd start 7. In my little lab environment, disable ssl checking by adding ``verify_ssl: False`` to server section of /etc/pulp/admin/admin.conf 8. Create an administrative account and enable it:: # pulp-admin login -u ${root} -p ${root_pwd} # pulp-admin auth user create --login dkoleary --password 'no-me-pwd' # pulp-admin auth role user add --role-id super-users --login dkoleary # mkdir -p -m 700 ~/.pulp # vi ~/.pulp/admin.conf # cat ~/.pulp/admin.conf # cat admin.conf [auth] username: dkoleary password: no-me-pwd [server] host: pulp.olearycomputers.com verify_ssl: False # pulp-admin logout # chmod 600 ~/.pulp/admin.conf # pulp-admin auth user list 9. Open required firewall ports on the pulp server 10. Create and sync repos. Admin Client: ============= * LL on ~/.pulp/admin.conf * Users: * Create users, roles, etc. Examples in command section. * Permissions. Create, read, update, delete, execute. Can be set on individual repos. Potentially useful. * Roles: * Default role: super-users * Rather than provide permissions to specific accounts, create roles which have permissions set, then add/delete users as needed. * Some users need to be in super-users to ensure maintainability: :: # pulp-admin auth role user add --role-id super-users --login dkoleary User [dkoleary] successfully added to role [super-users] * Groups: This will be the interesting one at work. Some interesting possiblilities just based on the reading:: pulp-admin rpm consumer group package update \ --name tzdata --consumer-group dev pulp-admin rpm consumer group update \ --consumer-group qa * Repos: * rpm repo create doesn't link a url with the pulp repo. Still need to find out how to do that. * I created centos6_base and centos6_updates for my testing. * I can group them together Creating repo mirrors: ====================== Using epel as the example: * Create the pulp repo:: pulp-admin rpm repo create \ --description 'Extra Packages for Enterprise Linux 7 - x86_64' \ --display-name 'Extra Packages for Enterprise Linux 7 - x86_64' \ --feed http://download.fedoraproject.org/pub/epel/7/x86_64 \ --relative-url=epel_el7_x86_64 \ --repo-id=epel_el7_x86_64 \ --remove-missing=true \ --retain-old-count=2 \ --repoview=true * Run the initial sync:: # pulp-admin rpm repo sync run --repo-id epel_el7_x86_64 * Publish the repo:: # pulp-admin rpm repo publish run --repo-id epel_el7_x86_64 * Set a schedule for resyncing:: # Sched=$(date -d "2AM tomorrow" +"%FT%TZ-600/P1DT") # echo ${Sched} 2017-12-22T02:00:00-600/P1D # pulp-admin rpm repo sync schedules create \ --schedule ${Sched} --repo-id epel_el7_x86_64 Follow the same process for any other repo. Watch space utilization in /var/lib/pulp. Epel alone took upwards of 13 gigs. A good place to start for CentOS 7:: #---------------------------------- name='CentOS-7 (x86_64) - Base' baseurl=http://mirror.centos.org/centos/7/os/x86_64/ #---------------------------------- name='CentOS-7 (x86_64) - Updates' baseurl=http://mirror.centos.org/centos/7/updates/x86_64/ #---------------------------------- name='CentOS-7 (x86_64) - Extras' baseurl=http://mirror.centos.org/centos/7/extras/x86_64/ #---------------------------------- name='CentOS-7 (x86_64) - Plus' baseurl=http://mirror.centos.org/centos/7/centosplus/x86_64/ and for CentOS 6:: #---------------------------------- name='CentOS-6 (x86_64) - Base' baseurl=http://mirror.centos.org/centos/6/os/x86_64/ #---------------------------------- name='CentOS-6 (x86_64) - Updates' baseurl=http://mirror.centos.org/centos/6/updates/x86_64/ #---------------------------------- name='CentOS-6 (x86_64) - Extras' baseurl=http://mirror.centos.org/centos/6/extras/x86_64/ #---------------------------------- name='CentOS-6 (x86_64) - Plus' baseurl=http://mirror.centos.org/centos/6/centosplus/x86_64/ #---------------------------------- name='CentOS-6 (x86_64) - Contrib' baseurl=http://mirror.centos.org/centos/6/contrib/x86_64/ Repositories: ============== * Add --feed=${url} to link a pulp repo to an external one. * Process: * Create the repo * Sync the repo * register consumers * Bind repos: