Douglas K. O'Leary
Objective: Senior System Administrator; increase company's productivity through efficient management & professional network/system/database administration
Home   Phone   Cell   Fax
dkoleary@olearycomputers.com   630-904-6098   630-248-2749   866-843-0899

Hardware/OS Experience (14 yrs):

Vendor Hardware OS # of years
HP HP 9000/800 - various classes HPUX 9.04/5, 10.X, 11.X
8
SUN Enterprise 10,000/4500/5500, Sparcs, Ultras SUNOS 4.1.3, Solaris 2.5/6/7/8
7
SGI Challenge XL, Origin 2000, Indy workstations Irix 5.3/6.X
3
IBM RS6000 980, 990, R24, Power PCs AIX 3.25, 4.1, 4.3
5
EMC Symmetrix, Connectrix and Broccade switches N/A
7


Software/Utilities

Perl
ksh
expect
HTML
CGI
sqlplus
Oracle
MySQL
MC ServiceGuard
HP Openview Network Node Mgr
HP Glance Plus
HP Perfview
HP Measureware
HP Omniback II
SNMP
SMTP/sendmail
Veritas Volume Manager
Veritas Netbackup
EMC Symmgr
EMC Timefinder
EMC SRDF
EMC Powerpath
EMC SDR
EMC Control Center
EMC Volume Logix
EMC Connectrix Manager
Openssh
OpenLDAP
 
F-Secure SSH
Snort (NIDS)  


Certifications
AIX Ver 4.3 System Administration
HP Openview Certified Consultant (NNM ver 6.X)
HP Star Certified Technical Professional (HP-UX 11.0)
SUN Certified System Administrator (Solaris 7 & 8)
SUN Certified Network Administrator (Solaris 8)

Work Experience:

Company O'Leary Computer Enterprises
Title Senior Unix Admin
Time Frame 08/01 - present





  1. Pepsi Bottling:
    1. Closed all nine audit gaps identified via third party security audit. Gaps dealt primarily with account and password management
    2. Conducted a more thorough security audit on HP UNIX systems resulting in the identification of 31 additional security issues, 18 of which would have allowed system compromise.
    3. Closed all critical security gaps and most of the remaining.
    4. Installed, configured, and documented opennssh secure shell environment using public key authentication.
    5. Inventoried 86 root level cron scripts across 22 systems. Identified security issues ranging from insecure permissions and ownership through remsh/rcp usage on 29 of those scripts. Corrected security issues by moving scripts to secure location and/or converting remsh/rcp usage to ssh/scp.
    6. Redesigned/rewrote script that maintains Omniback backup definitions for all Oracle databases. Script obtains list of datafiles that each database uses, then compares, contrasts the list against the backup definition maintained by Omniback. If different, the script rebuilds the Omniback backup definition. Centralized script to one system and enabled easier maintenance through user-modifiable configuration file. Converted from remsh/scp to ssh/scp.
  2. Bank One:
    1. Security administrator for SAP/Peoplesoft Environments:
      1. Responsible for monthly ISS scan remediation for 50+ HP systems.
      2. Eliminated rhost implementation.
    2. Implemented secure shell
      1. Compiled/distributed installation
      2. Configured public/private key authentication
      3. Redesigned rsh enabled scripts to use ssh
      4. Designed method for using non-interactive ssh-enabled scripts
      5. Documented ssh environment and procedures for using it.
    3. Documented tape vaulting procedure for six cell Omniback backup environment.
    4. Documented complex Autosys environment.
    5. Scripts:
      1. Developed and implemented a set of perl scripts to monitor Omniback backups for progress and failures. Script uses a centralized database of backup schedules to determine when to initiate a backup monitor on each of six cell servers. Resulted in identifying failed backups up to 18 hours sooner than before script implementation.
      2. Rewrote/standardized operating system backup ksh scripts resuling in a more easily maintained OS backup/restore process.
      3. Developed and implemented a perl script that transfers up to 15 Oracle archive logs from production system to DR site simultaneously via secure copy (scp)
      4. Developed and implemented a web based account creation function using perl as the common gateway interface (cgi). Script uses ssh to create remote accounts, set initial password, and displays results via another web page. Reduces by at least 75% the time it took to create user accounts on multiple systems.
      5. Developed and implemented web based backup tracking system:
        1. Tracks system (fbackup) results for all HP systems.
        2. Tracks Omniback backups for all cell servers.
        3. Tracks tape pool related information:
          1. Total number of tapes in the pool
          2. Number of scratch tapes in the pool
          3. Number of tapes by status (good, fair, poor)
    6. Participated in Bank One Disaster Recovery test in 11/2002.
      1. Failed over & back HP superdome SAP Central Instance and V2250 application server.
      2. Failed over & back system password and trusted password databases.
      3. Failed over & back Omniback implementation
      4. Troubleshot issues with Oracle archive log corruption
      5. Aided DBAs by scripting required file changes (system names/Oracle instance identifiers, etc)
  3. Govenors State University:
    1. Troubleshot DNS issue preventing transmission/reception of email
    2. Upgraded Veritas Volume Manager
    3. Troubleshot complex NFS cross mounted environment
    4. Reconfigured sendmail and anonymous ftp services
    5. Upgraded E450 operating system from Solaris 2.6 to 2.8.
    6. Installed, configured, and documented openssh environment.
    7. Designed, tested, and configured OpenLDAP server with goal of replacing NIS for dial-in accounts.
    8. Installed and configured Snort, a network intrusion detection system (NIDS). Documented configuration, implementation, and procedures for maintaining the NIDS sensor
  4. Acxiom:
    1. Created and documented an MC/Serviceguard cluster protecting mission critical Oracle SAP implementation.
    2. Created, tested, and implemented an Oracle hot backup script with similar functionality to EMC Timefinder software. Script splits HP LVM mirrors, mounts the split volumes, backs them up, then resilvers the mirrors. Backup times reduced from 12 hours of Oracle outage to no customer impact.
    3. Created, tested, and implemented import/export ksh scripts to maintain LVM consistency between disparate cell nodes (N4000/K460). Script identifies which HDS XP256 LUN is in use and identifies the specific disk device driver on each system. It then updates the HP LVM data structures using HP LVM commands.
  5. Rockwell Automation/Volt Services:
    1. Developed/implemented perl scripts which enabled system users to change their passwords on over 200 UNIX systems. Reduced the time required for this necessary security protocol from several hours to half an hour.
    2. Developed/implemented perl scripts which enabled help desk personnel to assume responsibility for account resets on nonprivileged user accounts. Script verified account status and last login time in accordance with corporate security policy. Script maintained system integrity through use of sudo to limit privileged exposure and comparison of encrypted security strings
    3. Shared in responsibility for management of trouble ticket queue which saw everything from account creations through performance evaluations to system crash analyses.
  6. Sevenspace/Nuclio:
    1. Troubleshot modifications to previously written SNMP/perl script.
    2. Developed/implemented perl-based script which scans up to 1800 routes/switches in five minutes and sends HP Openview ITO messages when configurable parameters break user definable thresholds. Script uses an external table to identify the parameter to check, the formula to use to identify the threshold and the action to take when the threshold is broken.
  7. Allstate:
    1. Installed/configured HP9000/800/L2000 as production system failover and test box.
    2. Developed and implemented a system hot backup script using HP LVM Mirroring and Omniback 4.X backup software. Reduced system outage required for backups from 12 hours to 15 minutes.
  8. CSC/Hyatt:
    1. Upgrade/configured sendmail.
      1. Identified and documented email requirements including relaying, masquerading, direct mail hosts, exposed users, and address translations.
      2. Compiled, installed sendmail and configured using m4 macros.
      3. Result is a well documented, easily maintained and consistent email environment across all corporate systems.
  9. General Board of Pensions and Health Benefits for the United Methodist Church
    1. Project management for installation, configuration, and documentation of a 3-node HP MC/Serviceguard cluster
    2. Verified minimal hardware/software requirements, drafted project plan, coordinated schedule, and verified required preparation work was completed.
    3. Re-IP.ed, renamed three systems and used originals as multiplexed, relocatable IP addresses to circumvent hours of client reconfigurations.
    4. Led newly trained cluster admins through cluster and package initialization and configuration
    5. Developed and disseminated documentation covering project plan, timeline, and cluster configuration
  • Certified AIX 4.3 System Administrator
  • Documented and published Lessons Learned web site on Linux box using Apache web server. Receiving upwards of 5000 hits/month.
  • Installed, configured, and am running Oracle on the Linux box.
  • Developed a time tracking system using Apache server, perl/cgi, and Oracle database.
    Company Nuclio
    Title Senior Technical Consultant
    Time Frame 10/00 - 08/01





    1. Designed and implemented EMC environment for Network Attached Storage (NAS) project. Environment consisted of dual EMC 8300s with EMC SRDF for DR and EMC Timefinder for backups.
    2. Developed, coordinated, and executed project plan to migrate two 120+ gigabyte Oracle databases from older disk systems to newer ones with a simultaneous cold HPUX OS install/upgrade.
    3. Ran performance ananlysis on several systems for one of our clients. Identified memory and CPU shortfalls on two systems and that other systems weren't running into bottlenecks.
    4. Planned, installed, and configured Veritas Netbackup on a ten node network for a client.
      1. Ensured backup classes and schedules met requirements.
      2. Wrote scripts to verify that backups ran the previous night and were successful.
      3. Verified backups periodically.
      4. Migrated master server from older Ultra 5 to a newer Enterprise 250.
      5. Migrated tapes from older L9 library to a four tape autoloader.
    5. Identified and worked issues involving the Oracle database that run Nuclio's ITO and Remedy systems. Wrote, documented, and implemented a script to scan the Oracle alert logs and report on problems as well as alerts DBAs and sysadmins when critical tablespaces are getting full.
    6. Wrote replacement script to scan client's 140 routers. Previous script used ICMP to check on routers and critical interfaces; caused performance problems on both the client network and Nuclio's ITO database. Replacement script uses SNMP to get interface statistics directly from the router; reduces network traffic by at least 66%; decreases runtime 80% on average.

    Company Sysix Technologies
    Title Senior Technical Consultant
    Time Frame 08/99 - 10/00





    1. Certified HPUX technical consultant, 98% first time testing.
    2. Certified Solaris System Administrator.
    3. Billed out a minimum of 85% of the time.
    4. Developed, documented, and disseminated procedures for standard system peformance evaluations.
    5. Developed, documented, and disseminated procedures for standard UNIX system security evaluations.
    6. Developed, documented, and disseminated procedures for standard Veritas Netbackup installation/configurations
    7. Developed, documented, and disseminated procedures for standard EMC Timefinder implementations
    8. Client Accomplishments:
      1. Numerous performance evaluations. Examples:
        1. V2200: ID'ed I/O bottleneck caused by large sequential writes. Developed, coordinated, and implemented plan to circumvent. Archive log switch performance improved 329%.
        2. K370: ID'ed CPU bottleneck that was causing unacceptable performance degredations for a warehouse control system.
        3. K580: Verified that the system had no systemic bottlenecks and helped identify a buggy system monitoring package as the cause for periodic system "hangs".
      2. Planned breakup of NIS domain which was allowing all company personnel to log onto the primary and backup HR/payroll systems.
      3. Planned and implemented IP change on a geographically distant system.
      4. Repaired, documented, and/or updated system configuration:
        1. Removed badly misconfigured MC/ServiceGuard configuration, allowing Oracle data volume groups to be activated on boot.
        2. Repaired NFS mount configuration, allowing ERP applications to start automatically on system boot.
        3. ID'ed, ordered, and configured requisite hardware to increase system's fault tolerance.
        4. ID'ed and fixed numerous security weaknesses such as user modifiable scripts running via root's cron, system directory ownership/permissions, etc
      5. Scripts:
        1. Documented complex, poorly designed ksh scripts; cleaned up and debugged where necessary.
        2. Designed and developed web based dynamic system performance graphs.
          1. Graphed data included cpu, filesystem, memory, and Openmail capacity/performance information.
          2. Graphs were generated dynamically and automatically removed when appropriate.
          3. Graphs used for capacity planning and performance troubleshooting.
        3. System documentation script which formats and prints information on
          1. CPU
          2. Memory
          3. Disks
          4. Network configuration
          5. Printers
          6. Kernel
          7. etc
      6. Designed/developed policies/procedures for:
        1. Change control
        2. Security policies/procedures
        3. Backup procedures
        4. User access additions/deletions/modifications
        5. etc


    Company: Acxiom - May & Speh
    Title: Associate:  System Administrator
    Time frame: 1/96 through 8/99





    1. Overall management responsibility of UNIX support team which provides support for Inter/Intranet services, system performance monitoring, tuning and maintenance for two separate data warehousing functions supporting company's direct mail clients. Database products are Fastcount andOracle. Network services include DNS, sendmail, firewall, and www
    2. Developed, coordinated, and implemented site security policy, detailing policies for change management, user installation/modification/deletion, system configuration and maintenance, and quarterly system audits. Oversaw the development of the procedures required to support and implement the security policy.
    3. Team lead on installation, configuration, and support for SUN Enterprise 10,000, currently running with 28 CPUs, 24 gigs of RAM and approximately 2.5 terabytes of disk supporting 6 domains.  Support consists of routine system maintenance, performance monitoring and tuning, system troubleshooting and, when necessary, crash analysis.
    4. Migrated 4.3 terabytes of data from older SUN/EMC disk to newer EMC 3930 disk array.
      1. ID'ed storage requirements and coordinated disk system purchase.
      2. Designed and tested various migration strategies depending on system, resources, clients, and allowable outage window.
      3. Coordinated, scheduled, and oversaw/implemented data migration.
    5. Researched upgrade from SGI Challenge XL to Origin 2000; drafted upgrade proposal; coordinated installation, certification, testing, and migration of proprietary database clients. System response time increased by minimum of 30% based on database benchmarks.
    6. Coordinated the reinvestigation of disk storage technology; formed and led the team that generated the project requirements; coordinated appropriate vendor briefings, and helped to develop the benchmarks for data warehousing evaluations which were used to determine the company's standard disk vendor.
    7. Developed web based change control logging system which allows system and database administrators to log proposed changes.
    8. Developed and implemented web based system documentation which employs network programming via perl to display real-time system configuration information and performance metrics.
    9. Researched, coordinated, and installed system to directly transfer data from company mainframes to various UNIX platforms. Reduced Oracle database load times from two weeks to three days. Reduced Fastcount load times from 7 days to 1.5 days.
    10. Planned, coordinated and upgraded SGI Challenge XL to IRIX 6.2 with simultaneous filesystem reorganization. Improved client proprietary database performance by 23%, measured by system accounting statistics.
    11. Migrated multimillion dollar client company between IBM RS/6000 servers. Coordinated; organized tests for system certification; reorganized existing filesystems and users for greater efficiency; transferred existing RAID controllers with no loss of data and minimum downtime.
    12. Identified and acted on a problem with SGI system performance; using IRIX 5.3 sar and OS performance monitors, identified scope of I/O bottlenecks and designed tests to determine best fit of filesystems to controllers and I/O. Striped filesystems and enabled command tag queuing; resulted in over 1000% drop in average time spent waiting I/O.


    Company: Litton Data Systems
    Title: UNIX System Administrator
    Time frame: 03/95 through 01/96





    1. Researched virtual memory problems for HP systems. Identified configuration problem; proposed, coordinated, scheduled, and implemented virtual memory reconfiguration. Resulted in 10% increase in system performance measured via system memory statistics and timing studies
    2. Documented Litton DSD network: inventoried all PC/HP systems, traced cable plant, and organized network patch panels. Developed automated process for obtaining DOS based PC software configurations; reduced required time for configuration audit process by 75%.
    3. Helped plan & moved systems/network; coordinated IP address transition with corporate headquarters, planned and implemented new network patches. Litton DSD Chief Information Officer reported the move the smoothest that he'd seen in over 20 years


    Company: USAF
    Title: Airborne Cryptologic Linguist
    Time frame: 11/82 through 03/95





    1. Designed, coordinated, and implemented 120 node local area network; integrated Windows and UNIX operating systems; Internetworked unit LAN with a classified wide area network using TCP/IP protocol suite
    2. Designed and implemented integrated network file system using DOS, AIX Ver 3.2.5, SUNOS 4.1.3, System V, rel 3.2.2. Provided necessary security through UNIX permissions, NFS mount restrictions, and netgroups. Tuned mount operations for more efficient operations; Developed comprehensive training program for subordinate novice and junior system administrators
    3. Configured DNS and NIS; Integrated NIS and DNS between SUN Sparcs and IBM RISC/6000s; Provided assistance/training to other DNS administrators in USAF and Internet/civilian sector; Developed comprehensive training program for subordinate novice and junior system administrators
    4. Configured unit specific mail system; Configured mail hub using sendmail to send mail to Microsoft Mail SMTP gateway or local workstations as appropriate

    Management Experience

    1. Three years with May & Speh and Acxiom as UNIX team lead. Supervised a team of between two and eight administrators of various skill levels. Assigned projects and training based on experience and capability; tracked project status and provided feedback as needed
    2. Seven years experience managing 13 man Communications Computers branch for the USAF; developed work schedules, branch budgets, training plans and programs; trained subordinates from novice to advanced and senior system administrators
    3. Project management:
      1. Project manager for Litton DSD network connectivity during move. Provided tasking, schedules, and support for 7 man team tasked with planning and accomplishing the network move.
      2. Project manager for client migration. Planned, coordinated, and accomplished the requisite steps to move multimillion dollar client with no data loss and only six hours down time
      3. Team lead for SUN E10K installation, configuration, and client migration from older HP systems.
      4. Project manager for SUN->EMC Disk migration.
        1. ID storage requirements; coordinated purchase with vendor and corporate resoources.
        2. Coordinated installation/configuraiton of new EMC equipment.
        3. Designed specific migration strategies and procedures for each client and system.
        4. Coordinated removal of old disk equipment.

    Training




    Org Courses
    SUN:
    1. Solaris Admin I/II
    2. SUN Fault Analysis
    3. SUN Server Administration
    4. Veritas Volume Management
    5. SUN Enterprise 10,000 administration
    HP:
    1. HP Openview
    2. HP Advanced administration
    3. HP performance monitoring and tuning.
    SGI:
    1. SGI Advanced administration.
    USAF:
    1. Completed 13 semester hours in Leadership and Managerial Communications through professional military education in residence courses
    EMC:
    1. SRDF
    2. Timefinder
    3. Powerpath
    4. Symmgr
    5. Control Center
    6. Fibre Zone
    Misc:
    1. Accredited course from Loyola on management and leadership communications
    2. UNIX Security Administration
    3. Network Security Administration
    4. sqlplus
    5. Advanced sqlplus
    6. Oracle Database Administration.