Mandatory Packages in Kickstart

When building a %packages section in a kickstart file you may want to remove some packages you don’t need. But they might get installed anyways. It turns out Anaconda requires some packages be installed. So, what are these mandatory packages?

Looking at the code for RHEL7 we can see that some basic packages from Storage, Realm, AuthConfig, and FirewallD need to be included.

https://git.fedorahosted.org/cgit/anaconda.git/tree/pyanaconda/install.py?h=rhel7-branch#n210

    packages = storage.packages + ksdata.realm.packages
    packages += ksdata.authconfig.packages + ksdata.firewall.packages

    if not ksdata.bootloader.disabled:
        packages += storage.bootloader.packages

    if network.is_using_team_device:
        packages.append("teamd")
 

storage.packages comes from Blivet and will include packages needed to configure your storage. Things like lvm2, or mdadm, or device-mapper-multipath. You can dig through the source to see which packages might be included depending on what storage you are choosing to configure.

If you use the kickstart command realm to join an Active Directory domain then the realm package will be installed.

authconfig is simply a required package to configure authentication on the system.

firewalld is also a required package.

The bootloader is usually grub2 and so will install the needed package. If you are using a non-standard bootloader check the source of bootloader.py for what packages you will get.

If you are using network teaming or bonding, then you will get teamd installed.

So be aware of these extra packages not listed in the @core group of packages.

Advertisements

What is a minimal install?

In my post Basics of Kickstart I talked about doing a minimal install. So what is included in the @core group for each major version of CentOS? I’ll document here for the current versions as it’s possible these values will change.

CentOS 5.11

  • Mandatory Packages
    • SysVinit
    • authconfig
    • basesystem
    • bash
    • centos-release
    • coreutils
    • cpio
    • dhclient
    • dhcpv6-client
    • e2fsprogs
    • ed
    • file
    • filesystem
    • glibc
    • hdparm
    • hmaccalc
    • initscripts
    • iproute
    • iputils
    • kbd
    • kudzu
    • libgcc
    • libhugetlbfs
    • libtermcap
    • mkinitrd
    • openssh-server
    • passwd
    • policycoreutils
    • prelink
    • procps
    • readline
    • redhat-logos
    • rootfiles
    • rpm
    • selinux-policy-targeted
    • setools
    • setserial
    • setup
    • shadow-utils
    • sysklogd
    • termcap
    • util-linux
    • vim-minimal
    • yum
  • Default Packages
    • Deployment_Guide-en-US
    • grub
    • sysfsutils
    • udftools

CentOS 6.8

  • Mandatory Packages
    • acl
    • attr
    • audit
    • authconfig
    • basesystem
    • bash
    • coreutils
    • cpio
    • cronie
    • dhclient
    • e2fsprogs
    • filesystem
    • glibc
    • initscripts
    • iproute
    • iptables
    • iptables-ipv6
    • iputils
    • kbd
    • ncurses
    • openssh-server
    • passwd
    • policycoreutils
    • procps
    • rootfiles
    • rpm
    • rsyslog
    • selinux-policy-targeted
    • setup
    • shadow-utils
    • sudo
    • system-config-firewall-base
    • util-linux-ng
    • vim-minimal
    • yum
  • Default Packages
    • aic94xx-firmware
    • atmel-firmware
    • b43-openfwwf
    • bfa-firmware
    • efibootmgr
    • grub
    • ipw2100-firmware
    • ipw2200-firmware
    • ivtv-firmware
    • iwl100-firmware
    • iwl1000-firmware
    • iwl3945-firmware
    • iwl4965-firmware
    • iwl5000-firmware
    • iwl5150-firmware
    • iwl6000-firmware
    • iwl6000g2a-firmware
    • iwl6050-firmware
    • kernel-firmware
    • kexec-tools
    • libertas-usb8388-firmware
    • netxen-firmware
    • postfix
    • ql2100-firmware
    • ql2200-firmware
    • ql23xx-firmware
    • ql2400-firmware
    • ql2500-firmware
    • rdma
    • rt61pci-firmware
    • rt73usb-firmware
    • xorg-x11-drv-ati-firmware
    • zd1211-firmware

CentOS 7.2.1511

  • Mandatory Packages
    • audit
    • basesystem
    • bash
    • biosdevname
    • btrfs-progs
    • coreutils
    • cronie
    • curl
    • dhclient
    • e2fsprogs
    • filesystem
    • firewalld
    • glibc
    • hostname
    • initscripts
    • iproute
    • iprutils
    • iptables
    • iputils
    • irqbalance
    • kbd
    • kexec-tools
    • less
    • man-db
    • ncurses
    • openssh-clients
    • openssh-server
    • parted
    • passwd
    • plymouth
    • policycoreutils
    • procps-ng
    • rootfiles
    • rpm
    • rsyslog
    • selinux-policy-targeted
    • setup
    • shadow-utils
    • sudo
    • systemd
    • tar
    • tuned
    • util-linux
    • vim-minimal
    • xfsprogs
    • yum
  • Default Packages
    • NetworkManager
    • NetworkManager-team
    • NetworkManager-tui
    • aic94xx-firmware
    • alsa-firmware
    • dracut-config-rescue
    • ivtv-firmware
    • iwl100-firmware
    • iwl1000-firmware
    • iwl105-firmware
    • iwl135-firmware
    • iwl2000-firmware
    • iwl2030-firmware
    • iwl3160-firmware
    • iwl3945-firmware
    • iwl4965-firmware
    • iwl5000-firmware
    • iwl5150-firmware
    • iwl6000-firmware
    • iwl6000g2a-firmware
    • iwl6000g2b-firmware
    • iwl6050-firmware
    • iwl7260-firmware
    • iwl7265-firmware
    • kernel-tools
    • libsysfs
    • linux-firmware
    • microcode_ctl
    • postfix
    • rdma

So if you use --nobase in your kickstart file, you will get everything in Mandatory and Default lists. You can choose to exclude some of the defaults with minus (-) to slim up your install though, or use --nodefaults to skip them entirely.

To find these lists yourself, just ask yum! yum groupinfo core

Basics of Kickstart

If you need to install CentOS over and over, one useful thing is to create a kickstart file. This is a text config file that directs the install program and can make an install entirely unattended.

You can find a full reference of all options on Red Hat’s documentation site.23.3. Kickstart Syntax Reference.

Sources

So what do we need? And what are some nice addons?

First, we can say we want to install. This is optional, but encouraged. Followed by a source for the packages. This can be a local media like cdrom or harddisk, or a network share like nfs, or my favorite url.

The URL you specify here should be to the os folder on a mirror and have as a subfolder repodata. This URL will have all the packages needed to install CentOS. You can find a list of mirrors on the CentOS site, or just provide a mirrorlist URL instead. A mirrorlist URL will give YUM a place to fetch a list of mirrors to try and it will attempt to get the fastest one.

You can also specify additional repos for the installer to pull packages from as it sees fit or that you specify. I like to at least include the updates repo, so that we install the latest packages on the first try, and don’t have to do a yum update after the install. Here is our kickstart file so far.

# Do an install
install

# From this hard coded URL
# url --url=http://mirror.its.sfu.ca/mirror/CentOS/7/os/x86_64/
# Or better yet, from a mirrorlist with variables
url --mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
# Extra repos let us install the latest versions
repo --name="Updates" --mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
# Optional if you want packages from EPEL
repo --name="epel" --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-$releasever&arch=$basearch

Install Settings

We can select the kind of display we want from the installer. graphical, text, or cmdline. I choose cmdline as it provides the most helpful debug output. text mode gives you the classic ncurses display.

We can also set what the installer should do after finishing. I find reboot to be the most helpful, but you can also choose halt and poweroff.

Lastly, we’ll disable firstboot. That’s the “helper” you get on the first boot up asking you to make a user and such. Since we are trying to automate things, we don’t want to be bothered.

# Install mode
cmdline

# reboot when finished the install
reboot

# Disable firstboot
firstboot --disabled

System Config

Here we specify some settings for the system we are building. We’ll set the language, keyboard layout, timezone, and SELinux. We’ll also set the default password storage policy to the strongest available.

# System settings
lang en_US.UTF-8
keyboard us
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone UTC

Network

Most places I use Linux there is Software Defined Networking (SDN) and it handles all the firewalling, so I just disable it in the system. We also want to turn off IPv6 as its just extra junk we don’t need. And we’ll stick to DHCP here.

network --bootproto dhcp --noipv6
firewall --disabled

Root Password

This just sets the root password. You can grab the hash for an existing user from /etc/shadow or just use a plaintext password.

#rootpw --plaintext mycoolpassword
rootpw --iscrypted $6$BHils6Q1$hTRN8PUTpmQG6y7bkeSPqWrWxCV9uja9EMhsmf5qk4rDhdnKHznYiz5CxBmFqiaO14I7utwu7ToH6y7gMwFeq/

Disk Space

Now we want to specify the disk layout. Do we want basic partitions or LVM? How big should stuff be? I usually go with a 1GB swap, 1GB /tmp and the rest as root disk. I add some safety options to /tmp to make sure evil things don’t try and exec from there.

# Set up the drive
bootloader --location=mbr
zerombr
clearpart --all --initlabel
part swap --asprimary --size=1024
part /tmp --fstype=ext4 --asprimary --size=1024 --fsoptions="defaults,nosuid,noexec"
part / --fstype=ext4 --grow --asprimary --size=100

Package Selection

I like to install the minimal possible and handle the rest with config management. The minimal install uses the @core group and not the @base group. @core includes a lot of packages by default that we probably don’t need. WiFi drivers, RAID card drivers, and junk like that. I’m usually building a image for VM use, so can exclude most of that by putting a minus (-) in front of the name. You can also use an asterisk (*) as a wildcard to match a bunch of packages. There are a few packages from @base I do like to include though, like acpid.

%packages --nobase
acpid
-aic94xx-firmware
-alsa-firmware
-bfa-firmware
-ivtv-firmware
-iwl*-firmware
-rdma
%end

Post Script

After the install is complete, you can run some shell scripts before the reboot to help get your system just right. I make some tweaks to grub and re-install it. Then I import all the RPM keys, so that when I run yum it doesn’t ask about importing them the first time.

%post
# Reduce timeout for faster boot
sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=1/' /etc/default/grub
# Set consoles for proper logging and vnc
# Be noisey to help debugging
sed -i 's/GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet"/GRUB_CMDLINE_LINUX="crashkernel=auto console=ttyS0,115200n8 console=tty0"/' /etc/default/grub
# Rebuild grub config
grub2-mkconfig -o /boot/grub2/grub.cfg
# Import all the keys
/bin/rpm --import /etc/pki/rpm-gpg/*
%end

And with all that, we are done a basic kickstart. Be sure to read the docs and customize as you see fit!

yum clean all – will miss stuff

So I learnt that yum clean all won’t delete folders from /var/cache/yum for repos that aren’t currently enabled. This can bloat VM images you are building. So it’s best to just rm -rf /var/cache/yum to make sure you get everything after a yum clean all.

Here is the full set of yum cleaning I do in my image build scripts to make them as small as possible.

yum history new
yum clean all
rm -f /var/lib/rpm/__db*
rm -rf /var/cache/yum
rm -rf /var/lib/yum

Rotating tcpdump

Edit: Turns out someone is smarter than me on Stack Overflow’s sub site for Unix & Linux. You can do time based rotation, but you must limit your filename to only include the part of a timestamp that you are rotating on. So only include %H if you are doing -G 3600 to get hourly files. When the day rolls over, the file with hour 00 will get overwritten. Thanks dfernan.

So if you are trying to catch an intermittent networking issue, sometimes you need to leave tcpdump running for a while and then grab the logs after something goes wrong.

Want to create a new log file every X seconds and timestamp them? Use -G and supply some formatted filename. The command below will log one hours worth of data to a file named after the start time of that file. See strftime(3) for formatting details.

tcpdump -w 'trace__%Y-%m-%d_%H:%M:%S.pcap' -G 3600

This will happily run forever and eat all of your disk space though. If you want to limit to just a set number of files, use -W

tcpdump -w 'trace__%Y-%m-%d_%H:%M:%S.pcap' -G 3600 -W 5

This will cause tcpdump to stop after 5 hours.

But what if we want rotating files? Like with logs, so we can run forever and not fill up the disk. Well we need to abandon the fancy naming of our files to get it to work. We need to specify a maximum file size and remove the time constraint.

tcpdump -w trace -W 5 -C 1024

This will capture 5 files named trace0, trace1, trace2, etc. They will contain one gigabyte of data each. Once 5 files exist, the oldest will be deleted and replaced with new data.

If you try and use -G and -C at the same time, weird things happen. If you use a basic filename, tcpdump will go back to file 0 when either it runs out of space in the 5th file (a gig of data) OR the timer runs out. So if you get less than a gig per hour, you’ll only ever have one file with the current timers data in it.

If you use a formatted file name, then -W is just ignored and you get infinite files of either an hour or a gig of data, whichever comes first.

So to get log style rotation, you need to use -W and -C without -G or formatted filenames.

Fun with diff

So I needed to find and print only the lines in one file that were not in another file. I found a sweet answer on StackOverflow with some helpful infos. You can specify the format for old, new, and unchanged. By setting some of these to empty string you can simply suppress their output.

diff --old-line-format="" --unchanged-line-format="" old.txt new.txt

You can also pass in shells instead of files to diff using redirection.

diff --old-line-format="" --unchanged-line-format="" <(cut -d' ' -f 3 md5sum.txt | sort ) <(ls *.bin)

This lets you get up to all kinds of tricks.

How to block an IP with FirwallD

The internet is full of lies. This is a very simple thing to do but most posts out there go into complicated direct rules or rich rules to solve this. Here is an example of the hard way to do it.

# firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.0.1' reject"

FirewallD comes with some built in Zones. These zones serve different purposes and make things way simpler. Let’s get a list of the zones already existing in the default config.

# firewall-cmd --permanent --get-zones
block dmz drop external home internal public trusted work

Oooo look! A zone for “block” and one for “drop”. Bet those will be handy. What are the base rules of one of them?

# firewall-cmd --permanent --zone drop --list-all
drop
 interfaces:
 sources:
 services:
 ports:
 masquerade: no
 forward-ports:
 icmp-blocks:
 rich rules:

OK, that doesn’t tell us much. Lets dig a bit deeper.

# firewall-cmd --permanent --zone drop --get-target
DROP

Ah, there we go. Some zones have a target that is the equivalent of the old -j in iptables. The block zone uses REJECT and the trusted zone use ACCEPT. All other default zones don’t have a target. So to block an IP or network, just add it to the block or drop zone. To accept all traffic from an IP or network, add it to the trusted zone.

# firewall-cmd --permanent --zone drop --add-source 192.168.1.0/24
success
# firewall-cmd --reload
# firewall-cmd --permanent --zone drop --list-all
drop
 interfaces:
 sources: 192.168.1.0/24
 services:
 ports:
 masquerade: no
 forward-ports:
 icmp-blocks:
 rich rules:

Wasn’t that simpler than using rich rules? If you want to see the default rules and a description for each zone, check out their xml files in /usr/lib/firewalld/zones/

# cat /usr/lib/firewalld/zones/drop.xml
<?xml version="1.0" encoding="utf-8"?>
<zone target="DROP">
 <short>Drop</short>
 <description>Unsolicited incoming network packets are dropped. Incoming packets that are related to outgoing network connections are accepted. Outgoing network connections are allowed.</description>
</zone>

Currently enforced zones and their rules can be found in /etc/firewalld/zones/. Linode has a good post about FirewallD basics.

mysqldump and DEFINER troubles

If you are using mysqldump to transfer a database between servers as a non-super user and you dump contains VIEWs or FUNCTIONs that have a definer (possibly your user even) then you will get an error during import due to not being super enough.

ERROR 1227 (42000) at line XXXX: Access denied; you need (at least one of) the SUPER privilege(s) for this operation

This is what you typically get for a VIEW

/*!50001 CREATE ALGORITHM=UNDEFINED */
/*!50013 DEFINER=`slashterix`@`%` SQL SECURITY DEFINER */
/*!50001 VIEW `neater_data` AS (select `neat_data`.`foo` AS `foo`,`neat_data`.`bla` AS `bla` from `neat_data`) */;

And you will get one of these for a FUNCTION depending on your MySQL version.

/*!50003 CREATE*/ /*!50020 DEFINER=`slashterix`@`%`*/ /*!50003 FUNCTION `myCoolFunc` (
CREATE DEFINER=DEFINER=`slashterix`@`%` FUNCTION `myCoolFunc` (

For the VIEW and the second FUNCTION a simple sed will work.

sed -r 's/DEFINER=`?\w+`?@`[^`]+`//'

The -r turns on extended regex support. For the middle case though, this will strip out the definer and leave us with an empty version comment that some versions don’t like.

/*!50020 */

Use this one first to fix those cases.

sed -r 's/\/\*![0-9]+ DEFINER=`?\w+`?@`[^`]+`\*\/ //'

I string both seds together for maximum compatibility with all versions of MySQL.

Parallel SSH

The knife command can use a lot of memory when doing a search. So knife ssh "name:web-*" can fail after eating all you RAMs and getting OOM killed.

parallel -j 10 -i \
  ssh -o StrictHostKeyChecking=no \
      -o UserKnownHostsFile=/dev/null \
      -i private.key \
      user@{}.example.com \
      'hostname; sudo chef-client;' \
  -- $( knife node list | grep '^web-' )

Different versions of parallel have different options, this was used on CentOS 7.

  • -j number of jobs to run in parallel
  • -i enable use of {} to insert the strings
  • — end of args, next is list of things

UTF8 in PHP and MySQL

Iñtërnâtiônàlizætiøn

Can your code handle that? I found a lot of great info on the PHP WACT site for making your code and database work well with complex character sets. This is important even if your intended audience is North American english speaking only. MS Word uses smart quotes which are UTF8 characters and will cause havoc on your site.

So what’s the tl;dr of the PHP WACT site?

  1. All MySQL tables and columns set to utf8-general-ci (or other sorting of your choice).
  2. Ensure your connection to the database is UTF8 with SET NAMES 'utf8';
  3. Send a header to declare your page is UTF8. This also ensures POST content is sent to you in UTF8. The browser will help convert for you. header('Content-Type: text/html; charset=utf-8');
  4. Use htmlspecialentities() for making user submitted or untrusted text safe to display in HTML or XML. It will do the bare minimum and nothing more. htmlspecialchars($utf8_string, ENT_COMPAT, 'UTF-8');

Max Woolf has a GitHub repo of Naughty Strings you can use in testing your code to make sure it supports everything.

Bonus reading: Emoji and MySQL use utf8mb4