Advanced Kickstart Scripting

John T. Mills
John T. Mills
Advanced Kickstart Scripting

Kickstart is a complex and extensive topic, which goes beyond what can be covered in a single article. In this post I'll cover some of the posibilities which can be leveraged for scalable and flexible design. The RHEL8 Guide - Performing an Advanced RHEL Installation has extensive kickstart appendices in Part V, appendix A and B. Also in PDF.

Dual UEFI/BIOS Boot

UEFI and BIOS use differing boot partition schemes, and because we add partitions outside of script interpreted sections, this requires some trickery. The keys here are %include and %pre. %include is run after the %pre section. It pastes the code from the included file into the kickstart, as if the contents were there at the point of the %include. We will be running bash executions in %pre allowing for variables, computation, and programmatic inclusion.

With the BIOS+LVM boot, we have:

clearpart --all --initlabel
part /boot       --fstype xfs --size=1024 --asprimary   --ondisk sda
part pv.1 --fstype="lvmpv" --ondisk=sda --size=1 --grow
volgroup rootvg --pesize=4096 pv.1
logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg 
logvol swap                   --size=1024 --name=swaplv --vgname=rootvg

%pre
dd if=/dev/zero of=/dev/sda count=1000 bs=1M
%end

With the UEFI+LVM boot, we have:

clearpart --all --initlabel
part /boot       --fstype xfs --size=512  --asprimary   --ondisk sda
part /boot/efi   --fstype xfs --size=1024 --asprimary   --ondisk sda
part pv.1 --fstype="lvmpv" --ondisk=sda --size=1 --grow
volgroup rootvg  --pesize=4096 pv.1
logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg
logvol swap                   --size=1024 --name=swaplv --vgname=rootvg

Dual BIOS/UEFI with LVM, we have:

%include /tmp/uefi
%include /tmp/bios

%pre
if [ -d /sys/firmware/efi ]
   then
   FILE=/tmp/uefi
   echo "clearpart --all --initlabel" > $FILE
   echo "part /boot       --fstype xfs --size=512  --asprimary   --ondisk sda" >> $FILE
   echo "part /boot/efi   --fstype xfs --size=1024 --asprimary   --ondisk sda" >> $FILE
else
   dd if=/dev/zero of=/dev/sda count=1000 bs=1M
   FILE=/tmp/bios
   echo "clearpart --all --initlabel" > $FILE
   echo "part /boot       --fstype xfs --size=1024 --asprimary   --ondisk sda" >> $FILE
fi
echo "part pv.1 --fstype="lvmpv" --ondisk=sda --size=1 --grow" >> $FILE
echo "volgroup rootvg  --pesize=4096 pv.1" >> $FILE
echo "logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg" >> $FILE
echo "logvol swap                   --size=1024 --name=swaplv --vgname=rootvg" >> $FILE
touch /tmp/uefi /tmp/bios # one will be empty
%end

All we have to do is check for the presence of /sys/firmware/efi in the installation, and we can provide the UEFI partitioning stanza.

First Boot Scripting

Kickstart provides a lot of good customization functionality, but some tasks can't be accomplished through it. For example, when you kickstart an image for use in templating (OVA/OVF, Imaging, etc.). These needs can be mitigated with orchestration tools like cloud-init and others, but wouldn't it be nice if we could do it though the image itself?

Hard-coded Approach: Sometimes immutable is good.

%post
RCL=/etc/rc.d/rc.local
FBS=/root/firstboot
echo "#!/bin/bash" >> $FBS
echo "#" >> $FBS
# Script
echo "echo \"Hello World.\"" >> $FBS
# Run once and remove from RCL
echo "sed -i \"s,^$FBS,,\" $RCL" >> $FBS
# Add it to RCL, for next boot
echo "$FBS" >> $RCL
chmod +x $RCL $FBS
%end

Dependency Approach: Sometimes flexibility is needed.

%post
RCL=/etc/rc.d/rc.local
FBS=/root/firstboot
# Download Script
wget -q http://192.168.0.37/scripts/firstboot -O $FBS
# Run once and remove from RCL
echo "sed -i \"s,^$FBS,,\" $RCL" >> $FBS
# Add it to RCL, for next boot
echo "$FBS" >> $RCL
chmod +x $RCL $FBS
%end

Mirrored Boot Drives (That actually work)

We may want to mirror a server's storage for a variety of reasons. There are a few ways to do this. We can do controller-based mirroring, server-based hardware mirroring, and server-based software mirroring. Controller-based mirroring only presents a single virtual disk to the server, so we don't have any kickstart changes for those. Server-based mirroring can seem a bit more complex, but it exposes the health of the RAID directly to the OS layer for management and alerting.

MDADM Mirroring with Partitions: Partition mirroring will be necessary for /boot for every solution. Here's what it looks like as a complete solution, with no LVM partitions.

bootloader --location=mbr --driveorder=sda,sdb
clearpart --all --initlabel
part raid.1   --fstype=xfs   --size=1024  --asprimary   --ondisk=sda
part raid.2   --fstype=xfs   --size=1024  --asprimary   --ondisk=sdb
part raid.3   --fstype=xfs   --size=8192                --ondisk=sda
part raid.4   --fstype=xfs   --size=8192                --ondisk=sdb
part raid.5                  --size=1024                --ondisk=sda
part raid.6                  --size=1024                --ondisk=sdb

raid /boot    --fstype=xfs   --device=md0 --level=RAID1 raid.1 raid.2
raid /        --fstype=xfs   --device=md1 --level=RAID1 raid.3 raid.4
raid swap                    --device=md2 --level=RAID1 raid.5 raid.6

%pre
dd if=/dev/zero of=/dev/sda count=1000 bs=1M
dd if=/dev/zero of=/dev/sdb count=1000 bs=1M
%end

The VM will boot with this configuration: example image

If a failure occurs, this system will continue to run and be able to boot. example image

MDADM Mirroring with LVM: /boot isn't under LVM, as /boot needs partition mirroring. We can use LVM for all of the other partitions.

bootloader --location=mbr --driveorder=sda,sdb
clearpart --all --initlabel
part raid.1   --fstype=xfs   --size=1024  --asprimary   --ondisk=sda
part raid.2   --fstype=xfs   --size=1024  --asprimary   --ondisk=sdb
part raid.3                  --size=1     --grow        --ondisk=sda
part raid.4                  --size=1     --grow        --ondisk=sdb

raid /boot    --fstype=xfs   --device=md0 --level=RAID1 raid.1 raid.2
raid pv.1     --fstype=lvmpv --device=md1 --level=RAID1 raid.3 raid.4

volgroup rootvg --pesize=4096 pv.1
logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg 
logvol swap                   --size=1024 --name=swaplv --vgname=rootvg

%pre
dd if=/dev/zero of=/dev/sda count=1000 bs=1M
dd if=/dev/zero of=/dev/sdb count=1000 bs=1M
%end

The VM will boot with this configuration: example image

If a failure occurs, this system will continue to run and be able to boot. example image

Platform Generalization

Generalization can be a large topic. In this case, I'll show how to use a single kickstart across several different platforms.

Disk attachment: This supports sda, hda, xda, and vda for UEFI and BIOS.

%pre
disk=`ls /dev/[s,h,v,x]da | sed "s,/dev/,,"`
if [ -d /sys/firmware/efi ]
   then
   FILE=/tmp/uefi
   echo "clearpart --all --initlabel" > $FILE
   echo "part /boot       --fstype xfs --size=512  --asprimary   --ondisk $disk" >> $FILE
   echo "part /boot/efi   --fstype xfs --size=1024 --asprimary   --ondisk $disk" >> $FILE
else
   dd if=/dev/zero of=/dev/$disk count=1000 bs=1M
   FILE=/tmp/bios
   echo "clearpart --all --initlabel" > $FILE
   echo "part /boot       --fstype xfs --size=1024 --asprimary   --ondisk $disk" >> $FILE
fi
echo "part pv.1 --fstype="lvmpv" --ondisk=$disk --size=1 --grow" >> $FILE
echo "volgroup rootvg  --pesize=4096 pv.1" >> $FILE
echo "logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg" >> $FILE
echo "logvol swap                   --size=1024 --name=swaplv --vgname=rootvg" >> $FILE
touch /tmp/uefi /tmp/bios # one will be empty
%end

Guest Tools: Install tools after determining the virtualization platform. We need to handle this in %post.

%post
enable_esx()
   {
   yum -y install open-vm-tools
   systemctl --enable --now vmtoolsd.service
}
enable_waa()
   {
   yum -y install WALinuxAgent
   systemctl --enable --now waagent.service
}
virtual=`dmesg | grep "Hypervisor detected" | awk {'print $5,$6'}`
case $virtual in
            VMware) enable_esx   ;;
   MicrosoftHyperV) enable_waa   ;;
esac
%end

Swap Sizing

This is a fairly simple requirement to meet with a %pre script. Here are two scenarios:

Numeric Scaling: Using a simple multiplier works in some cases.

%include /tmp/bios

%pre
dd if=/dev/zero of=/dev/sda count=1000 bs=1M
FILE=/tmp/bios

mem=`free -m | grep ^Mem | awk {'print $2'}`
let swap=$mem*2 # 2x swap to memory

echo "clearpart --all --initlabel" > $FILE
echo "part /boot       --fstype xfs --size=1024 --asprimary   --ondisk sda" >> $FILE
echo "part pv.1 --fstype="lvmpv" --ondisk=sda --size=1 --grow" >> $FILE
echo "volgroup rootvg  --pesize=4096 pv.1" >> $FILE
echo "logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg" >> $FILE
echo "logvol swap                   --size=$swap --name=swaplv --vgname=rootvg" >> $FILE
%end

Threshold Scaling: The following provides for buckets of rules based on memory allocated to the server.

Memory Available Swap Recommendation
64 GB and above 64 GB
64 - 48 GB 125% of Memory
48 - 32 GB 150% of Memory
32 - 16 GB 175% of Memory
16 GB or less 200% of Memory 
%include /tmp/bios

%pre
dd if=/dev/zero of=/dev/sda count=1000 bs=1M
FILE=/tmp/bios

mem=`free -m | grep ^Mem | awk {'print $2'}`
if   [ "$mem" -ge "65536" ]
   then
   swap=65536                                           # 100% mem, and max swap
elif [ "$mem" -ge "49152" ]
   then
   swap=$(echo "$mem*1.25" | bc | awk -F. {'print $1'}) # 125% mem
elif [ "$mem" -ge "32768" ]
   then
   swap=$(echo "$mem*1.50" | bc | awk -F. {'print $1'}) # 150% mem
elif [ "$mem" -ge "16384" ]
   then
   swap=$(echo "$mem*1.75" | bc | awk -F. {'print $1'}) # 175% mem
else
   swap=$(echo "$mem*2.00" | bc | awk -F. {'print $1'}) # 200% mem
fi
echo "clearpart --all --initlabel" > $FILE
echo "part /boot       --fstype xfs --size=1024 --asprimary   --ondisk sda" >> $FILE
echo "part pv.1 --fstype="lvmpv" --ondisk=sda --size=1 --grow" >> $FILE
echo "volgroup rootvg  --pesize=4096 pv.1" >> $FILE
echo "logvol /         --fstype=xfs --size=8192 --name=rootlv --vgname=rootvg" >> $FILE
echo "logvol swap                   --size=$swap --name=swaplv --vgname=rootvg" >> $FILE
%end

Encrypted Passwords

Encrypt root by creating a secure string:

# openssl passwd -1 "correcthorsebatterystaple"
$1$sp4v.reQ$Po068zaL2OQ7fIvdph5te1

And add it to kickstart:

rootpw --iscrypted $1$sp4v.reQ$Po068zaL2OQ7fIvdph5te1

Encrypt grub by creating a secure string:

# grub2-mkpasswd-pbkdf2
Enter password:
Reenter password:
PBKDF2 hash of your password is grub.pbkdf2.sha512.10000.EBD680C8A9514561F3719DD230B437DD037A6A1D6CF9CAB739534890906DC73ED0284E22EA1BDDE6FFCFB68CFCECE2D3C61ADBEAEE2A837AE48FA6D7308716AE.D1A5FB00CD90910D962CCC9F5378C531DBA9B94D8A1EEDAA9252F5569D4FBD4D9F5FE0CCE7EC381D71B9727764370AA3FFDDC9382A6A5D41AC26BE827AC18FF1

And add it to kickstart:

bootloader --iscrypted --password=grub.pbkdf2.sha512.10000.EBD680C8A9514561F3719DD230B437DD037A6A1D6CF9CAB739534890906DC73ED0284E22EA1BDDE6FFCFB68CFCECE2D3C61ADBEAEE2A837AE48FA6D7308716AE.D1A5FB00CD90910D962CCC9F5378C531DBA9B94D8A1EEDAA9252F5569D4FBD4D9F5FE0CCE7EC381D71B9727764370AA3FFDDC9382A6A5D41AC26BE827AC18FF1

Conclusion

I hope some of these options find their way into your kickstart scripts!