Mimic

The Mimic Xen host is capable of efficiently running a number of virtual machines. If you are reinstalling over an existing host, then you will first need to backup

• the Xen domains’ disk images,
• the Xen domains’ configurations,
• data on partitions that are passed through to Xen domains (e.g., using rsync --inplace --delete -aHA * /BACKUP-DISK-MOUNT-POINT),
• /etc/NetworkManager/system-connections/*, and
• a public SSH key.

To ensure a GPT partition label, even on disks less than 2 GB, provide the inst.gpt argument to the Fedora installer’s boot process. Set aside partitions for any virtual machines that will require direct disk access; I used the following scheme:

Follow the following steps while executing a Fedora install:

1. Review the host’s BIOS/firmware menu to ensure its virtualization instructions are active.
2. Add nomodeset to the kernel’s command-line arguments if necessary (both before booting the installer and in /etc/default/grub on the resulting filesystem).
3. During the Fedora install:
   1. Set the hostname to mimic.flyn.org.
   2. Activate the root account, and set its password.
   3. Select the “Fedora Custom Operating System” set from the Software Selection spoke.
   4. Set the timezone.
   5. Layout the disk as described above, taking care to ensure the partition numbers are assigned as indicated.
4. Remove the avahi, avahi-libs, firewalld-filesystem, firewalld, and systemd-resolved packages.
5. Run dnf update.
6. Install the following. See Red Hat Bugzilla #2512811 for the reason two dnf runs are needed.
   1. GRUB modules: dnf install grub2-efi-modules.
   2. Xen, syslog-ng, net-tools, and the network bridge utilities: dnf install xen syslog-ng net-tools bridge-utils.
7. The combination of Xen and AVX2 instructions presents a vulnerability when using some CPUs under certain circumstances. RHEL 10 and compatible distributions require the AVX2 instructions. You might need to add GRUB_CMDLINE_XEN_DEFAULT="spec-ctrl=gds-mit=no" to /etc/default/grub to turn off the related mitigation and enable AVX2 instructions.
8. Install a public SSH key, deactivate password-based SSH logins, and run systemctl restart sshd.
9. Configure syslog-ng (replace the host names that use example.com):
   1. Generate an internal-CA-signed certificate for mimic.example.com.
   2. Place the CA certificate at /etc/pki/ca-trust/source/anchors/ca.cert.
   3. Identify the CA certificate’s hash by running openssl x509 -hash -noout -in ca.cert.
   4. Link ca.cert to hash.0, where hash is the value from the previous step.
   5. Create /etc/pki/syslog-ng/, and place the host’s certificate and private key therein.
   6. Set the permissions on the private key to 0600.
   7. Write the following to /etc/syslog-ng/syslog-ng.conf (replace the host names that use example.com):

@version: 3.35
@include "scl.conf"

options {
      flush_lines (0);
      time_reopen (10);
      log_fifo_size (1000);
      chain_hostnames (off);
      use_dns (no);
      use_fqdn (no);
      create_dirs (no);
      keep_hostname (yes);
};

source s_sys {
      system();
      internal();
};

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

destination d_net {
      syslog("logserver.example.com" port(6514)
              transport("tls")
              tls(
                      ca-dir("/etc/pki/ca-trust/source/anchors")
                      cert-file("/etc/pki/syslog-ng/mimic.flyn.org.cert")
                      key-file("/etc/pki/syslog-ng/mimic.flyn.org.key")
              )
      );
};

filter f_kernel    { facility(kern); };
filter f_default   { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); };
filter f_auth      { facility(authpriv); };
filter f_mail      { facility(mail); };
filter f_emergency { level(emerg); };
filter f_news      { facility(uucp) or (facility(news) and level(crit..emerg)); };
filter f_boot      { facility(local7); };
filter f_cron      { facility(cron); };

log { source(s_sys); filter(f_kernel); destination(d_net); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_net); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_net); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_net); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_net); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_net); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_net); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_net); destination(d_cron); };

@include "/etc/syslog-ng/conf.d/*.conf"

10. Configure a Xen network bridge device for each network interface on Mimic; for example /etc/NetworkManager/system-connections/xenbr0.nmconnection (replace xenbr0 and XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX, set method to auto if you would like to assign an IP address or link-local if you would not, and select the corresponding IPv6 configuration):

[connection]
id=xenbr0
uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
type=bridge
interface-name=xenbr0

[bridge]
stp=false

[ipv4]
method=auto   # Or method=link-local.

[ipv6]
dhcp-iaid=mac # Or dhcp-iaid=mac.
method=auto   # Or method=link-local.

Ensure this file is owned by root and bears the permissions 0600.

11. Associate each network interface on Mimic with a bridge; for example /etc/NetworkManager/system-connections/bridge-slave-eno0.nmconnection (replace eno0, XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX, and xenbr0):

[connection]
id=bridge-slave-eno0
uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
type=ethernet
interface-name=eno0
master=xenbr0
slave-type=bridge

Ensure this file is owned by root and bears the permissions 0600.

12. Place Xen guest configurations in /etc/xen, create links to /etc/xen/auto, and place guest disk images in /var/lib/xen/images. Here is an example guest configuration which boots an OpenWrt installation (replace guest and xx:xx:xx:xx:xx:xx):

name    = "guest"
memory  =  1024
vcpus   =  1
builder = "hvm"
vif     = [ "model=e1000,script=vif-bridge,bridge=xenbr1,mac=xx:xx:xx:xx:xx:xx" ]
disk    = [ 
  "tap2:tapdisk:aio:/var/lib/xen/images/guest-openwrt-15.05.1-x86-64-combined-ext4.img,xvda,w",
  "tap2:qcow:/var/lib/xen/images/guest-data.qcow,xvdb,w"
          ]
serial  = "pty"

13. Allow xenstored to run under SELinux’s policy by running setsebool -P domain_can_mmap_files=true.
14. Ensure the Xen hypervisor is the default boot selection (replace X and Y with versions found in /etc/grub2.cfg):

grub2-set-default "Fedora, with Xen X and Linux Y"
grub2-mkconfig -o /boot/grub2/grub.cfg

15. Setup the second disk with the volumes that will be provided to virtual machines:
    1. parted /dev/sdN mklabel gpt.
    2. parted -a optimal /dev/sdN mkpart primary btrfs 0% 37%.
    3. parted -a optimal /dev/sdN mkpart primary btrfs 37% 100%.
    4. mkfs.btrfs /dev/sdN1.
    5. mkfs.btrfs /dev/sdN2.
16. Setup a mail forwarder, which will allow disk monitoring to send notifications:
    1. Install msmtp and s-nail.
    2. Write the following to /etc/msmtprc, replacing SMTP-SERVER, EMAIL, and PASSWORD:

    defaults
    auth           on
    tls            on
    tls_starttls   on
    syslog         on
    account        default
    host           SMTP-SERVER
    port           587
    from           EMAIL
    user           EMAIL
    password       PASSWORD
    

    3. Set the permissions on /etc/msmtprc to 0600.
17. Setup disk monitoring:
    1. Install smartmontools.
    2. Write the following to /etc/smartmontools/smartd.conf, replacing EMAIL:

       DEVICESCAN -a -o on -S on -s (S/../.././02|L/../../6/03) -W 4,40,55 -m EMAIL -M test
       

    3. Modify /lib/systemd/system/smartd.service to contain ConditionVirtualization=.
    4. Run systemctl enable smartd.
18. Optionally restore the backed up data from the previous install.