Mimic

Mar 12, 2020

The Mimic Xen host is capable of efficiently running a number of virtual machines. If you are reinstalling over an existing host, then you will first need to backup

the Xen domains’ disk images,
the Xen domains’ configurations,
data on partitions that are passed through to Xen domains (e.g., using rsync --inplace --delete -aHA * /BACKUP-DISK-MOUNT-POINT),
/etc/NetworkManager/system-connections/*, and
a public SSH key.

To ensure a GPT partition label, even on disks less than 2 GB, provide the inst.gpt argument to the Fedora installer’s boot process. Set aside partitions for any virtual machines that will require direct disk access; I used the following scheme:

Partition	Type	Mount point	Name	Size
1	BIOS boot	n/a	n/a	1,024 KiB
2	Standard/ext4	/boot	boot	1,024 MiB
3	btrfs (single subvolume)	/	root	200 GiB
4	Standard/ext4	n/a	herald	1.25 TiB
5	Standard/ext4	n/a	golem	Remaining space

Perform a minimal Fedora install on the computer. After finishing the install, complete the following steps:

Review the host’s BIOS/firmware menu to ensure its virtualization instructions are active.
Add nomodeset to the kernel’s command-line arguments if necessary (/etc/default/grub).
During the Fedora install:
1. Set the hostname to mimic.flyn.org.
2. Activate the root account, and set its password.
3. Select the minimal install package set.
4. Set the timezone.
5. Layout the disk as described above, taking care to ensure the partition numbers are assigned as indicated.
Remove the firewalld-filesystem and firewalld packages.
Run dnf update.
Install Xen, syslog-ng, net-tools, and the network bridge utilities: dnf install xen grub2-efi-modules syslog-ng net-tools bridge-utils.
Deactivate resolved with systemctl disable systemd-resolved and systemctl stop systemd-resolved.
Install a public SSH key, deactivate password-based SSH logins, and run systemctl restart sshd.
Configure syslog-ng (replace the host names that use example.com):
1. Generate an internal-CA-signed certificate for mimic.example.com.
2. Place the CA certificate at /etc/pki/ca-trust/source/anchors/ca.cert.
3. Identify the CA certificate’s hash by running openssl x509 -hash -noout -in ca.cert.
4. Link ca.cert to hash.0, where hash is the value from the previous step.
5. Place the host’s certificate and private key in /etc/pki/syslog-ng/.
6. Restrict the permissions on the private key.
7. Write the following to /etc/syslog-ng/syslog-ng.conf (replace the host names that use example.com):

@version: 3.35
@include "scl.conf"

options {
        flush_lines (0);
        time_reopen (10);
        log_fifo_size (1000);
        chain_hostnames (off);
        use_dns (no);
        use_fqdn (no);
        create_dirs (no);
        keep_hostname (yes);
};

source s_sys {
        system();
        internal();
};

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

destination d_net {
        syslog("logserver.example.com" port(6514)
                transport("tls")
                tls(
                        ca-dir("/etc/pki/ca-trust/source/anchors")
                        cert-file("/etc/pki/syslog-ng/mimic.flyn.org.cert")
                        key-file("/etc/pki/syslog-ng/mimic.flyn.org.key")
                )
        );
};

filter f_kernel    { facility(kern); };
filter f_default   { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); };
filter f_auth      { facility(authpriv); };
filter f_mail      { facility(mail); };
filter f_emergency { level(emerg); };
filter f_news      { facility(uucp) or (facility(news) and level(crit..emerg)); };
filter f_boot      { facility(local7); };
filter f_cron      { facility(cron); };

log { source(s_sys); filter(f_kernel); destination(d_net); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_net); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_net); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_net); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_net); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_net); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_net); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_net); destination(d_cron); };

@include "/etc/syslog-ng/conf.d/*.conf"

Configure a Xen network bridge device for each network interface on Mimic; for example /etc/NetworkManager/system-connections/xenbr0.nmconnection (replace xenbr0 and XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX, and set method to auto if you would like to assign an IP address):

[connection]
id=xenbr0
uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
type=bridge
interface-name=xenbr0

[bridge]
stp=false

[ipv4]
method=link-local

[ipv6]
dhcp-duid=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
method=link-local

Ensure this file is owned by root and bears the permissions 0600.

Configure each network interface on Mimic; for example /etc/NetworkManager/system-connections/bridge-slave-eno1.nmconnection (replace eno1, XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX, and xenbr0):

[connection]
id=bridge-slave-eno1
uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
type=ethernet
interface-name=eno1
master=xenbr0
slave-type=bridge

Ensure this file is owned by root and bears the permissions 0600.

Place Xen guest configurations in /etc/xen, create links to /etc/xen/auto, and place guest disk images in /var/lib/xen/images. Here is an example guest configuration which boots an OpenWrt installation (replace guest and xx:xx:xx:xx:xx:xx):

name    = "guest"
memory  =  1024
vcpus   =  1
builder = "hvm"
vif     = [ "model=e1000,script=vif-bridge,bridge=xenbr1,mac=xx:xx:xx:xx:xx:xx" ]
disk    = [ 
  "tap2:tapdisk:aio:/var/lib/xen/images/guest-openwrt-15.05.1-x86-64-combined-ext4.img,xvda,w",
  "tap2:qcow:/var/lib/xen/images/guest-data.qcow,xvdb,w"
          ]
serial  = "pty"

Allow xenstored to run under SELinux’s policy by running setsebool -P domain_can_mmap_files=true.
Ensure the Xen hypervisor is the default boot selection:

grub2-set-default "Fedora, with Xen hypervisor"
grub2-mkconfig -o /boot/grub2/grub.cfg

Optionally restore the backed up data from the previous install.