Installing ArchLinux the opinionated way

Similar to official instructions, but with some opinionated decisions. This is a work-in-progress manual on the things I did during my last ArchLinux installation. I will try to keep it updated as my system evolves.

These instructions follow the official ArchLinux installation instructions, but divert in an opinionated way in some steps. Follow with caution, and only if you know what you are doing!

Installation system, partitioning and formatting

Booting from installation medium

Boot from installation iso, set up keymap and font for installation environment:

loadkeys de-latin1
setfont ter-132b

It’s recommended to use a Ventoy boot stick, as we’ll also rely on using a rEFInd boot loader later on. For that, you can just put the official refind boot img on the Ventoy stick you have.

Partitioning

For partition#ing, we use gdisk:

# gdisk /dev/nvme0n1

1. Delete all partitions by creating a new GPT partition table with o.
2. Then create partitions with this layout:

   Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         2099199   1024.0 MiB  EF00  EFI system partition
   2         2099200         4196351   1024.0 MiB  EA00  XBOOTLDR partition
   3         4196352      3907028991   1.8 TiB     8304  ArchLinux-Crypt
   

3. Note new partitions can be created with n, printing works with p, and c allows to rename a partition.
4. Finally, write out the table with w.
5. Note that 8304 is required even for crypted volume to allow for GPT automounting.

Formatting disks

mkfs.fat -F 32 /dev/nvme0n1p1
mkfs.ext4 /dev/nvme0n1p2
# Takes good defaults into account, may want to check sector size via https://wiki.archlinux.org/title/Advanced_Format#NVMe_solid_state_drives ...
cryptsetup luksFormat -s 512 /dev/nvme0n1p3
# Check with:
cryptsetup luksDump /dev/nvme0n1p3
cryptsetup open /dev/nvme0n1p3 ArchLinux
mkfs.btrfs -O block-group-tree /dev/mapper/ArchLinux

Setup BTRFS

Setup BTRFS with separate volumes for rootfs and home for usage of btrbk:

mount --mkdir -t btrfs -o defaults,noatime,compress-force=zstd:6,ssd /dev/mapper/ArchLinux /mnt/btrfs_pool
btrfs subvol create /mnt/btrfs_pool/rootfs
btrfs subvol create /mnt/btrfs_pool/home
mkdir /mnt/btrfs_pool/_btrbk_snap
mount --mkdir -t btrfs -o defaults,noatime,compress-force=zstd:6,ssd,subvol=rootfs /dev/mapper/ArchLinux /mnt/rootfs
btrfs subvolume set-default /mnt/rootfs
mount --mkdir -t btrfs -o defaults,noatime,compress-force=zstd:6,ssd,subvol=home /dev/mapper/ArchLinux /mnt/rootfs/home
btrfs subvolume list /mnt/btrfs_pool
mount --mkdir /dev/nvme0n1p1 /mnt/rootfs/efi
mount --mkdir /dev/nvme0n1p2 /mnt/rootfs/boot

Bootstrapping Arch and initial config

Initial pacstrap

pacman-key --init
pacman-key --populate
pacman -Sy archlinux-keyring
pacstrap -K /mnt/rootfs base linux linux-lts sof-firmware linux-firmware intel-ucode wireless-regdb networkmanager nano man-db man-pages texinfo btrfs-progs dosfstools e2fsprogs openssh gdisk

Initial config

Follow usual manual for initial config, basically:

genfstab -U /mnt >> /mnt/rootfs/etc/fstab
arch-chroot /mnt/rootfs
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
hwclock --systohc

“Savepoint”

In case you break off here, you can continue by booting from the installation medium again and setting up keys etc. (see Booting from the Installation medium), then issue:

cryptsetup open /dev/nvme0n1p3 ArchLinux
mount --mkdir -t btrfs -o defaults,noatime,compress-force=zstd:6,ssd /dev/mapper/ArchLinux /mnt/rootfs
mount -t btrfs -o defaults,noatime,compress-force=zstd:6,ssd,subvol=home /dev/mapper/ArchLinux /mnt/rootfs/home
mount /dev/nvme0n1p1 /mnt/rootfs/efi
mount /dev/nvme0n1p2 /mnt/rootfs/boot
arch-chroot /mnt/rootfs

Set up locales

In /etc/locale.gen, uncomment

en_US.UTF-8 UTF-8
de_DE.UTF-8 UTF-8

Then, run locale-gen.
Finally, in /etc/locale.conf, set:

LANG=de_DE.UTF-8

and in /etc/vconsole.conf, set:

KEYMAP=de-latin1

Prepare the network setup

Edit /etc/hostname, set:

myhostname-i-have-though-about

Then, enable NetworkManager:

systemctl enable NetworkManager

In case you need eduroam, remember to install python-dbus so the CAT tool works:

pacman -S python-dbus

For IPv6 with working DNS with most home routers, you may need:

pacman -S avahi-daemon
systemctl enable avahi-daemon

You may want to enable privacy extensions by creating /etc/sysctl.d/ipv6-priv.conf with content (adapt to interface names!):

# Enable ipv6 privacy extensions
net.ipv6.conf.wlp0s20f3.use_tempaddr = 2
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2

Warning

While dhcpcd is still a great tool, I do not use it with NetworkManager in this way anymore, i.e. I am using the internal DHCP client. Reasoning: It seems that NetworkManager uses randomized MACs for scanning even while connected, which seems to confuse dhcpcd as the MAC changes and it obediently drops the IP address lease due to this, which in turn breaks connectivity.

You might also want to install dhcpcd:

pacman -S dhcpcd

and tell NetworkManager to use it instead of it’s integrated implementation by adding these lines to /etc/NetworkManager/NetworkManager.conf:

[main]
dhcp=dhcpcd

Set up initrd

Edit /etc/mkinitcpio.conf, ensure sd-encrypt is added:

HOOKS=(base systemd autodetect microcode modconf kms keyboard keymap sd-vconsole block sd-encrypt filesystems fsck)

Run mkinitcpio -P.

Set root password

passwd

Add refind_linux.conf for boot configuration

Create the file /boot/refind_linux.conf with content:

"Boot with standard options"            "zswap.enabled=0 rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0 quiet splash rw"
"Boot with standard options no EHT"     "zswap.enabled=0 rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0 quiet splash rw iwlwifi.disable_11be=1"
"Boot without plymouth"                 "zswap.enabled=0 rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0 plymouth.enable=0 disablehooks=plymouth rw"
"Boot to terminal"                      "zswap.enabled=0 rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0 rw systemd.unit=multi-user.target"
"Boot with full UUIDs"                  "rd.luks.name=bdf5159d-ad5d-4d1d-bfac-ce833c92048d=ArchLinux root=/dev/mapper/ArchLinux zswap.enabled=0 rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0 rw"
"Boot to single user mode"              "zswap.enabled=0 rd.luks.options=timeout=0 rootflags=x-systemd.device-timeout=0 rw single"
"Boot with minimal options"             "ro"

Note the special iwlwifi setting is due potential to problems with EHT/MLO support in case of my hardware, the UUID of course needs to be adapted, and zswap is turned off since we are using zram which will be set up later. The timeout settings for LUKS are important in case the password entry is delayed, as otherwise systemd would time out if the password is not entered within 90 seconds and you’d be left with an unbootable system which can only be turned off hard, see this link to the ArchWiki.

Test booting

At this point, refind is not installed yet. As outlined in Booting from the Installation medium, we’ll now use a rEFInd boot loader from an external medium, e.g. a Ventoy boot stick. You should be able to boot with that by now.

This should work fine, if yes, you are ready to continue to the next stage!

Creating a user and making system self-bootable

Activate sshd

systemctl enable --now sshd

Create user

useradd -m olifre
passwd olifre

Remote login

For more comfort (e.g. to set up a system with small screen or keyboard), you may now want to log in via ssh remotely.

Securing SSH

Configure sshd, i.e. copy over pubkey with ssh-copy-id, then set in /etc/ssh/sshd_config:

PasswordAuthentication no

and restart service:

systemctl restart sshd

Install yay

Install yay, see GitHub project (will be needed for easier installation of bootloader / shim-signed):

pacman -Sy vi vim

Need to grant sudo permissions to regular user so yay can be used:

visudo

There, allow wheel users with password to use sudo. Then:

usermod -a -G wheel olifre

Then, as user (ensure to re-login to get group membership!):

sudo pacman -S --needed git base-devel
mkdir AUR
cd AUR
git clone https://aur.archlinux.org/yay.git
cd yay
makepkg -si
yay -Y --devel --save

Configure for speed, create /etc/makepkg.conf.d/multicore.conf with content:

NPROC=8
MAKEFLAGS="-j8"

Adapt the /etc/fstab

In preparation for btrbk and more, you should adapt the /etc/fstab. It should look like this:

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/mapper/root
UUID=8fae96ce-42b0-4933-88ac-f4cdb41155ad       /               btrfs           rw,noatime,commit=60,compress-force=zstd:6,ssd,space_cache=v2,subvol=/rootfs      0 0

# /dev/mapper/root
UUID=8fae96ce-42b0-4933-88ac-f4cdb41155ad       /home           btrfs           rw,noatime,commit=60,compress-force=zstd:6,ssd,space_cache=v2,subvol=/home        0 0

# /dev/mapper/root pool directory
UUID=8fae96ce-42b0-4933-88ac-f4cdb41155ad       /mnt/btrfs_pool btrfs           rw,noatime,commit=60,compress-force=zstd:6,ssd,space_cache=v2,subvolid=5,noauto   0 0

# /dev/nvme0n1p1
UUID=1542-2E81          /efi            vfat            noauto,x-systemd.automount,x-systemd.idle-timeout=1min,rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro   0 2

# /dev/nvme0n1p2
UUID=8fae96ce-42b0-4933-88ac-f4cdb41155ad       /boot           ext4            noauto,commit=60,x-systemd.automount,x-systemd.idle-timeout=1min,rw,relatime      0 2

The important things we added here are the /mnt/btrfs_pool mountpoint and the automount settings for /efi amnd /boot such that they should only be mounted when actually accessed. You may want to regenerate the initrd at this point:

mkinitcpio -P

and you should for sure generate the /mnt/btrfs_pool mountpoint:

mkdir /mnt/btrfs_pool

and of course make sure the UUIDs match your system (use blkid to check)!

Set up discard for crypto devices, and increase performance for SSDs

Be sure you are aware of the security implications! We do this to increase the SSD lifetime. For the performance trick, see the ArchWiki for more details.

cryptsetup --allow-discards --perf-no_read_workqueue --perf-no_write_workqueue --persistent refresh root

Install the bootloader

We can finally install the boot loader. We will be using Secure Boot, with MOK (Machine-Owner Keys), so do as user:

yay -S refind sbsigntools shim-signed

Then, as root, run:

refind-install --shim /usr/share/shim-signed/shimx64.efi --localkeys

Configure refind by editing /efi/EFI/refind/refind.conf. Notably, comment out all those dummy menuentry a the bottom, and make sure to add those settings:

timeout 15
use_nvram false
banner dell_logo.bmp
use_graphics_for osx,linux,windows
showtools shell, bootorder, gdisk, memtest, mok_tool, apple_recovery, windows_recovery, about, hidden_tags, reboot, exit, firmware, fwupdate
fold_linux_kernels false
extra_kernel_version_strings "linux-hardened,linux-rt-lts,linux-zen,linux-lts,linux-rt,linux"
write_systemd_vars true

For ease of maintainability, you may want to add these below to the existing descriptions. Note the write_systemd_vars is critical for GPT automounting (i.e. this will cause the partitions to be detected and mounted automatically from the same disk which holds the ESP), extra_kernel_version_strings is important for the Arch kernel naming scheme, and fold_linux_kernels is helpful in case you also want to install the LTS kernel later on.

Copy over an icon from UEFI BGRT for themeing and a kernel icon:

cp /sys/firmware/acpi/bgrt/image /efi/EFI/refind/dell_logo.bmp
cp /efi/EFI/refind/icons/os_arch.png /boot/vmlinuz-linux.png
cp /efi/EFI/refind/icons/os_arch.png /boot/vmlinuz-linux-lts.png

Now, follow the ArchWiki on how to set up kernel signing, and put the following in the hook file:

keypairs=(/etc/refind.d/keys/refind_local.key /etc/refind.d/keys/refind_local.crt)

Do not forget to make the hook executable! Then, as user:

yay -S linux linux-lts

This will reinstall the kernel and sign it.

Also, add a refind update hook, create /etc/pacman.d/hooks/refind.hook with content:

[Trigger]
Operation=Upgrade
Type=Package
Target=refind

[Action]
Description = Updating rEFInd on ESP
When=PostTransaction
Exec=/usr/bin/refind-install --shim /usr/share/shim-signed/shimx64.efi --localkeys

Now, a reboot should show a Secure Boot warning and MokManager should pop up. In there, enroll the MOK from:

esp/EFI/refind/keys/refind_local.cer

If this works as expected, you will have a booting system! You can unenroll the key of your Ventoy medium (if used) at this point.

System services and base system installation

Set up NTP

systemctl enable --now systemd-timesyncd.service

Set up some base services

These are ArchLinux-specific.

pacman -S reflector
pacman -S pacman-contrib
systemctl enable --now paccache.timer
systemctl enable --now fstrim.timer
systemctl enable --now reflector.timer

You may want to edit /etc/xdg/reflector/reflector.conf to contain your country, e.g.:

--country Germany

Install some basic tools

Some tools from Arch repos:

pacman -S powertop guvcview chromium firefox firefox-i18n-de thunderbird thunderbird-i18n-de nextcloud-client fwupd stress-ng mpv libreoffice-fresh libreoffice-fresh-de power-profiles-daemon keepassxc wl-clipboard xclip waypipe rsync biber python-pygments xorg-xlsclients inkscape screen strace iftop iotop-c htop tcpdump compsize scrcpy emacs-wayland wireshark-qt tcpdump gimp speedtest-cli iperf3 freerdp wakeonlan github-cli fortune-mod syncthing zathura zathura-pdf-poppler zathura-ps zathura-cb usbutils arandr jq yq wev yubikey-personalization-gui yubikey-manager root jupyter-metakernel gnuplot python-matplotlib python-numpy python-pandas python-scipy pv python-pip perf tigervnc networkmanager-openconnect bind hid-tools sshpass ethtool ndisc6 xrootd kdiff3 apptainer diffpdf diffoscope

Then, the groups:

pacman -S texlive

and from AUR:

yay -S syncthingtray-qt6 powerstat afc charliecloud

Configure nano

Edit /etc/nanorc, set:

set cutfromcursor

Install the desktop environment with apps

yay -S plasma-meta kde-applications-meta

Then, execute:

systemctl enable --now plasmalogin

Configure SDDM (not used anymore!)

Warning

I have since migrated to plasma-login-manager, activated above. So this is not needed anymore, It uses wayland and runs rootless out of the box.

Set it up to use wayland (rootless):

mkdir /etc/sddm.conf.d/
cd /etc/sddm.conf.d/

Create /etc/sddm.conf.d/05-base.conf with content:

[Theme]
# Current theme name
Current=breeze

# Cursor theme used in the greeter
CursorTheme=breeze_cursors

Create 10-wayland.conf with content:

[General]
DisplayServer=wayland
GreeterEnvironment=QT_WAYLAND_SHELL_INTEGRATION=layer-shell

[Wayland]
CompositorCommand=kwin_wayland --drm --no-lockscreen --no-global-shortcuts --locale1

Finally, restart it:

systemctl restart sddm.service

Set up Plymouth

yay -S plymouth plymouth-kcm

Then, in /etc/mkinitcpio.conf, add plymouth to HOOKS after systemd, but before sd-encrypt, then:

mkinitcpio -P

You might want to set the theme bgrt which is the ArchLinux default in any case, as can be confirmed with:

plymouth-set-default-theme

On an older system on which BGRT does not receive an image from UEFI, another interesting theme could e.g. be the breeze theme provided by the breeze-plymouth package.

Set up firewalld

Install firewalld, firewall-applet and firewall-config:

yay -S firewalld firewall-applet firewall-config

Note that the integration into KDE is not too helpful at this point, it does not support zones. After installation, activate:

systemctl enable --now firewalld

For further configuration, you can start firewall-config (GUI) and allow syncthing, kdeconnect. Alternatively, you can run:

firewall-cmd --zone=public --add-service syncthing
firewall-cmd --zone=public --add-service kdeconnect
firewall-cmd --runtime-to-permanent

Note ssh and dhcpv6-client are already on by default, see:

firewall-cmd --info-zone=public

Set up zram

Install zram-generator:

yay -S zram-generator

then edit /etc/systemd/zram-generator.conf, should contain (swap and personal scratch space):

[zram0]
zram-size = min(ram / 2, 16384)
compression-algorithm = zstd

[zram1]
zram-size = min(ram / 2, 16384)
mount-point = /var/tmp/olifre
options = X-mount.owner=1000,X-mount.group=1000

Create /etc/sysctl.d/99-vm-zram-parameters.conf with content:

vm.swappiness = 180
vm.watermark_boost_factor = 0
vm.watermark_scale_factor = 125
vm.page-cluster = 0

Set up locate

Install package plocate:

yay -S plocate

Edit /etc/updatedb.conf and set (to include btrfs filesystems):

PRUNE_BIND_MOUNTS = "no"

You may want to enable the timer (but also happens on reboot) or trigger the service for an initial indexing:

systemctl start plocate-updatedb.timer
systemctl start plocate-updatedb.service

Set up logrotate

Install package logrotate:

yay -S logrotate

Edit /etc/logrotate.conf (uncomment / add as-needed):

# better compression when activated for a logfile pattern
compresscmd /usr/bin/xz
uncompresscmd /usr/bin/xz
compressext .xz
compressoptions "-9"

notifempty

Note we already set up things here for the backup we’ll set up later. Create file /etc/logrotate.d/restic with content:

/var/log/restic/*.log {
    weekly
    missingok
    rotate 100
    copytruncate
    minsize 10M
    compress
	dateext
}

You’ll also want to create this:

mkdir /var/log/restic

Create file /etc/logrotate.d/btrbk with content:

/var/log/btrbk.log {
    weekly
    missingok
    rotate 100
    copytruncate
    minsize 10M
    compress
	dateext
}

Enable timer and trigger once:

systemctl enable --now logrotate.timer
systemctl start logrotate

Set up cronie

systemctl enable --now cronie

Activate monthly BTRFS scrub

Results of the scrubs can then be found in the system journal.

systemctl enable --now btrfs-scrub@-.timer

The - is the systemd-escape variant of the / filesystem.

Set up dnsmasq

Install package dnsmasq:

yay -S dnsmasq

then, edit /etc/NetworkManager/NetworkManager.conf and add:

[main]
dns=dnsmasq

For more safe and easy usage of VPNs, you may want to create /etc/NetworkManager/dnsmasq.d/fritzbox with content:

server=/fritz.box/192.168.22.1

(assuming this is your home router hostname and IP). Finally, apply:

systemctl restart NetworkManager

Tinc VPN

Install tinc-pre (from AUR):

yay -S tinc-pre

Execute:

tinc -n homeroute init myhostname

Note that this does 2048 RSA, we want 4096, so:

tinc -n homeroute generate-keys 4096

Now, clean out the old keys, i.e. the commented parts of:

• /etc/tinc/homeroute/{ed25519_key,rsa_key}.priv
• /etc/tinc/homeroute/hosts/myhostname

Copy over config parts from existing tinc cluster, i.e. up/down scripts, other hosts, tinc.conf parts. If you use static addressing, do not forget to adapt IPs in up/down scripts and add a static Address to this host’s config! Finally, copy over the host config file to all other nodes.

Set up Bluetooth

systemctl enable --now bluetooth.service

Set up hardware acceleration and similar

Install packages:

yay -S vulkan-intel vulkan-mesa-layers intel-media-driver libva-utils

Check things work:

vainfo
vulkaninfo

BEES (for btrfs deupe)

Install with:

yay -S bees

Then copy over config:

cp /etc/bees/beesd.conf.sample /etc/bees/beesd_root.conf

and adapt it, set UUID to the UUID returned for lsblk -f. You should also set:

OPTIONS="-v 6"

for reduced verbosity, and you also might want to make the DB_SIZE default setting explicit:

DB_SIZE=$((1024*1024*1024)) # 1G in bytes

If you already have many btrbk snapshots, you may want to reduce the number of snapshots first.

Finally, start the service using the UUID, for example:

systemctl enable --now beesd@b8a34ebc-029a-4c77-ac2c-33290c18b461.service

Check the journal on progress, and also /var/run/bees contains status information.

Note that after the first completed bees run, you might want to make sure to remove old snapshots from pre-bees to ensure they do not remain with duplicated data.

You might also want to check out statistics in /mnt/btrfs_pool/.beeshome (note that /mnt/btrfs_pool is not mounted by default).

Set up Backup

We use a two-fold backup, one is btrbk for local snapshotting and versioning (self-protection) and one is an external backup with restic.

btrbk

Create /usr/local/bin/run-btrbk.sh with content:

#!/bin/bash
UMOUNTAFTER=1
if grep -qs '/mnt/btrfs_pool' /proc/mounts; then
    # Already mounted by user, do not umount after!
    echo "/mnt/btrfs_pool already mounted."
    UMOUNTAFTER=0
else
    echo "Mounting /mnt/btrfs_pool."
    mount /mnt/btrfs_pool
    UMOUNTAFTER=1
fi

if [ $# -eq 0 ]; then
    btrbk --progress -v run
else
    btrbk $@
fi

if [ $UMOUNTAFTER -eq 1 ]; then
    echo "Unmounting /mnt/btrfs_pool."
    umount -l /mnt/btrfs_pool
else
    echo "NOT unmounting /mnt/btrfs_pool."
fi

and make it executable:

chmod +x /usr/local/bin/run-btrbk.sh

Then, install btrbk and mbuffer and copy over the example config:

cp /etc/btrbk/btrbk.conf.example /etc/btrbk/btrbk.conf

and adapt it, uncomment all the “complex examples” and “retention policy” at the end, then add:

snapshot_preserve_min   2d
snapshot_preserve       12h 7d
snapshot_create         always

timestamp_format        long-iso

volume /mnt/btrfs_pool
  subvolume  rootfs
  subvolume  home

Then, create /etc/systemd/system/btrbk.service with content:

[Unit]
Description=btrbk backup

[Service]
Type=oneshot
ExecStart=/usr/local/bin/run-btrbk.sh

Then, create /etc/systemd/system/btrbk.timer with content:

[Unit]
Description=btrbk hourly backup

[Timer]
OnCalendar=hourly
AccuracySec=5min
Persistent=true

[Install]
WantedBy=multi-user.target

Enable all that:

systemctl daemon-reload
systemctl enable --now btrbk.timer

Restic

Install and setup restic for backup.

pacman -S restic

Create dir for configs:

mkdir -p /etc/restic

Within, create /etc/restic/restic_root.conf and /etc/restic/restic_home.conf, follow this scheme:

RESTIC_PASSWORD="secret"
RESTIC_COMPRESSION="max"
AWS_ACCESS_KEY_ID="secret"
AWS_SECRET_ACCESS_KEY="secret"
RESTIC_REPOSITORY="s3:rgw.example.com:7480/my-machine-home"
PRE_BACKUP_COMMAND=""
POST_BACKUP_COMMAND=""
KEEP_WITHIN="2d"
KEEP_LAST=""
KEEP_HOURLY=""
KEEP_DAILY="28"
KEEP_WEEKLY="26"
KEEP_MONTHLY="12"
KEEP_YEARLY="2"
VERBOSITY=1
ONE_FILE_SYSTEM=1
EXCLUDE_CACHES=1
PATH_TO_BACKUP="/home"
EXCLUDE_PATTERNS="'/home/olifre/.cache'"
IEXCLUDE_PATTERNS=""
EXCLUDE_IF_PRESENT_LIST=""

You will of course want to change the secrets and backup server address. For the EXCLUDE_PATTERNS you may want to set for the home backup:

EXCLUDE_PATTERNS="'/home/olifre/.cache' '/home/olifre/some_cloud_sync'"

and for the root backup:

EXCLUDE_PATTERNS="'/home' '/var/cache/pacman' '/root/.cache' '/var/lib/cvmfs' '/mnt/btrfs_pool'"

Make sure the repository names contain different bucket names, e.g. myhostname-home and myhostname-root!

Finally, make sure the files have permissions 0640 for security, and also protect the directory:

chmod 0750 /etc/restic
chmod 0640 /etc/restic/*

Then, create the log directory:

mkdir -p /var/log/restic/

Create two service files, first is /etc/systemd/system/restic-backup@.service with content:

[Unit]
Description=restic backup

[Service]
Type=oneshot
ExecStart=/bin/bash -c "/usr/local/bin/restic_backup.sh backup /etc/restic/restic_%i.conf 2>&1 | cat -v | tee -a /var/log/restic/restic_%i.log > /dev/null"

Second is /etc/systemd/system/restic-check-and-prune@.service:

[Unit]
Description=restic check-and-prune      

[Service]
Type=oneshot
ExecStart=/bin/bash -c "/usr/local/bin/restic_backup.sh check-and-prune /etc/restic/restic_%i.conf 2>&1 | cat -v | tee -a /var/log/restic/restic_%i.log > /dev/null"

The actual script in /usr/local/bin/restic_backup.sh should be created with the following content:

#!/bin/bash

# Check provided parameters
if [ ${#*} -ne 2 ]; then
  echo "Usage: $(basename $0) <mode> <config_file>"
  exit 1
fi

MODE=$1
CONFIG_FILE=$2

if [ "$MODE" != "backup" ] && [ "$MODE" != "check-and-prune" ]; then
  echo "Error, you passed mode = ${MODE}, but be one of: backup, check-and-prune!"
  exit 1
fi

echo "########## START - $(date) ##########"

if [ ! -r ${CONFIG_FILE} ]; then
  echo "Config file ${CONFIG_FILE} can not be accessed / does not exist!"
  exit 1
fi

set -o allexport
. ${CONFIG_FILE}
set +o allexport

# Ensure HOME is set (needed for cache).
export HOME=/root

# Init repo if absent.
restic snapshots > /dev/null 2> /dev/null
if [ ! $? -eq 0 ]; then
  restic init
  echo "Restic repository created at \"${RESTIC_REPOSITORY}\"."
else
  echo "Using existing restic repository at \"${RESTIC_REPOSITORY}\"."
fi

if [ "$MODE" = "backup" ]; then
  # Handle actual backup commandline arguments.
  RESTIC_BACKUP_PARS=()
  if [ "x${ONE_FILE_SYSTEM}" = "x1" ]; then
    RESTIC_BACKUP_PARS+=("--one-file-system")
  fi
  if [ "x${EXCLUDE_CACHES}" = "x1" ]; then
    RESTIC_BACKUP_PARS+=("--exclude-caches")
  fi
  if [ -n "${EXCLUDE_PATTERNS}" ]; then
    eval "excl_dir_array=($EXCLUDE_PATTERNS)"
    for excl_dir in "${excl_dir_array[@]}"; do
      RESTIC_BACKUP_PARS+=("--exclude")
      RESTIC_BACKUP_PARS+=("${excl_dir}")
    done
  fi
  if [ -n "${IEXCLUDE_PATTERNS}" ]; then
    eval "iexcl_dir_array=($IEXCLUDE_PATTERNS)"
    for iexcl_dir in "${iexcl_dir_array[@]}"; do
      RESTIC_BACKUP_PARS+=("--iexclude")
      RESTIC_BACKUP_PARS+=("${iexcl_dir}")
    done
  fi
  if [ -n "${EXCLUDE_IF_PRESENT_LIST}" ]; then
    eval "excl_if_present_array=($EXCLUDE_IF_PRESENT_LIST)"
    for excl_if_present in "${excl_if_present_array[@]}"; do
      RESTIC_BACKUP_PARS+=("--exclude-if-present")
      RESTIC_BACKUP_PARS+=("${excl_if_present}")
    done
  fi
  
  # Now finally the actual backup.
  SECONDS=0
  echo "Starting backup of \"${PATH_TO_BACKUP}\" to \"${RESTIC_REPOSITORY}\" at $(date)..."
  if [ -n "${PRE_BACKUP_COMMAND}" ]; then
    echo "Running pre-backup-command \"${PRE_BACKUP_COMMAND}\"..."
    $PRE_BACKUP_COMMAND
    echo "Done!"
  fi
  echo "Running restic..."
  restic --verbose=${VERBOSITY} backup "${RESTIC_BACKUP_PARS[@]}" ${PATH_TO_BACKUP}
  echo "Done!"
  if [ -n "${POST_BACKUP_COMMAND}" ]; then
    echo "Running post-backup-command \"${POST_BACKUP_COMMAND}\"..."
    $POST_BACKUP_COMMAND
    echo "Done!"
  fi
  echo "Backup finished at $(date) (after ${SECONDS} seconds)."
  
  # Forget metadata for old snapshots.
  RESTIC_FORGET_PARS=()
  if [ -n "${KEEP_WITHIN}" ]; then
    RESTIC_FORGET_PARS+=("--keep-within" "${KEEP_WITHIN}")
  fi
  if [ -n "${KEEP_LAST}" ]; then
    RESTIC_FORGET_PARS+=("--keep-last" "${KEEP_LAST}")
  fi
  if [ -n "${KEEP_HOURLY}" ]; then
    RESTIC_FORGET_PARS+=("--keep-hourly" "${KEEP_HOURLY}")
  fi
  if [ -n "${KEEP_DAILY}" ]; then
    RESTIC_FORGET_PARS+=("--keep-daily" "${KEEP_DAILY}")
  fi
  if [ -n "${KEEP_WEEKLY}" ]; then
    RESTIC_FORGET_PARS+=("--keep-weekly" "${KEEP_WEEKLY}")
  fi
  if [ -n "${KEEP_MONTHLY}" ]; then
    RESTIC_FORGET_PARS+=("--keep-monthly" "${KEEP_MONTHLY}")
  fi
  if [ -n "${KEEP_YEARLY}" ]; then
    RESTIC_FORGET_PARS+=("--keep-yearly" "${KEEP_YEARLY}")
  fi
  SECONDS=0
  echo "Starting forgetting of old snapshots at $(date)..."
  restic --verbose=${VERBOSITY} forget --cleanup-cache "${RESTIC_FORGET_PARS[@]}"
  echo "Forgetting of old snapshots finished at $(date) (after ${SECONDS} seconds)."
fi

if [ "$MODE" = "check-and-prune" ]; then
  SECONDS=0
  echo "Start of checking at $(date)."
  restic --verbose=${VERBOSITY} check --retry-lock 1h --read-data
  CHECK_RES=$?
  if [ $CHECK_RES -ne 0 ]; then
    echo "Error: Check was not successful, exiting here, not pruning!"
    exit 1
  fi
  echo "End of successful check at $(date). Duration: $SECONDS seconds."

  SECONDS=0
  echo "Start of pruning at $(date)."
  restic --verbose=${VERBOSITY} prune --repack-small
  echo "End of pruning at $(date). Duration: $SECONDS seconds."
fi

echo "########## STOP - $(date) ##########"

Afterwards, make it executable:

chmod +x /usr/local/bin/restic_backup.sh

Note the following assumes we will call the restic-backup@.service once every day via a timer (created below), which also marks snapshots for forgetting, but never prunes, as this might destroy data e.g. in case of bad RAM or otherwise corrupted backups.
For that, there is restic-check-and-prune@.service which can be one-shotted manually when there is a stable connection. This will likely not be used on the road with a laptop, as it reads back all data before pruning. Make sure to check the logs when running this.

Now, create the timer for the backup, i.e. /etc/systemd/system/restic-backup@.timer:

[Unit]
Description=restic daily backup

[Timer]
OnCalendar=*-*-* 23:15:00
AccuracySec=5min
Persistent=true

[Install]
WantedBy=multi-user.target

Enable things:

systemctl daemon-reload
systemctl enable --now restic-backup@root.timer
systemctl enable --now restic-backup@home.timer

You may want to trigger the service units manually for the initial backup.

Hardware specifics

IPU 7 webcam

This follows along the Guide on the ArchWiki for a specific laptop model.

Before starting, you’ll want to install libcamera-tools to check whether the situation has improved in the meantime:

yay -S libcamera-tools

You can use:

cam -l
qcam

for testing things.

If this does not show something useful (see the Guide on the ArchWiki, install DKMS and kernel headers so DKMS can be used:

yay -S dkms linux-lts-headers linux-headers

Then, install Intel Vision Drivers (DKMS) from AUR:

yay -S intel-vision-drivers-dkms-git

Finally, create /etc/modules-load.d/intel_cvs.conf with content:

intel_cvs

and run, just to be safe:

mkinitcpio -P

and reboot. The commands from above should work now, you should see a camera picture with non-ideal quality.

Now, install the tooling necessary to use the camera in actual applications.

FIXME, this is still work in progress!

Set up user environment

Set up shell

yay -S zsh
chsh

Choose /usr/bin/zsh.

Then, install oh-my-zsh (see (upstream docs)[https://ohmyz.sh/#install]).

Also, install from AUR:

yay -S autojump

In ~/.zshrc, activate these plugins:

plugins=(git github git-extras gitignore svn lol catimg compleat wakeonlan battery autojump colorize web-search dirhistory screen rand-quote hitchhiker)

In case you are using ssh-agent via systemd for KeepassXC for example, add to ~/.zshrc:

export SSH_AUTH_SOCK=$XDG_RUNTIME_DIR/ssh-agent.socket

This should trigger the user unit via ssh-agent.socket.

You may also want to copy over more things from ~/.oh-my-zsh/custom of one of your existing machines, e.g. themes etc.

You might also want to add to your .zshrc:

export SYNCTEX_EDITOR="emacsclient --no-wait +%{line} %{input}"

export LESSCOLORIZER='pygmentize -O style=solarized-dark'

You might also want to install:

yay -S zsh-syntax-highlighting zsh-autosuggestions

and activate these with:

cd ~/.oh-my-zsh/custom
ln -s /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.plugin.zsh .
ln -s /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh .

and configure by creating ~/.oh-my-zsh/custom/zsh-autosuggestions-config.zsh with content:

ZSH_AUTOSUGGEST_HIGHLIGHT_STYLE='fg=60'
#ZSH_AUTOSUGGEST_USE_ASYNC=true
ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE=40

You may also want to create ~/bin and copy over files you already have there, and then add to .zshrc:

export PATH=$PATH:~/bin/

In case you are using Grid tools, you might also want to set:

export RUCIO_ACCOUNT=MYCERNACCOUNT

Keep extended ZSH history

Create ~/.oh-my-zsh/custom/history.zsh with content:

HISTFILE=~/.histfile
HISTSIZE=2000000
SAVEHIST=2000000

## Extended history.
## Instead of just a list of commands, append it with this:
## `:<beginning time since epoch>:<elapsed seconds>:<command>'.
setopt extended_history

You may want to save ZSH history periodically, e.g. via Syncthing, useful for easier recovery or to search through history from other machines you use.

Run crontab -e as user, and add a line like:

*/30   *  * * * cp -a /home/olifre/.histfile /home/olifre/Sync/Sync/ZSH-History/histfile-myhostname

Syncthing setup

Enable the syncthing-tray-qt6 widget by enabling the tray icon, which automatically causes it to autostart and also starts the setup wizard.

You might also want to do (should in principle be done by the tray widget):

systemctl --user --now enable syncthing

Set up ssh-agent

systemctl enable --user ssh-agent

Edit ~/.zshrc, set:

export SSH_AUTH_SOCK=$XDG_RUNTIME_DIR/ssh-agent.socket

Also, edit ~/.config/plasma-workspace/env/ssh-agent.sh (may need to create directory) and add the same line.

Re-login after this.

WireGuard VPN

yay -S wireguard-tools
nmcli conn import type wireguard file somefilewithoutspaces.conf

Note that you may want to adapt the config in NetworkManager graphically afterwards, as VPNs imported this way are autoconnect / always-on by default.

Other things you might want to do

• Firefox plugins such as:
  • FoxyProxy
  • Greasemonkey
  • Stylus
  • KeePassXC-Browser
  • Plasma Integration
  • Privacy Badger
  • uBlock Origin
  • User-Agent Switcher
  • FoxReplace
  • Form History Control
  • DownThemAll!
• Firefox configuration:
  • User Scripts
  • User Styles
  • May want to set up opening of last opened tabs, disable search results before history results, set as default browser.
  • May also want to activate Passkey support in the KeepassXC browser plugin.
  • If you are working with the LHC computing grid, you may want to copy over ~/.globus and add the certificate to Firefox, and enable a master password there.
  • You may want to disable password saving in Firefox (in case you use KeepassXC).
• In KDE/Plasma, you may want to:
  • Create more virtual desktop, e.g. 6 in total, and set up navigation to rotate.
  • You may want to configure the virtual desktop switcher to show icons for apps.
  • You may also want to disable gliding of the background image in the glide effect in virtual desktop setup.
  • You may want to configure the application switcher with icons not to limit itself to applications from the current virtual desktop.
  • Edit window decoration => window title bar buttons, add “always on top” and “on all desktops”.
  • In keyboard setup, enable the compose key, e.g. re-use the “menu” key for that.
  • You might want to change “Desktop” settings and add mouse bindings: Vertical Scroll to switch the vertical desktop and middle click to switch windows.
  • You might also wantr to reconfigure the calendar to show calendar week numbers and important dates, e.g. astronomical events, regional holidays etc.

Set up Git

Install git-lfs:

yay -S git-lfs
git lfs install

You will likely want to use a ~/.gitconfig like this:

[user]
        email = freyermuth@physik.uni-bonn.de
        name = Oliver Freyermuth
		signingkey = PUTYOURGPGKEYHERE
[color]
        ui = auto
        diff = auto
        branch = auto
        interactive = auto
        status = auto
[credential]
        helper = cache --timeout=3600
[core]
        compression = 9
        bigfilethreshold = 180m
        whitespace = blank-at-eof,blank-at-eol,cr-at-eol,indent-with-non-tab,space-before-tab
[push]
        default = simple
        followTags = true
[merge]
[grep]
        lineNumber = on
[diff "bin"]
        textconv = hexdump -v -C
[format]
        pretty = fuller
[filter "lfs"]
        required = true
        clean = git-lfs clean -- %f
        smudge = git-lfs smudge -- %f
        process = git-lfs filter-process

Set up fonts

yay -S adobe-source-code-pro-fonts

I use this for example in Emacs.

Set up zathura

Create ~/.config/zathura/zathurarc with content:

#set recolor-keephue true
set recolor-darkcolor "#93A1A1"
set recolor-lightcolor "#002B36"
#set recolor-reverse-video true
#set recolor true
#set render-loading false
set render-loading true
#set render-loading-bg #000000
#set render-loading-fg #000000
set page-cache-size 300
set page-store-threshold 300
#set smooth-scroll true
set font "Source Code Pro normal 7"
set first-page-column 1:2

Configure yt-dlp

Create ~/.config/yt-dlp/config with content:

--mtime

Set up conky

Install conky:

yay -S conky

Restore your favourite configuration to ~/.config/conky, then create ~/.config/autostart/conky.desktop with content:

[Desktop Entry]
Type=Application
Name=conky
Exec=conky --daemonize --pause=5
StartupNotify=false
Terminal=false

I am using this theme repo.

Set up Konsole

You may want to copy the built-in profile, then set up infinite history, and also set a transparency of 20 % for the terminal.

Set up printing

In case you use a central CUPS server, things are easy:

mkdir ~/.cups

then, create ~/.cups/client.conf with content:

Encryption IfRequested
ServerName cups.example.com

and ~/.cups/lpoptions with content:

Default my-favourite-printer

Site-specific setups

Kerberos 5 setup

Edit /etc/krb5.conf and make sure it contains the following (note mdbook sadly converts all tabs to spaces, you may want to use tabs here):

[libdefaults]
	default_realm = UNI-BONN.DE
	forwardable = true
	proxiable = true
	default_ccache_name = KEYRING:persistent:%{uid}
	ticket_lifetime = 1day
	renew_lifetime = 7days

[realms]
	CERN.CH = {
		default_domain = cern.ch
		kdc = cerndc.cern.ch
		admin_server = cerndc.cern.ch
		kpasswd_server = cerndc.cern.ch
	}
	UNI-BONN.DE = {
		kdc = kdc.uni-bonn.de
		kdc = kdc1.uni-bonn.de
		kdc = localhost:8987
		default_domain = uni-bonn.de
		admin_server = kdc.uni-bonn.de
		admin_server = kdc1.uni-bonn.de
		admin_server = localhost:8987
		#port 749 ?
	}
[domain_realm]
	cern.ch = CERN.CH
	.cern.ch = CERN.CH
	uni-bonn.de = UNI-BONN.DE
	.uni-bonn.de = UNI-BONN.DE
	rhrz.uni-bonn.de = UNI-BONN.DE
	.rhrz.uni-bonn.de = UNI-BONN.DE
	
[logging]
#       kdc = CONSOLE

[appdefaults]
login = {
	forwardable = true
	krb5_run_aklog = true
	krb5_get_tickets = true
	krb4_get_tickets = false
	krb4_convert = false
}
kinit = {
	forwardable = true
	proxiable = true
	krb5_run_aklog = true
}

You can of course leave existing domain configuration in.

Note that the localhost part here is for forwarding the KDC via socat.

Firefox

Set network.negotiate-auth.trusted-uris in about:config to:

https://zabbix.physik.uni-bonn.de,https://zabbix-test.physik.uni-bonn.de,https://web.physik.uni-bonn.de,https://web-dev.physik.uni-bonn.de,https://login.cern.ch,https://auth.cern.ch

OIDC Agent setup

Install the package:

yay -S oidc-agent

then enable this in ~/.zshrc by adding:

eval `oidc-agent-service use` > /dev/null

You may also want to create the file ~/.config/oidc-agent/custom_parameters.config with content:

[
  {
    "parameter": "claims_in_tokens",
    "value": "id_token token",
    "for_issuer": [
      "https://login.helmholtz.de/oauth2",
      "https://login-dev.helmholtz.de/oauth2"
    ],
    "request": [
      "auth_url"
    ]
  }
]

depending on your use case.

Grid VOMS tools and Java JDK

Java JRE may be required by some GUIs, and JDK to build some:

yay -S jdk-openjdk

Also install VOMS tools and grid certificate authorities:

yay -S voms-clients igtf-trust-anchors

CVMFS

Install package:

yay -S cvmfs

Edit /etc/cvmfs/default.local and add:

CVMFS_REPOSITORIES=cvmfs-config.cern.ch,atlas.cern.ch,atlas-nightlies.cern.ch,atlas-condb.cern.ch,belle.cern.ch,grid.cern.ch,sft.cern.ch,sft-nightlies.cern.ch,lhcb.cern.ch,lhcbdev.cern.ch,unpacked.cern.ch
CVMFS_QUOTA_LIMIT='2048'
CVMFS_HTTP_PROXY='http://somesquidproxy-i-can-se.example.com:3128;DIRECT'

Then, execute as root in a separate terminal:

. /etc/cvmfs/default.local
for A in $(echo $CVMFS_REPOSITORIES | tr ',' ' '); do
  echo $A;
  echo "$A /cvmfs/$A cvmfs noauto,x-systemd.automount,x-systemd.requires=network-online.target,x-systemd.idle-timeout=5min,x-systemd.requires-mounts-for=/cvmfs/cvmfs-config.cern.ch,_netdev 0 0" >> /etc/fstab;
  mkdir /cvmfs/$A;
done

You might want to organize /etc/fstab a bit after this, for example, add a headline # CVMFS and add some empty lines before and after the mounts. You should also remove the x-systemd.requires-mounts-for=/cvmfs/cvmfs-config.cern.ch from the config repository itself! Afterwards, finalize and test things with:

systemctl daemon-reload
for A in /cvmfs/*; do mount $A; done

You may also want to regenerate the initrd just in case:;

mkinitcpio -P

and also reboot to get actual automounting working (this will enable the .automount units).

Finally, configure the user part. For this, create the file ~/.oh-my-zsh/custom/setupATLAS.zsh with content:

setupATLAS() {
	if ls /cvmfs/atlas.cern.ch > /dev/null 2>&1; then
		echo "Setting up CVMFS environment..."
		export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
		# ArchLinux, force container usage.
		export ALRB_containerSiteOnly=YES
		source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh "$@"
		return $?
	else
		echo " ERROR: CVMFS not available on this host"
		return 1
	fi
}

and the file ~/.oh-my-zsh/custom/setupBelle.zsh with content:

setupBelle() {
	if ls /cvmfs/belle.cern.ch > /dev/null 2>&1; then
		echo "Setting up CVMFS environment..."
		source /cvmfs/belle.cern.ch/tools/b2setup
		return $?
	else
		echo " ERROR: CVMFS not available on this host"
		return 1
	fi
}

Regular system maintenance

Staying informed

You will want to subscribe to the very low traffic Arch-announce mailing list, which announces important packaging changes.

Installing updates and purging things

yay
yay -Qtdq | yay -Rns -

The last line will for example remove build-only dependencies.

Prune old Restic backups

As outlined in Set up Backup: Restic, you should regularly run the following (when there is a good and stable connection, as this will read back all data):

systemctl start restic-check-and-prune@root.service
systemctl start restic-check-and-prune@home.service

Check logs after this!

Power Saving

Increase dirty writeback

Note this may lead to data loss up to 60 seconds. We have already adapted the commit time in /etc/fstab earlier.

Create /etc/sysctl.d/dirty.conf with content:

vm.dirty_writeback_centisecs = 6000

You may want to run mkinitcpio -P afterwards.

Enable WiFi power-save

Create /etc/modprobe.d/iwlwifi.conf with content:

options iwlwifi power_save=1

You may want to run mkinitcpio -P afterwards.

Activate PCI Runtime Power Management

Create /etc/udev/rules.d/pci_pm.rules with content:

SUBSYSTEM=="pci", ATTR{power/control}="auto"
SUBSYSTEM=="ata_port", KERNEL=="ata*", ATTR{device/power/control}="auto"

You may want to run mkinitcpio -P afterwards.

Activate USB power saving

Create /etc/udev/rules.d/50-usb_power_save.rules with content:

# blacklist for usb autosuspend
# ACTION=="add", SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", ATTR{idProduct}=="9205", GOTO="power_usb_rules_end"

ACTION=="add", SUBSYSTEM=="usb", TEST=="power/control", ATTR{power/control}="auto"
LABEL="power_usb_rules_end"

Note that the blacklist can be used in case you encounter a device which has problems with that. You may want to run mkinitcpio -P afterwards.

Intel Low-Power-Mode Daemon

Install service:

yay -S intel-lpmd

Enable it:

systemctl enable --now intel_lpmd

Still need to determine whether that helps (may need to set intel_lpmd_control AUTO or write an actual XML config).

Check in:

systemctl status intel-lpmd

then copy over config to more matching config, e.g.:

cp /etc/intel_lpmd/intel_lpmd_config_F6_M189.xml /etc/intel_lpmd/intel_lpmd_config_F6_M189_T17.xml

and edit to your liking, for example:

<PerformanceDef>0</PerformanceDef>
<BalancedDef>0</BalancedDef>
<PowersaverDef>1</PowersaverDef>

Note: Switching to powersave with this example config means only the E cores will be used. In case performance is not required, this can increase runtime noticeably.