Linux

Table of contents

1. Setup
   1. Setup SSH
      1. Connect to SSH via Cloudflare Tunnel
   2. Install Firewall
   3. Install FTP server
   4. Cron - Job scheduler
   5. Cockpit - Remote administration of server
      1. Advanced configuration
   6. Install on VirtualBox
2. Tips & Tricks
   1. Command-line kung-fu
      1. Screen Terminal Multiplexer
      2. Vi Text Editor
   2. CPU & Processes
   3. Services with systemctl
   4. Disk & Data
      1. View SMART disk data
      2. Test read/write speed
      3. Format & mount a USB stick
   5. Files
      1. Virtual Systems: Mount QCoW2
   6. Network & Internet
      1. Set up Let’s Encrypt with Cloudflare DNS
   7. Users & Groups
   8. OS-related
   9. Hardware-related
      1. Power & Battery
      2. BIOS
3. Security
4. Recover
   1. Check logs
   2. Bootloader
5. KVM - Virtual Machines
   1. Setup KVM
   2. Manage VM

---

Please note: I mainly use Ubuntu Server, so your mileage may vary with other distros.

Setup

• On a laptop: disable standby on lid close: in /etc/systemd/logind.conf, uncomment and change HandleLidSwitch=ignore

Setup SSH

• Config lives in /etc/ssh/sshd_config
• PermitRootLogin no to deny root to log in
• AllowUsers toto to only allow toto to log in
• sudo systemctl restart ssh to restart the service

Connect to SSH via Cloudflare Tunnel

1. In the Cloudflare Zero Trust dashboard
   1. Set up an Access Application with an associated access rule
   2. Set up an SSH Tunnel to the Linux server
2. On the target client
   1. Grab the latest release of cloudlared
   2. Set up a local proxy connected to your hostname cloudflared-windows-amd64.exe access ssh --hostname example.com --url localhost:1234
   3. Connect with your favorite SSH client to [email protected]:1234

Install Firewall

• A good tutorial here
• Install sudo apt install ufw
• Configuration
  • Check status & list configuration sudo ufw status
  • Set default incoming to deny ufw default deny incoming and ufw default allow outgoing for instance
  • Allow a port sudo ufw allow PORT
  • Remove an allowance sudo ufw delete allow PORT or sudo ufw delete RULE_NUMBER
  • Rate limit a port (>6 connections within 30 sec) sudo ufw limit PORT
  • PORT can be replaced with a known app like SSH or specific protocol like PORT/tcp
  • Enable sudo ufw enable
  • On alpine rc-update add ufw default
• Verify
  • Check added rules sudo ufw show added
  • Check existing rules with numbers sudo ufw status numbered
• Enable sudo ufw enable

Install FTP server

1. sudo apt install vsftpd
2. sudo nano /etc/vsftpd.conf to edit configuration
3. sudo ufw allow 21
4. sudo systemctl restart vsftpd

Cron - Job scheduler

• Edit cron of a user sudo crontab -e -u USER (-u to edit specific user)
• Check if the cron expression is correct: crontab guru
• Useful commands:
  • @reboot COMMAND

Cockpit - Remote administration of server

1. sudo apt-get install cockpit and with:
   • cockpit-machines for KVM
   • cockpit-docker for Docker
2. sudo ufw allow 9090
3. sudo systemctl restart cockpit.socket
4. If you cannot update packages, network manager may be in use, check with nmcli d
   • sudo systemctl disable network-manager.service
   • sudo systemctl stop network-manager.service

Advanced configuration

• See this guide to change port. Create file and directory if needed.
• Configuration file is /etc/cockpit/cockpit.conf, not created by default - see reference
• Setup with Cloudflare Tunnel - Based on this blog post.
  1. Setup a Let’s Encrypt certificate below
  2. Setup a copy script - See example in Nextcloud

      #!/bin/sh
      FQDN="DOMAIN.COM"
     
      echo "SSL certificates renewed"
      cp /etc/letsencrypt/live/$FQDN/fullchain.pem /etc/cockpit/ws-certs.d/$FQDN.crt
      cp /etc/letsencrypt/live/$FQDN/privkey.pem /etc/cockpit/ws-certs.d/$FQDN.key
      chown cockpit-ws:cockpit-ws /etc/cockpit/ws-certs.d/$FQDN.crt /etc/cockpit/ws-certs.d/$FQDN.key
     
      echo "Restarting Cockpit"
      systemctl restart cockpit
     

  3. Setup /etc/cockpit/cockpit.conf

      [WebService]
      Origins = https://URL:PORT
      #ProtocolHeader = X-Forwarded-Proto
      AllowUnencrypted = false
     
      [Session]
      IdleTimeout = 10
     

  4. Setup Cloudflare Tunnel and Access

Install on VirtualBox

1. Install gcc make perl packages
2. Install Guest addition
3. Disable animations gsettings set org.gnome.desktop.interface enable-animations false
4. Allow user to access Shared Folder sudo adduser $USER vboxsf

Tips & Tricks

• Date & Time
  • timedatectl to check
  • sudo timedatectl set-timezone TZ

Command-line kung-fu

• less to redirect output to a pager
• watch to watch regularly an output
• | grep x to pipe to grep which is going to select lines with x
• clear to clear the command line
• Check this Ars introduction about redirection, grep, sed, awk.

Screen Terminal Multiplexer

Complete guide here

• Start screen
• List sessions ID screen -ls
• In a session - Hotkey CTRL+A then:
  • d: Detach
  • [: Copy mode
    • Page-Up to scroll up, Page-Down to scroll down
    • Space to start selection and Space again to stop it
  • ]: Paste mode
  • :sessionname <Your_session_name>: Rename
• Detach screen -d session_ID
• Reattach screen -r session_ID

Vi Text Editor

• Command mode: ESC, then all commands starts with :
  • :w : save
  • :x : save and quit
  • :q : quit
  • :q!: quit and discard changes
• Text insert
  • i : insert before cursor
  • a : append after cursor
• Text delete
  • x : delete character to the right of the cursor
  • X : delete character to the left of the cursor

CPU & Processes

• Check system stats
  • htop for processes
  • nmon for the whole system
• Health
  • Check CPU frequency cat /proc/cpuinfo
• System information dmidecode

Services with systemctl

• Service management
  • sudo systemctl restart SERVICE
  • sudo systemctl start SERVICE
  • sudo systemctl stop SERVICE
  • sudo systemctl kill SERVICE
• Service enable/disable
  • sudo systemctl enable SERVICE
  • sudo systemctl disable SERVICE
  • sudo systemctl is-enabled SERVICE
• Services mask/unmask
  • sudo systemctl unmask SERVICE
  • sudo systemctl mask SERVICE
• Service status
  • sudo systemctl list-timers
  • sudo systemctl status SERVICE

Disk & Data

• Check free disk space df -h
• Tree file & folder with space taken ncdu
• Check for partition corruption fsck /dev/sdaXX00
• Stop a disk
  1. Unmount sudo umount /dev/sdXX
  2. Spin down sudo hdparm -Y /dev/sdXX
• Erase a disk sudo dd if=/dev/urandom of=/dev/sdXX bs=10M - Source

View SMART disk data

Source

1. Install smartmontools
2. View SMART data & status sudo smartctl -i /dev/sdXX
3. Find estimate & current SMART test sudo smartctl -c /dev/sdXX
4. Launch a test: sudo smartctl -t short /dev/sdXX
   • Possible options instead of short:
   • long
   • conveyance (after transport) for ATA disks

   • If the HDD go to sleep (especially true for USB ones), try the following:

       while true; do
           dd if=/dev/sdXX iflag=direct count=1 of=/dev/null
           sleep 60
       done
     

5. Read the result sudo smartctl -l selftest /dev/sdXX

Test read/write speed

Source

1. sync; dd if=/dev/zero of=tempfile bs=1M count=256; sync: write to tempfile 256 Mb
2. sudo /sbin/sysctl -w vm.drop_caches=3: clear the cache
3. dd if=tempfile of=/dev/null bs=1M count=256: read tempfile

Format & mount a USB stick

1. Several options to see devices
   • sudo blkid to locate block devices
   • sudo lsblk to list mounted block devices
   • ls /dev to check where the USB stick is mounted
2. sudo fdisk /dev/sdXX
   1. p to list existing partition
   2. g to create a new GPT partition table or o for a DOS partition table
   3. n to create a new partition
   4. w write to disk and exit
3. sudo mkfs.ext4 /dev/sdXX00 to create an ext4 partition
4. sudo mkdir /media/usbdrive to create a directory that will host the partition
   • Choose /mnt for temporary mounts
   • Choose /media for automatics mounts
5. sudo mount /dev/sdXX00 /media/usbdrive -o umask=000 to mount the USB key with all user access
6. sudo umount /dev/sdXX00 to unmount
7. If you want to make it permanent
   • Edit sudo nano /etc/fstab
   • Add a line /dev/sdXX00 /media/usbdrive ext4 defaults 0 0
   • (Check how fstab works here)

Files

• zip -r directoryname.zip ./ to zip current directory
• tar -xvf file.tar.gz to extract tarball and zip (x for extract, v for verbose, f for file)
• find / -name filename to find filename in entire system (/) or active folder (.), and sudo to avoid permissions problems

Virtual Systems: Mount QCoW2

1. sudo apt install libguestfs-tools to install required tool
2. sudo guestmount -a /var/lib/libvirt/images/IMAGE.qcow2 -m DEVICE --ro /mnt/FOLDER
   • DEVICE being the partition within this image. Make it bogus so a list will be shown
   • --ro if you only want read-only
3. sudo guestunmount /mnt

Network & Internet

• View current information (IP, netmask, …) ifconfig
  • Assign an IP to a network sudo ifconfig wlan0 192.168.65.1 netmask 255.255.255.0 up for instance
• Edit network configuration
  • with netplan /etc/netplan/00-installer-config.yaml

• Disable IPv6 - Source

    sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
    sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
    sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
  

• Check Wi-Fi adapter status:
  • rfkill list wlan to list status
  • rfkill unblock wlan to unblock

Set up Let’s Encrypt with Cloudflare DNS

• Source
• See this documentation on Cloudflare certbot
  • cloudflare.ini can be placed in /etc/cloudflared
  • Define proper permission with chmod 400
• Steps
  1. sudo snap install --classic certbot certbot-dns-cloudflare You may need to proceed with additional commands as explained in the console
  2. sudo ln -s /snap/bin/certbot /usr/bin/certbot
  3. sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini -d <EXAMPLE.COM>
  4. Automated renewal should be set up by the snap, check systemctl list-timers

Users & Groups

• Users
  • useradd -m USERNAME add a new user with a Home directory (-m)
  • passwd USERNAME define a password
  • id $user to get user PUID & GUID
• Groups
  • usermod -a -G GROUP $USER to add self to GROUP
  • cat /etc/group to list all groups
• Setup SFTP for without Shell: nice tutorial from Digital Ocean

OS-related

• Know which OS, flavor or version you are running cat /etc/os-release
• Install fonts, check package fonts-liberation - can help you to fix Fontconfig error: Cannot load default config file

Hardware-related

• Sometime reboot is not accessible … then use systemctl reboot

Power & Battery

• Check system temperature
  1. sudo apt install lm-sensors
  2. sudo apt-get install lm-sensors
  3. sudo /etc/init.d/kmod start
  4. sensors
• Check status & stats about battery
  1. List paths upower -e
  2. Get status upower -i <path>
• Plan a reboot: shutdown -r 0:00 to restart at midnight for instance

BIOS

• Check BIOS version sudo dmidecode | less

Security

Source:OVH, DigitalOcean, DigitalOcean

• Change all account passwords
• Regarding SSH
  • Disable all unnecessary users logins, like root - Example
  • Change default port (between 49152 and 65535) - Generator
  • Use SSH Keys instead of passwords
• Regarding network
  • Configure firewall like ufw
  • Configure fail2ban
  • Configure an Intrusion Detection System - Example
• Disable unneeded services - Example, sudo ss -atpu
• Implement unattended upgrades and livepatch - Example
• Regularly
  • Update software
  • Check logs
  • Check for malware - Example: maldet
  • Make backups

Recover

Check logs

• System messages: cat /var/log/messages | tail -n 50

Bootloader

Try Super Grub2 Disk

KVM - Virtual Machines

Setup KVM

Based on this great tutorial by OSTechnix.

1. Install QEMU and some other libs
   1. sudo apt install qemu qemu-kvm libvirt-clients libvirt-daemon-system virtinst bridge-utils
   2. systemctl status libvirtd - must be enabled and started
2. Configure network bridge
   1. ip a list existing network, including virbr which is the preinstalled bridge
   2. Disable netfilter on bridge

      1. Create /etc/sysctl.d/bridge.conf and add the following

          net.bridge.bridge-nf-call-ip6tables=0
          net.bridge.bridge-nf-call-iptables=0
          net.bridge.bridge-nf-call-arptables=0
         

      2. Create /etc/udev/rules.d/99-bridge.rules and add ACTION=="add", SUBSYSTEM=="module", KERNEL=="br_netfilter", RUN+="/sbin/sysctl -p /etc/sysctl.d/bridge.conf"

   3. Remove default KVM bridge
      1. virsh net-destroy default
      2. virsh net-undefine default
      3. ip link check that virbr interfaces are gone
   4. Edit /etc/netplan/00-installer-config.yaml
      1. Backup first

      2. Add a new section under defined interface - Careful about the indentation

          network:
            ethernets:
              enp0sXYZ:
                #config
            bridges:
              br0:
                interfaces: [ enp0s3 ]
                addresses: [192.168.225.52/24]
                gateway4: 192.168.225.1
                mtu: 1500
                nameservers:
                  addresses: [8.8.8.8,8.8.4.4]
                parameters:
                  stp: true
                  forward-delay: 4
                dhcp4: no
                dhcp6: no
            version: 2
         

      3. sudo netplan --debug apply apply
      4. ip a to check, also brctl show br0
   5. Configure KVM to use this bridge

      1. Edit host-bridge.xml somewhere and add:

          <network>
            <name>host-bridge</name>
            <forward mode="bridge"/>
            <bridge name="br0"/>
          </network>
         

      2. virsh net-define host-bridge.xml
      3. virsh net-start host-bridge
      4. virsh net-autostart host-bridge
      5. virsh net-list --all to check

Manage VM

• virt-install to create a Virtual Machine
• sudo virsh --all …
  • list to list VM
  • Status
    • start ID/Name
    • reboot ID/Name
    • suspend ID/Name
    • shutdown ID/Name
  • To edit a VM
    • edit ID/Name
  • To delete a VM
    1. undefine ID/Name
    2. sudo virsh destroy ID/Name

---