Raspberry PI4 and PXE with NFS4

  • dhcp server
  • NFS server, preferably NFSv4
  • rpi 4 with raspbian (buster)

DHCP server settings

We need to add code 66 to DHCP options. I’m using mikrotik, so I’ve added one DHCP option

  • name: ‘tftp-server-name’
  • code: 66
  • value: s’192.168.1.100′

please note the value with preceding ‘s’, it’s required. Then add one dhcp option group, add this option and add this option group to certain dhcp lease

NFS server configuration

Configure your server with privileges from given IPs, sys, async. You’ll need one mountpoint (or if you need more security, you can define one per host), like /pxe with rw privileges, limited to given IP(s). My directory structure looks like

- pxe
  - tftp-server
    - [serial]
  - rpis
    - [serial]

Whereas tftp-server/[serial] contains boot code for certain IP, and rpis/[serial] is root.

To obtain serial, use this code.

vcgencmd otp_dump | grep 28: | sed s/.*://g

TFTP server

To obtain boot from network, you need one TFTP server. Ideally, use one of mountpoints. Root of your TFTP must be set to /pxe/tftp-server

Preparing eeprom to PXE boot

You need to do following steps – update apt and upgrade to the lastest, and install latest rpi-eeprom package

apt update
apt full-upgrade
apt install rpi-eeprom

Then copy latest .bin to the new one and extract parameters

cp /lib/firmware/raspberrypi/bootloader/stable/pieeprom-[find latest].bin pieeprom.bin
rpi-eeprom-config pieeprom.bin > bootconf.txt
vim bootconf.txt

and edit bootconf.txt like this

[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
DHCP_TIMEOUT=45000
DHCP_REQ_TIMEOUT=4000
TFTP_FILE_TIMEOUT=30000
TFTP_IP=
TFTP_PREFIX=0
BOOT_ORDER=0x21
SD_BOOT_MAX_RETRIES=3
NET_BOOT_MAX_RETRIES=5
[none]
FREEZE_VERSION=0

The most important part is BOOT_ORDER. 0x21 tells to try sd card, then network boot. Now to update eeprom.

rpi-eeprom-config --out pieeprom-new.bin --config bootconf.txt pieeprom.bin
rpi-eeprom-update -d -f ./pieeprom-new.bin
reboot

Filesystem preparation

as there’s no swap possible, remove dphys-swap

apt remove -y --purge dphys-swapfile
rm /var/swap

We need to disable resize2fs if enabled

systemctl disable resize2fs_once

Now we need to copy our /boot into /pxe/tftp-server/[serial]. So prepare mount and do the copy

mkdir -p /mnt/pxe
mount 192.168.1.100:/pxe /mnt/pxe -t nfs -o proto=tcp,vers=4.2,port=2049
export serial=[serial from above]
copy -r /boot /mnt/pxe/tftp-server/$serial/

And now to copy root filesystem to NFS

rsync -xa --progress --exclude /mnt/pxe / /mnt/pxe/rpis/$serial/

Once the copy is done, edit fstab in the copy (in /mnt/pxe/rpis/$serial/etc/fstab) and remove all entries for / and replace /boot mountpoint. fstab should then look like this

proc            /proc           proc    defaults          0       0
192.168.1.100:/pxe/tftp-server/[serial] /boot nfs defaults,proto=tcp,port=2049,vers=4.2 0 0

Now it’s time to modify cmdline.txt in boot /mnt/pxe/tftp-server/[serial] directory, to specify root mountpoint

console=tty1 root=/dev/nfs nfsroot=192.168.1.100:/pxe/rpis/[serial],proto=tcp,port=2049,vers=4.2 rw ip=dhcp rootwait elevator=deadline

It must be one oneliner only.

Now it’s time to shutdown your rpi, remove SD card and let it boot from the network. Enjoy!

bash to resolve dependencies

#!/bin/bash

declare -A USE_MAP
declare -a ENABLES

# export_name dependency dependecy dependency...
USE_MAP['HBASE']='USE_HBASE HCATALOG HUE'
USE_MAP['PIG']='USE_PIG'
USE_MAP['HUE']='USE_HUE'
USE_MAP['HCATALOG']='USE_HCATALOG'
USE_MAP['PRESTO']='USE_PRESTO HCATALOG'
ENABLES=()

function use() {
  local dep=(${USE_MAP["$1"]})
  ENABLES+=(${dep[0]})
  dep=("${dep[@]:1}")
  if [ ! -z "$dep" ]; then
    for item in ${dep[@]}; do
      use $item
    done
  fi
}

function export_enabled() {
  sorted_use=($(echo "${ENABLES[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' '))
  for item in ${sorted_use[@]}; do
    echo "export $item";
  done
}

use HBASE
export_enabled

------

$ bash resolve.sh
export USE_HBASE
export USE_HCATALOG
export USE_HUE