Progress Log

Raspberry Pi 4 Headless First Boot

April 11, 202620:32Build Log

Raspberry Pi 4 Headless First Boot

Date: 2026-04-11 Type: Build Log

Context

Todd received the Raspberry Pi 4 and wanted to do a headless first-boot setup over WiFi from his Mac, with Claude Code driving everything possible from the terminal. The official build guide (docs/articles/build-01-bench-testing.md Step 3) had a single naive sentence about flashing with Pi Imager and "SSH in after a minute." Reality was significantly more involved. This log captures what actually happened and what the guides need to reflect.

What Changed

Shipped working artifact: a fully headless Raspberry Pi 4 named tickslayer, on home WiFi (Jurassic Park.Which one?), accessible from Todd's Mac via ssh tickslayer using key auth. No monitor, no keyboard, no ethernet remaining attached.

What we did, in order

Round 1 — Naive headless flash via custom.toml (FAILED)

  1. Identified the SD card on Todd's Mac as /dev/disk14 (64 GB, Built-In SDXC Reader, was NTFS) using diskutil list physical and diskutil info disk14. Confirmed not the Mac's main drive (disk0, 1 TB).
  2. Downloaded latest Raspberry Pi OS Lite 64-bit from https://downloads.raspberrypi.com/raspios_lite_arm64_latest to ~/tickslayer-pi-flash/raspios-lite-arm64.img.xz (487 MB compressed → 2.8 GB decompressed via xz -dk -T 0).
  3. Tried sudo dd directly from Claude — hit sudo: a terminal is required to read the password. The harness can't pipe sudo prompts into Claude's bash sessions. Solution: had Todd run sudo dd if=...img of=/dev/rdisk14 bs=4m himself via the ! prefix in the prompt. Wrote 2,986,344,448 bytes in 88 seconds (~34 MB/s).
  4. Wrote custom.toml to the FAT32 bootfs partition with hostname, user, hashed password (openssl passwd -6), SSH enable, WiFi (Jurassic Park.Which one?), and locale.
  5. Booted Pi. Filesystem expanded (so something ran), but Pi never appeared on network. ARP table never showed any Pi MAC prefix (b8:27:eb, dc:a6:32, e4:5f:01, d8:3a:dd, 2c:cf:67).

Round 2 — Switch to cloud-init mechanism (FAILED)

  1. Inspected bootfs after first failed boot. Found meta-data, user-data, network-config files (cloud-init style) — concluded this image uses cloud-init, not custom.toml.
  2. Removed custom.toml. Wrote proper cloud-init user-data (user todd, hostname tickslayer, SSH, locale) and network-config (netplan v2 WiFi block). Added userconf.txt and empty ssh file as belt-and-suspenders fallbacks.
  3. Booted Pi. Same result: silent failure, not on network.

Round 3 — Switch to eero guest network to test "is the SSID confusing wpa_supplicant?" (FAILED + DEAD END)

  1. Hypothesized the special characters in Jurassic Park.Which one? (period, space, question mark) might be breaking WPA supplicant. Asked Todd to create an eero guest network (786pi / SweeperPocket786).
  2. Updated network-config with new SSID. Bumped instance_id in meta-data so cloud-init would treat next boot as fresh.
  3. Booted Pi. Pi didn't appear. Checked eero app — only Todd's Mac was on the guest network, no Pi.
  4. Discovered the eero gotcha: ping-sweeping 192.168.11.0/24 from the Mac filled the ARP table with (incomplete) entries for every address except gateway and the Mac itself. Eero guest networks have AP isolation enabled by default and there is no toggle to disable it. This meant even if the Pi had joined, we couldn't have reached it. Worse than useless for SSH-from-Mac debugging.

Round 4 — Ethernet (WORKED, immediately)

  1. Got Todd to plug ethernet from Pi to eero LAN port (initially he plugged into the modem, which put the Pi on a totally separate ISP-managed network — Mac on 192.168.4.x, Pi on whatever the modem hands out, no mutual visibility).
  2. After moving cable to eero LAN port: ping tickslayer.local resolved to 192.168.4.31 immediately. SSH connected on first try. Password tickslayer-temp worked (so userconf.txt had created the user).

Round 5 — Ground truth from inside the Pi (ROOT CAUSE FOUND)

Once SSHed in via ethernet, ran diagnostics. Found:

  • rfkill list showed phy0: Wireless LAN — Soft blocked: yes. The WiFi radio was kernel-level RFKILL'd the entire time. This is why no SSID, no network config attempt, no nothing — the radio was off.
  • Why was it blocked? Pi requires a WiFi country code to be set before it'll enable the radio (regulatory compliance). We never set one. Cloud-init would have set it via the locale module, but...
  • systemctl is-enabled cloud-init.service returned not-found. Cloud-init is NOT enabled on Raspberry Pi OS Lite Bookworm by default. All our user-data / network-config YAML edits across rounds 2 and 3 were doing absolutely nothing. The user todd and hostname tickslayer got created by userconf.txt (legacy raspi-config first-run mechanism), our belt-and-suspenders fallback. Without that fallback we would have had no user at all.
  • /etc/netplan/ was empty. Bookworm Pi OS uses NetworkManager directly, not netplan. Cloud-init's network-config would have been bridged into NM if cloud-init were running, but it wasn't.

Round 6 — Fix from inside (WORKED)

sudo raspi-config nonint do_wifi_country US   # Set WiFi country
sudo rfkill unblock wifi                       # Unblock the radio
nmcli device wifi rescan                       # Rescan
sudo nmcli device wifi connect "Jurassic Park.Which one?" password "theonewhereleodies"

wlan0 immediately went from unavailable → disconnected → connected, picked up DHCP at 192.168.4.33, and ping -I wlan0 1.1.1.1 succeeded. Connection profile saved at /etc/NetworkManager/system-connections/Jurassic Park.Which one?.nmconnection with autoconnect=yes, so it survives reboots. Pulled ethernet, confirmed Pi still reachable via WiFi.

SSH key setup

Generated a project-specific ed25519 keypair at ~/.ssh/tickslayer_ed25519 (no passphrase, comment todd@mac -> tickslayer pi). Pushed the public key to the Pi via expect + ssh + cat >> ~/.ssh/authorized_keys. Added an SSH config alias to ~/.ssh/config:

Host tickslayer
    HostName tickslayer.local
    User todd
    IdentityFile ~/.ssh/tickslayer_ed25519
    IdentitiesOnly yes
    StrictHostKeyChecking accept-new

Now ssh tickslayer Just Works from anywhere on Todd's Mac with no password prompts.

Files Modified

  • ~/tickslayer-pi-flash/raspios-lite-arm64.img(.xz) — Downloaded Pi OS Lite image. Kept on disk for future re-flashes.
  • ~/.ssh/tickslayer_ed25519 + .pub — Project SSH keypair for Pi access.
  • ~/.ssh/config — Added Host tickslayer alias block.
  • /Volumes/bootfs/custom.toml — Created in Round 1, deleted in Round 2 (wrong mechanism for this image).
  • /Volumes/bootfs/user-data — Wrote cloud-init user config (created todd, set hostname, enabled SSH, set timezone). Did nothing because cloud-init isn't enabled, but harmless.
  • /Volumes/bootfs/network-config — Wrote netplan WiFi config. Did nothing for the same reason.
  • /Volumes/bootfs/userconf.txt — Belt-and-suspenders user creation. This is the file that actually did the work. Created todd:tickslayer-temp via the legacy raspi-config first-run service.
  • /Volumes/bootfs/ssh (empty) — Belt-and-suspenders SSH enable. Pi OS auto-removes this on first boot after enabling SSH.
  • /Volumes/bootfs/meta-data — Bumped instance_id from rpios-image to tickslayer-2026-04-11-guest-net. Pointless since cloud-init wasn't running, but I didn't know that yet.

On the Pi side (live changes via SSH)

  • /etc/wpa_supplicant/wpa_supplicant.conf — country=US set via raspi-config nonint do_wifi_country US.
  • /etc/NetworkManager/system-connections/Jurassic Park.Which one?.nmconnection — Created by nmcli connect. WPA2-PSK, autoconnect enabled, mode 600 root.
  • ~/.ssh/authorized_keys — Added Mac's tickslayer public key.

Key Takeaways

  • Raspberry Pi OS Lite Bookworm does NOT enable cloud-init by default. The meta-data / user-data / network-config files exist on the boot partition but nothing processes them. This contradicts a lot of online tutorials. Don't trust cloud-init headless config on this image.
  • custom.toml is also not the mechanism on plain pi-gen Pi OS Lite. It's a Pi Imager-specific feature that requires Imager to write a complementary firstrun.sh and modify cmdline.txt to invoke it. If you write custom.toml by hand without those, nothing happens.
  • userconf.txt is the actually-reliable headless user creation mechanism on current Pi OS. Format: username:openssl-passwd-6-hash. Lives in the FAT32 bootfs partition. Picked up by raspi-config's first-boot service. Independent of cloud-init.
  • Pi WiFi is RFKILL soft-blocked until a WiFi country code is set. This is regulatory compliance and applies to all current Pi OS images. If you don't set a country, the radio never comes up and wlan0 shows unavailable in nmcli. Set it with raspi-config nonint do_wifi_country US (or via wpa_supplicant.conf country= line).
  • The reliable way to configure WiFi from inside a running Pi (Bookworm) is nmcli, not netplan and not wpa_supplicant.conf. sudo nmcli device wifi connect "<SSID>" password "<PSK>" creates a persistent profile in /etc/NetworkManager/system-connections/ with autoconnect on by default. SSIDs with spaces and special characters work fine in double quotes.
  • Eero guest networks have non-disableable AP isolation. Useless for "let me put a new device on a clean network and SSH to it from my Mac" debugging because client-to-client traffic is blocked. The (incomplete) ARP entries across the entire /24 are the smoking gun.
  • Plugging ethernet into the modem ≠ plugging into the router. Devices on the modem port are on a separate ISP-managed network and can't be reached from the home LAN. Always use a LAN port on the eero (or whatever router) for headless debug.
  • The harness can't enter sudo passwords. For sudo dd and similar, have the user run it themselves via the ! prefix in the prompt. Pre-caching with sudo -v doesn't help because the cache is per-tty and Claude's bash sessions don't share a tty with the user's interactive shell.
  • networksetup -getairportnetwork en0 is broken on recent macOS. Returns "not associated" even when WiFi is connected. Use ifconfig en0 + netstat -rn or wdutil info instead.
  • expect is built into macOS and is the right tool when you need to script a single password prompt in non-interactive contexts (initial SSH key copy, etc).

Implications for the build guides

build-01-bench-testing.md Step 3 currently says "Flash Raspberry Pi OS Lite (64-bit) onto the MicroSD card using Raspberry Pi Imager... During imaging, set hostname, enable SSH, set username/password, configure WiFi" — this is fine for the GUI Imager path, but the guide has zero coverage of:

  • The CLI flash path (xz -d + dd) for users who'd rather script it
  • Any of the gotchas above (cloud-init not enabled, WiFi country requirement, eero guest isolation, modem-vs-router)
  • The agent-tip callout pattern Todd asked for (<agent-tip> blocks with prompts a user can copy-paste to their agent, plus links to required hardware/software)
  • Verification steps (ssh tickslayer, nmcli device status, rfkill list)

build-02-pi-setup.md Step 1 assumes you've already SSHed in. We need to add a "if SSH doesn't work, here's how to debug" sub-section that walks through the rfkill / cloud-init / userconf issues we hit.

Plan for next session: retrofit Guides 1 and 2 with the lessons above, build the <agent-tip> custom React component (src/components/agent-tip.tsx) and register it in src/app/docs/markdown-renderer.tsx next to the existing <part> component, and start sprinkling agent-tip callouts throughout the guides.

Time spent

~90 min for what the existing guide implies should be 5 min. Worth it for the documented lessons — every one of these is something the next person (or future-Todd, or another agent) would also hit.

← Build Guide Articles & Series NavigationPi Software Bootstrap (Build Guide 2 Steps 1-3)→