Building an ARM64 home server the hard way
source link: https://jforberg.se/blog/posts/2023-02-19-rockpro64/rockpro64.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Booting Linux
Pine provides ready-made disk images for several Linux distros, including Debian and Manjaro. Personally, I’ve been running Arch Linux on all my devices for some time now. I’m very satisfied with it and don’t see any reason to switch. Since Arch Linux is not officially supported, I had to craft my own disk image to boot from.
The Arch Linux ARM project provides a "generic" image for Aarch64 (ARM64) which contains a generic kernel and device trees for many chips, including the RockPro64. This image is "intended to be used by developers who are familiar with their system, and can set up the necessary boot functionality on their own".
The boot process for RK3399 is relatively straightforward. The hardcoded boot ROM will first try to boot from the on-board SPI flash memory. Failing that, it tries to read the micro SD card at sector offset 64 (byte offset 0x8000). If this sector contains a valid bootloader, it can then take over and finish the boot process. The job of the bootloader is to initialise the DRAM; find a Linux kernel image together with initramfs and device tree; load them all into DRAM; and then jump into the kernel itself.
I had the idea to steal the boot sectors from one of the official images and then patch them into Arch Linux ARM’s "generic" image. I went with the Manjaro image since the Manjaro distro is actually based on Arch Linux.
Inspecting the image
Let’s first have a look at the layout of the Manjaro image. We’ll use this a template for crafting our own image.
$ fdisk Manjaro-ARM-minimal-rockpro64-20.12.1.img
Disk Manjaro-ARM-minimal-rockpro64-20.12.1.img: 1.77 GiB, 1900019712 bytes, 3710976 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x1502a457 Device Boot Start End Sectors Size Id Type Manjaro-ARM-minimal-rockpro64-20.12.1.img1 62500 500000 437501 213.6M c W95 FAT32 (LBA) Manjaro-ARM-minimal-rockpro64-20.12.1.img2 500001 3710975 3210975 1.5G 83 Linux
The first line partition is /boot, containing the Linux kernel, initramfs, and device trees. It starts at sector 62500, meaning that the bootloader resides in the sectors below 62500.
Preparing our custom image
The next step is to insert our fresh micro SD card and open it in fdisk. I’ll leave 62500 empty sectors at the start like in the reference image, then a relatively small /boot partition in VFAT format.
It’s common to use VFAT for the boot partition as it’s a relatively simple filesystem. The boot partition contains the Linux kernel image and related files which need to be accessed directly by the bootloader. A typical bootloader lacks built-in drivers for more complicated filesystems.
The rest of the free space will be used for a standard Linux partition to serve as our provisional root filesystem. This is how it looks in fdisk:
# fdisk /dev/mmcblk0
Disk /dev/mmcblk0: 14.73 GiB, 15812526080 bytes, 30883840 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x4db9d122 Device Boot Start End Sectors Size Id Type /dev/mmcblk0p1 62500 1112063 1049564 512.5M c W95 FAT32 (LBA) /dev/mmcblk0p2 1112064 30883839 29771776 14.2G 83 Linux
Now we steal the boot sectors from the Manjaro image, copying them directly onto the memory card using dd. The sector size is 512 bytes and the first sector contains the partition table we just set up, it should not be overwritten.
# dd if=Manjaro-ARM-minimal-rockpro64-20.12.1.img of=/dev/mmcblk0 bs=512 count=62499 seek=1 skip=1
We can verify by looking at a hexdump of the card that it seems to be correctly set up:
# hexdump /dev/mmcblk0 | less
0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 00001b0 0000 0000 0000 0000 d122 4db9 0000 0200 00001c0 d0c5 030c dfd0 f424 0000 03dc 0010 0000 00001d0 e0c1 0383 ff10 f800 0010 4800 01c6 0000 00001e0 0000 0000 0000 0000 0000 0000 0000 0000 00001f0 0000 0000 0000 0000 0000 0000 0000 aa55 0000200 0000 0000 0000 0000 0000 0000 0000 0000 * 0008000 8c3b fcdc 9fbe 519d 30eb ce34 5124 981f 0008010 0cff 36f2 5005 bbc8 ec3f bddd 8506 b7fa
The first blob is the partition table, then at offset 0x8000 the bootloader starts with the magic number described in the boot documentation. All is well.
Now let’s format the partitions and mount them at some temporary mount points.
# mkfs.vfat /dev/mmcblk0p1 # mkfs.ext4 /dev/mmcblk0p2
# mkdir boot root test
# mount /dev/mmcblk0p1 boot # mount /dev/mmcblk0p2 root
The "generic" Arch Linux image is a simple tarball of how the complete filesystem should look. We extract the /boot part to the boot partition and the rest to the root partition.
# tar -C boot --strip-components=2 -xvf ArchLinuxARM-aarch64-latest.tar.gz ./boot/ # tar -C root --exclude=./boot -xvf ArchLinuxARM-aarch64-latest.tar.gz
Configuring the bootloader
An ARM bootloader has three main jobs: Initialise DRAM, load the main operating system components (kernel image, kernel command line, initramfs, and devicetree) into DRAM, and then jump into the kernel.
I guessed that Manjaro’s bootloader will automatically read the first partition on the SD card to find out what to do. At this point we don’t know exactly how to configure the boot loader beyond that. Let’s mount the unmodified /boot partition of Manjaro’s image and poke around a bit.
# mount -o offset=$((62500 * 512)) Manjaro-ARM-minimal-rockpro64-20.12.1.img test
There is a file called extlinux/extlinux.conf which looks promising:
# cat test/extlinux/extlinux.conf
LABEL Manjaro ARM KERNEL /Image FDT /dtbs/rockchip/rk3399-rockpro64.dtb APPEND initrd=/initramfs-linux.img console=tty1 console=ttyS2,1500000 root=LABEL=ROOT_MNJRO rw rootwait bootsplash.bootfile=bootsplash-themes/manjaro/bootsplash
Much of this is fine as is, but the root= parameter of the kernel command line is problematic. Let’s find the PARTUUID of our newly created boot partition:
# blkid
/dev/mmcblk0p1: UUID="555E-4BB5" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="4db9d122-01" /dev/mmcblk0p2: UUID="5059aba4-a312-469b-becb-91ee53a37635" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="4db9d122-02"
Armed with this information we change extlinux/extlinux.conf into:
LABEL Arch Linux ARM KERNEL /Image FDT /dtbs/rockchip/rk3399-rockpro64.dtb APPEND initrd=/initramfs-linux.img console=tty1 console=ttyS2,1500000 root=PARTUUID=4db9d122-02 rw rootwait
Booting up
Finally, sync the changes to persistent storage and then unmount the partitions and insert the micro SD-card into our RockPro64 board.
# sync # umount boot root test
To see what’s going on, I connect the serial adapter I purchased from Pine following the description given on the Pine forums. GNU screen can be used to monitor the serial port.
$ screen -h 1000000 /dev/ttyUSB0 1500000
Now, turn on the power!
U-Boot TPL 2020.10-2 (Dec 27 2020 - 15:46:04) Channel 0: LPDDR4, 50MHz BW=32 Col=10 Bk=8 CS0 Row=16/15 CS=1 Die BW=16 Size=2048MB Channel 1: LPDDR4, 50MHz BW=32 Col=10 Bk=8 CS0 Row=16/15 CS=1 Die BW=16 Size=2048MB 256B stride lpddr4_set_rate: change freq to 400000000 mhz 0, 1 lpddr4_set_rate: change freq to 800000000 mhz 1, 0 Trying to boot from BOOTROM Returning to boot ROM... U-Boot SPL 2020.10-2 (Dec 27 2020 - 15:46:04 +0000) Trying to boot from MMC1 U-Boot 2020.10-2 (Dec 27 2020 - 15:46:04 +0000) Manjaro ARM SoC: Rockchip rk3399 Reset cause: POR Model: Pine64 RockPro64 v2.1 DRAM: 3.9 GiB PMIC: RK808 MMC: mmc@fe310000: 2, mmc@fe320000: 1, sdhci@fe330000: 0 Loading Environment from SPIFlash... Invalid bus 0 (err=-19) *** Warning - spi_flash_probe_bus_cs() failed, using default environment In: serial Out: serial Err: serial Model: Pine64 RockPro64 v2.1 Net: eth0: ethernet@fe300000 Hit any key to stop autoboot: 0 Card did not respond to voltage select! switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... Found /extlinux/extlinux.conf Retrieving file: /extlinux/extlinux.conf 183 bytes read in 5 ms (35.2 KiB/s) 1: Arch Linux ARM Retrieving file: /initramfs-linux.img 7411886 bytes read in 319 ms (22.2 MiB/s) Retrieving file: /Image 40733184 bytes read in 1732 ms (22.4 MiB/s) append: initrd=/initramfs-linux.img console=tty1 console=ttyS2,1500000 root=PARTUUID=4db9d122-02 rw rootwait Retrieving file: /dtbs/rockchip/rk3399-rockpro64.dtb 79528 bytes read in 13 ms (5.8 MiB/s) Moving Image from 0x2080000 to 0x2200000, end=49c0000 ## Flattened Device Tree blob at 01f00000 Booting using the fdt blob at 0x1f00000 Loading Ramdisk to f1815000, end f1f268ae ... OK Loading Device Tree to 00000000f17fe000, end 00000000f18146a7 ... OK Starting kernel ... [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034] [ 0.000000] Linux version 5.16.13-1-aarch64-ARCH (builduser@leming) (aarch64-unknown-linux-gnu-gcc (GCC) 11.2.0, GNU ld (GNU Binutils) 2.38) #1 SMP Thu Mar 10 01:59:18 UTC 2022
The output ends with:
Arch Linux 5.16.13-1-aarch64-ARCH (ttyS2) alarm login:
Incidentally, the password for user "alarm" is "alarm" and the password for "root" is "root".
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK