Troubleshooting

Error records

First distinguish a builder, host verifier, and guest test failure. Host uname does not prove that the QEMU guest uses PREEMPT_RT. .build is used only when BUILD_ROOT is unset. Do not delete it first.

git rev-parse HEAD
git status --short
uname -a
gcc --version
make --version
BUILD_ROOT="${BUILD_ROOT:-$PWD/.build}"
printf 'build_root=%s\n' "$BUILD_ROOT"
if QEMU_TIMEOUT=300 ./scripts/verify_system.sh; then
  verify_status=0
else
  verify_status=$?
fi
printf 'verify_exit=%s\n' "$verify_status"

Dependencies and resources

For command not found, missing headers, or link failures, check PATH and development packages first. build_system.sh checks only some tools. A killed cc1 usually indicates memory pressure; “No space left on device” indicates disk exhaustion, and a source syntax error points to compiler output.

for tool in gcc make flex bison bc cpio gzip rsync python3 depmod; do
  command -v "$tool" || printf 'missing=%s\n' "$tool"
done
df -h .
free -h
command -v qemu-system-x86_64
command -v aarch64-linux-gnu-gcc
command -v riscv64-linux-gnu-gcc
test -r /usr/share/qemu/opensbi-riscv64-generic-fw_dynamic.bin

cpio --help | grep -- --reproducible
JOBS=2 ./scripts/build_system.sh

Images and timeouts

For missing images, complete the matching build or point the verifier at the real BUILD_ROOT. On timeout, check whether the log reached the kernel, init, tests, or shutdown. An ordinary cross-architecture image enters a BusyBox shell; only the guestcheck image validates and powers off.

BUILD_ROOT="${BUILD_ROOT:-$PWD/.build}"
for image in xtos-bzImage xtos-rootfs.cpio.gz kernel.config; do
  test -s "$BUILD_ROOT/images/$image" || printf 'missing_or_empty=%s\n' "$image"
done
grep -nE 'Linux version|Run /init|realtime=|xtos_modules=|guest_|XTOS_|Kernel panic|Power down|power down|error:' "$BUILD_ROOT/system-qemu.log"

QEMU_TIMEOUT=300 ./scripts/verify_guest_arm64.sh
QEMU_TIMEOUT=360 ./scripts/verify_guest_riscv64.sh

Real-time configuration

Check guest /sys/kernel/realtime for runtime state and /proc/config.gz for the running kernel configuration. A -rt15 suffix or PREEMPT banner is insufficient. An unmounted sysfs can produce 0; without /proc/config.gz, preempt_rt=y and xtos_sched=y are absent.

uname -r
cat /proc/mounts
cat /sys/kernel/realtime
zcat /proc/config.gz | grep -E '^CONFIG_(PREEMPT_RT|XTOS_SCHED|XTOS_BCI|XTOS_BCI_OPENBCI|IKCONFIG|IKCONFIG_PROC)='

BUILD_ROOT="${BUILD_ROOT:-$PWD/.build}"
grep -E '^CONFIG_(PREEMPT_RT|XTOS_SCHED|XTOS_BCI|XTOS_BCI_OPENBCI|IKCONFIG|IKCONFIG_PROC)=' "$BUILD_ROOT/kernel-build-arm64/.config"
grep -E '^CONFIG_(PREEMPT_RT|XTOS_SCHED|XTOS_BCI|XTOS_BCI_OPENBCI)=' "$BUILD_ROOT/kernel-build-arm64/include/config/auto.conf"

Expected values are PREEMPT_RT=y, XTOS_SCHED=y, XTOS_BCI=m, XTOS_BCI_OPENBCI=m, IKCONFIG=y, and IKCONFIG_PROC=y. Cross builders initialize configuration only when .config is absent, so an old BUILD_ROOT can retain wrong settings.

repair_build=$(mktemp -d "$HOME/xtos-repair-arm64.XXXXXX")
BUILD_ROOT="$repair_build" JOBS=2 ./scripts/build_system_arm64.sh
BUILD_ROOT="$repair_build" QEMU_TIMEOUT=300 ./scripts/verify_guest_arm64.sh

Module versions

The modprobe names are xtos_bci_core, xtos_bci_buffer, and openbci_cyton, not CONFIG_XTOS_BCI. A .ko in a build directory does not prove a matching module in guest /lib/modules. Load checks run in the target guest.

uname -r
find /lib/modules -maxdepth 2 -type f -name modules.dep -print
find /lib/modules -type f -name '*xtos*' -print
find /lib/modules -type f -name '*openbci*' -print
modprobe xtos_bci_core
modprobe xtos_bci_buffer
modprobe openbci_cyton
dmesg | tail -80
ls -d /sys/module/xtos_bci_core /sys/module/xtos_bci_buffer /sys/module/openbci_cyton

BUILD_ROOT="${BUILD_ROOT:-$PWD/.build}"
find "$BUILD_ROOT/rootfs-arm64/lib/modules" -type f -name 'xtos_bci_core.ko*' -exec modinfo -F vermagic {} \;
cat "$BUILD_ROOT/kernel-build-arm64/include/config/kernel.release"

Use the system builder to install modules, generate depmod indexes, and pack initramfs. Do not copy one .ko into another release directory; build matching kernel and rootfs in a new BUILD_ROOT.

Library architecture

file usually reports only current ar archive for .a files. For file in wrong format, inspect archive members and use the target objdump on the three libraries under rootfs/usr/lib.

BUILD_ROOT="${BUILD_ROOT:-$PWD/.build}"
aarch64-linux-gnu-objdump -f "$BUILD_ROOT/rootfs-arm64/usr/lib/libxtos-rt.a"
aarch64-linux-gnu-objdump -f "$BUILD_ROOT/rootfs-arm64/usr/lib/libxtos-signal.a"
aarch64-linux-gnu-objdump -f "$BUILD_ROOT/rootfs-arm64/usr/lib/libxtos-bci.a"
riscv64-linux-gnu-objdump -f "$BUILD_ROOT/rootfs-riscv64/usr/lib/libxtos-rt.a"

repair_source=$(mktemp -d "$HOME/xtos-archive-repair.XXXXXX")
git archive HEAD | tar -x -C "$repair_source"
cd "$repair_source"
JOBS=2 ./scripts/build_system_arm64.sh
QEMU_TIMEOUT=300 ./scripts/verify_guest_arm64.sh

ARM64/RISC-V64 builders rebuild SDK libraries in source directories, so a new BUILD_ROOT does not isolate those artifacts.

Use separate source copies and build serially when retaining both architectures.

If libraries are correct but tests are missing, inspect Stage 4 warnings; cross builders may continue packing, while strict guest reports guest_test_*=missing.

BCI nodes

Check module loading, driver binding, device registration, and session control separately. examples/bci_test.c defaults to /dev/xtos0 and accepts a path as its first argument. The default string does not prove a device exists; an ordinary serial port or empty manual node is not an XTOS BCI device.

id
ls -l /dev/xtos* /dev/ttyUSB* /dev/ttyACM* 2>/dev/null
find /sys/class -maxdepth 2 -iname '*bci*' -print
ls -d /sys/module/xtos_bci_core /sys/module/xtos_bci_buffer /sys/module/openbci_cyton
dmesg | tail -100

#define XTOS_BCI_GET_INFO _IOR(XTOS_BCI_IOC_MAGIC, 1, struct xtos_bci_info)
#define XTOS_BCI_START_STREAM _IO(XTOS_BCI_IOC_MAGIC, 2)
#define XTOS_BCI_CHECK_IMP _IOR(XTOS_BCI_IOC_MAGIC, 8, uint32_t[32])

Public read_samples copies read bytes directly into a float buffer without numeric unit conversion. The example µV text does not prove sample units, layout, or acquisition correctness; mmap does not change that boundary.

The driver outputs int32 nanovolts; session.c has no nV-to-µV conversion. calibrate (校准) is a placeholder, impedance checking is unsupported, and the set_channel gain field is not passed to the driver. mmap maps raw storage only; its consumer protocol is insufficient and it does not provide converted float samples.

Tests and reports

In the public baseline, libxtos-rt and libxtos-signal have test targets; libxtos-bci has no test target. Some tools Makefiles refer to Python files absent from the public tree, so those targets are not a complete public test suite.

make -C sdk/libxtos-rt all test
make -C sdk/libxtos-signal all test
make -C sdk/libxtos-bci all
./scripts/check_abi.sh

check_abi.sh reads shared libraries from source sdk/ directories; the x86 system build stages them under BUILD_ROOT/userspace-source/sdk/.

For no built shared object found, run all in the source SDK directories first.

The script checks SONAME and filename major/minor, not full layout or symbol compatibility.

• Report the host OS, source commit, worktree changes, and target architecture.
• Report compiler and QEMU versions, exact commands, environment variables, and script status.
• Attach full error output and serial logs, and identify the first missing or failed marker.
• State the new build directory and whether physical hardware exists and is bound.

Source: public build and verifier scripts, sdk/libxtos-bci/Makefile and src/session.c, examples/bci_test.c, kernel and SDK BCI UAPI, scripts/check_abi.sh, and public Makefiles.