Test Types¶

BIT provides three distinct types of tests, each designed for different stages of the system lifecycle and different monitoring needs.

Overview¶

PBIT (Power-On Built-In Test)¶

Purpose¶

PBIT tests verify that hardware components are present, functional, and properly configured when the system boots. Think of them as a "system health check" at power-on.

Characteristics¶

• Run once at system startup
• Quick execution (typically 1-5 seconds each)
• Boolean result - system is either healthy or not
• No frequency setting in configuration
• Validation-focused - checks against expected state

When to Use PBIT¶

• ✓ Verify hardware after system boot
• ✓ Detect missing or failed components early
• ✓ Validate security configuration
• ✓ Check system meets baseline requirements
• ✓ CI/CD validation gates

Example PBIT Tests¶

PBIT tests cover hardware validation, resource checks, security configuration, and device whitelists. Examples include:

• Hardware: CPU cores, disk health, network interfaces, GPIO, CAN buses
• Resources: CPU usage, memory, disk space, temperature sensors
• Security: SELinux/AppArmor, SSH config, firewall, file permissions
• Whitelists: USB devices, PCI devices

See the PBIT Test Reference for complete details on all 21 tests.

PBIT Execution Flow¶

In Service Mode (bit_manager.service):

1. System boots
2. bit_manager.service starts
3. All enabled PBIT tests load
4. Each test runs once sequentially
5. Results logged to systemd journal
6. CBIT tests begin periodic execution

In One-Shot Mode:

1. User runs bit-manager -o
2. All enabled PBIT tests load
3. Each test runs once
4. Results print to stdout
5. Process exits with status code

PBIT tests can be enabled/disabled remotely using bit-monitor-cli or bit-monitor-gui. Use --enable-suite PBIT or --disable-suite PBIT to control all PBIT tests at once.

CBIT (Continuous Built-In Test)¶

Purpose¶

CBIT tests continuously monitor system health, watching for degradation, resource exhaustion, or configuration drift over time.

Characteristics¶

• Run periodically at configured intervals
• Ongoing monitoring until system shutdown
• Frequency-based execution (e.g., every 30 seconds)
• Trend detection possible with historical data
• Alert-focused - notify when issues develop

When to Use CBIT¶

• ✓ Monitor resource utilization (CPU, memory, disk)
• ✓ Detect performance degradation
• ✓ Watch for temperature increases
• ✓ Verify interfaces stay operational
• ✓ Detect unauthorized configuration changes
• ✓ Monitor for security policy violations

Example CBIT Tests¶

CBIT tests provide continuous monitoring across resources, hardware, system health, security, and device whitelists. Examples include:

• Resources: CPU usage, memory consumption, disk space growth, temperature
• Hardware: Disk SMART status, network interfaces, CAN buses, GPIO, serial ports
• System: Kernel messages (dmesg), system logs, power consumption
• Security: SELinux/AppArmor, SSH config, firewall, file permissions
• Devices: USB whitelist, PCI whitelist

See the CBIT Test Reference for complete details on all 21 tests.

CBIT Execution Flow¶

In Service Mode (bit_manager.service):

1. Test loads with frequency setting from config
2. Test executes immediately (first run)
3. Sleeps for frequency seconds
4. Repeats execution and sleep until service stops
5. Each iteration logs status to systemd journal

Typical Frequencies:

• Fast (5-10s): Security checks (USB/PCI whitelist)
• Medium (30-60s): Resource monitoring (CPU, memory, disk, temperature)
• Slow (300-600s): Configuration drift, disk health

CBIT tests can be controlled remotely using bit-monitor-cli or bit-monitor-gui:

• --enable-suite CBIT / --disable-suite CBIT - Control all CBIT tests
• --run-enabled CBIT - Trigger immediate execution of all enabled CBIT tests
• Individual tests can be enabled/disabled without restarting bit-manager

FBIT (Factory Built-In Test)¶

Purpose¶

FBIT tests validate hardware functionality during manufacturing and environmental testing, testing actual data throughput and hardware capabilities beyond presence checks.

Characteristics¶

• Run on demand (not automatic)
• Longer execution time (seconds to minutes)
• Invasive testing - creates files, sends data
• Hardware-specific - requires actual interfaces
• Production validation focused

When to Use FBIT¶

• ✓ Manufacturing line testing
• ✓ Hardware acceptance testing
• ✓ Environmental and EMC testing
• ✓ Post-repair validation
• ✓ Performance benchmarking
• ✓ Hardware certification

Example FBIT Tests¶

FBIT tests validate communication interfaces, storage performance, and I/O functionality. Examples include:

• Communication: CAN loopback, serial loopback (100KB transfer), network throughput (iPerf)
• Storage: SSD performance testing
• I/O: GPIO loopback, USB device testing
• System: Internal monitoring, video device validation, PCI checks

See the FBIT Test Reference for complete details on all 9 tests, including hardware setup requirements.

FBIT Execution¶

FBIT tests run only when explicitly triggered:

# Run specific FBIT test
bit-manager -t fbit_serial_data -o

Or trigger remotely:

# Via bit-monitor-cli
bit-monitor-cli --hostname=<device> --run-test fbit_serial_data

# Run all FBIT tests
bit-monitor-cli --hostname=<device> --run-suite FBIT

See bit-manager for more execution options.

Comparing Test Types¶

Relationship Between Types¶

Many tests come in PBIT/CBIT pairs with shared configuration:

Shared Whitelist Pattern: - PBIT generates baseline (detect current hardware) - CBIT monitors against baseline (detect changes) - Example: pbit_usb_whitelist / cbit_usb_whitelist

Shared Threshold Pattern: - Both use same threshold values - PBIT checks once at boot, CBIT monitors continuously - Example: pbit_cpu_usage / cbit_cpu_usage

Independent Tests: - FBIT tests are typically standalone - Different purpose and configuration than PBIT/CBIT - Example: fbit_serial_data (loopback test)

See Configuration Guide for TOML structure details.

Choosing the Right Test Type¶

Use PBIT When:¶

• ❓ "Is the hardware present and working at boot?"
• ❓ "Does the system meet baseline requirements?"
• ❓ "Are critical components operational?"

Use CBIT When:¶

• ❓ "Is the system staying healthy over time?"
• ❓ "Are resources being exhausted?"
• ❓ "Have there been unauthorized changes?"

Use FBIT When:¶

• ❓ "Does the hardware actually perform as expected?"
• ❓ "Can we transfer data successfully?"
• ❓ "Does the device meet performance specs?"

Test Selection Strategy¶

Embedded System Deployment¶

Minimal (resource-constrained):

# Enable only critical PBIT tests
pbit_cpu_cores
pbit_disk_health
pbit_memory_usage

# Essential CBIT monitoring
cbit_disk_usage (frequency = 300)
cbit_temperature (frequency = 120)

Standard (typical deployment):

# Most PBIT tests
pbit_cpu_usage, pbit_memory_usage, pbit_disk_*
pbit_ethernet, pbit_temperature
pbit_usb_whitelist, pbit_pci_whitelist

# Active CBIT monitoring
cbit_cpu_usage (30s), cbit_memory_usage (30s)
cbit_disk_usage (60s), cbit_temperature (60s)
cbit_usb_whitelist (5s), cbit_pci_whitelist (5s)

Comprehensive (security-critical):

# All PBIT tests
All 21 PBIT tests enabled

# All CBIT monitoring
All 21 CBIT tests with appropriate frequencies

Manufacturing Line¶

# Run all FBIT tests
fbit_can_data      # CAN loopback
fbit_serial_data   # Serial loopback
fbit_gpio_data     # GPIO loopback
fbit_ssd           # Storage performance
fbit_usb           # USB functionality
fbit_tcp_data      # Network throughput

Best Practices¶

PBIT Best Practices¶

1. Enable all critical hardware checks
2. Set realistic thresholds (not too tight)
3. Don't disable security checks
4. Test after configuration changes

CBIT Best Practices¶

1. Choose appropriate frequencies (don't poll too fast)
2. Monitor critical resources (CPU, memory, disk)
3. Enable security monitoring (whitelist checks)
4. Review logs regularly for trends

FBIT Best Practices¶

1. Run in controlled environment (not production)
2. Ensure proper hardware setup (loopbacks, etc.)
3. Document pass/fail criteria clearly
4. Include in manufacturing checklist

Next Steps¶

• PBIT Test Reference - Complete details on all 21 PBIT tests
• CBIT Test Reference - Complete details on all 21 CBIT tests
• FBIT Test Reference - Complete details on all 9 FBIT tests with hardware requirements
• bit-learn Guide - Generate test configurations
• bit-inspect Guide - Inspect test details
• Running Tests - Execute tests