Test Types¶

BIT provides three distinct types of tests, each designed for different stages of the system lifecycle and different monitoring needs.

Overview¶

PBIT (Power-On Built-In Test)¶

Purpose¶

PBIT tests verify that hardware components are present, functional, and properly configured when the system boots. Think of them as a "system health check" at power-on.

Characteristics¶

• Run once at system startup
• Quick execution (typically 1-5 seconds each)
• Boolean result - system is either healthy or not
• No frequency setting in configuration
• Validation-focused - checks against expected state

When to Use PBIT¶

• ✓ Verify hardware after system boot
• ✓ Detect missing or failed components early
• ✓ Validate security configuration
• ✓ Check system meets baseline requirements
• ✓ CI/CD validation gates

Common PBIT Tests¶

Hardware Validation: - pbit_cpu_cores - Verify expected CPU count - pbit_disk_health - Check disk SMART status - pbit_ethernet - Verify network interfaces exist - pbit_gpio - Check GPIO pins are accessible - pbit_can - Verify CAN interfaces are available

Resource Checks: - pbit_cpu_usage - Ensure CPU isn't overloaded at boot - pbit_memory_usage - Verify sufficient memory available - pbit_disk_usage - Check disk space not exhausted - pbit_temperature - Verify sensors within range

Security & Configuration: - pbit_selinux_apparmor_status - Verify security system active - pbit_ssh_configuration - Check SSH daemon settings - pbit_firewall_configuration - Verify firewall status - pbit_permissions_verification - Check file permissions

Device Whitelists: - pbit_usb_whitelist - Verify only approved USB devices - pbit_pci_whitelist - Verify only approved PCI devices

Example PBIT Configuration¶

[pbit_cpu_usage]
enabled = true
threshold = 42  # Fail if CPU > 42% at boot

[pbit_disk_health]
enabled = true

[[disk]]
device = "/dev/sda"

[[disk]]
device = "/dev/sdb"

PBIT Execution Flow¶

1. System boots
2. bit_manager.service starts
3. All enabled PBIT tests load
4. Each test runs once sequentially
5. Results logged
6. CBIT tests begin periodic execution

CBIT (Continuous Built-In Test)¶

Purpose¶

CBIT tests continuously monitor system health, watching for degradation, resource exhaustion, or configuration drift over time.

Characteristics¶

• Run periodically at configured intervals
• Ongoing monitoring until system shutdown
• Frequency-based execution (e.g., every 30 seconds)
• Trend detection possible with historical data
• Alert-focused - notify when issues develop

When to Use CBIT¶

• ✓ Monitor resource utilization (CPU, memory, disk)
• ✓ Detect performance degradation
• ✓ Watch for temperature increases
• ✓ Verify interfaces stay operational
• ✓ Detect unauthorized configuration changes
• ✓ Monitor for security policy violations

Common CBIT Tests¶

Resource Monitoring: - cbit_cpu_usage - Continuous CPU utilization monitoring - cbit_memory_usage - Track memory consumption over time - cbit_disk_usage - Watch disk space growth - cbit_temperature - Monitor thermal conditions

Hardware Monitoring: - cbit_disk_health - Continuous SMART monitoring - cbit_ethernet - Monitor interface status - cbit_can - Watch CAN interface health - cbit_gpio - Monitor GPIO pin states - cbit_serial_ports - Track serial port availability

System Monitoring: - cbit_dmesg_check - Watch kernel messages for errors - cbit_syslog_analysis - Monitor system logs - cbit_power_consumption - Track power usage

Security Monitoring: - cbit_selinux_apparmor_status - Ensure security stays enabled - cbit_ssh_configuration - Watch for SSH config changes - cbit_firewall_configuration - Monitor firewall status - cbit_permissions_verification - Detect permission changes

Device Monitoring: - cbit_usb_whitelist - Detect unauthorized USB devices - cbit_pci_whitelist - Detect unauthorized PCI devices

Example CBIT Configuration¶

[cbit_memory_usage]
enabled = true
frequency = 30      # Check every 30 seconds
threshold = 90      # Alert if > 90%

[cbit_temperature]
enabled = true
frequency = 60      # Check every 60 seconds

[[thermal_zone]]
label = "Core 0"
threshold = 85.0

[[thermal_zone]]
label = "Core 1"
threshold = 85.0

CBIT Execution Flow¶

1. Test loads with frequency setting
2. Test executes immediately
3. Sleeps for frequency seconds
4. Repeats step 2-3 until service stops
5. Each iteration logs status

Choosing CBIT Frequencies¶

Fast (5-10 seconds): - Critical security (USB/PCI whitelist) - Rapid state changes (CAN/network interfaces)

Medium (30-60 seconds): - Resource utilization (CPU, memory, disk) - Temperature monitoring - Log analysis

Slow (300-600 seconds): - Configuration drift checks - Disk health (SMART data) - BSP version monitoring

FBIT (Factory Built-In Test)¶

Purpose¶

FBIT tests validate hardware functionality during manufacturing, testing actual data throughput and hardware capabilities beyond presence checks.

Characteristics¶

• Run on demand (not automatic)
• Longer execution time (seconds to minutes)
• Invasive testing - creates files, sends data
• Hardware-specific - requires actual interfaces
• Production validation focused

When to Use FBIT¶

• ✓ Manufacturing line testing
• ✓ Hardware acceptance testing
• ✓ Post-repair validation
• ✓ Performance benchmarking
• ✓ Hardware certification

Available FBIT Tests¶

Communication Tests: - fbit_can_data - CAN bus loopback test (requires physical loopback) - fbit_serial_data - Serial port loopback test (100KB transfer) - fbit_tcp_data - Network throughput test (requires iPerf server)

Storage Tests: - fbit_ssd - SSD performance test (creates 1MB file)

I/O Tests: - fbit_gpio_data - GPIO loopback test (requires wiring) - fbit_usb - USB device test (creates test file in /tmp)

System Tests: - fbit_system_data - Internal systems monitoring - fbit_video - V4L2 video device validation - fbit_pci - PCI capability and status checks

Example FBIT Configuration¶

[fbit_serial_data]
enabled = true
baud_rate = 115200
data_size_kb = 100

[[port]]
tx_port = "/dev/ttyS0"
rx_port = "/dev/ttyS1"

[fbit_ssd]
enabled = true
test_file_size_mb = 1
target_directory = "/tmp"

FBIT Execution¶

# Run specific FBIT test
bit-manager -t fbit_serial_data -o

# Run all FBIT tests
bit-manager \
  -t fbit_can_data \
  -t fbit_gpio_data \
  -t fbit_serial_data \
  -t fbit_ssd \
  -t fbit_usb \
  -t fbit_video \
  -t fbit_pci \
  -t fbit_system_data \
  -t fbit_tcp_data \
  -o

Hardware Requirements¶

Many FBIT tests require physical setup:

Loopback Requirements: - CAN: Physical loopback connector or two CAN interfaces cross-wired - Serial: TX/RX pins wired together or null modem cable - GPIO: Output pins wired to input pins

Network Requirements: - TCP: iPerf server accessible on network

Storage Requirements: - SSD: Writable filesystem with sufficient space

Comparing Test Types¶

Test Execution Comparison¶

Configuration Comparison¶

PBIT Config:

[pbit_test]
enabled = true
# Test-specific settings
threshold = 80

CBIT Config:

[cbit_test]
enabled = true
frequency = 30     # Only difference - adds periodic execution
threshold = 80

FBIT Config:

[fbit_test]
enabled = true
# Hardware-specific settings
port = "/dev/ttyS0"
data_size = 100

Relationship Between Types¶

Many tests come in PBIT/CBIT pairs:

Pattern 1: Shared Configuration - PBIT generates whitelist/baseline - CBIT monitors against that baseline - Example: USB/PCI whitelists

Pattern 2: Same Checks, Different Timing - Both use same thresholds - PBIT checks once, CBIT monitors continuously - Example: CPU, memory, disk usage

Pattern 3: Independent - Different purposes - Different configurations - Example: FBIT tests are standalone

Choosing the Right Test Type¶

Use PBIT When:¶

• ❓ "Is the hardware present and working at boot?"
• ❓ "Does the system meet baseline requirements?"
• ❓ "Are critical components operational?"

Use CBIT When:¶

• ❓ "Is the system staying healthy over time?"
• ❓ "Are resources being exhausted?"
• ❓ "Have there been unauthorized changes?"

Use FBIT When:¶

• ❓ "Does the hardware actually perform as expected?"
• ❓ "Can we transfer data successfully?"
• ❓ "Does the device meet performance specs?"

Test Selection Strategy¶

Embedded System Deployment¶

Minimal (resource-constrained):

# Enable only critical PBIT tests
pbit_cpu_cores
pbit_disk_health
pbit_memory_usage

# Essential CBIT monitoring
cbit_disk_usage (frequency = 300)
cbit_temperature (frequency = 120)

Standard (typical deployment):

# Most PBIT tests
pbit_cpu_usage, pbit_memory_usage, pbit_disk_*
pbit_ethernet, pbit_temperature
pbit_usb_whitelist, pbit_pci_whitelist

# Active CBIT monitoring
cbit_cpu_usage (30s), cbit_memory_usage (30s)
cbit_disk_usage (60s), cbit_temperature (60s)
cbit_usb_whitelist (5s), cbit_pci_whitelist (5s)

Comprehensive (security-critical):

# All PBIT tests
All 21 PBIT tests enabled

# All CBIT monitoring
All 21 CBIT tests with appropriate frequencies

Manufacturing Line¶

# Run all FBIT tests
fbit_can_data      # CAN loopback
fbit_serial_data   # Serial loopback
fbit_gpio_data     # GPIO loopback
fbit_ssd           # Storage performance
fbit_usb           # USB functionality
fbit_tcp_data      # Network throughput

Best Practices¶

PBIT Best Practices¶

1. Enable all critical hardware checks
2. Set realistic thresholds (not too tight)
3. Don't disable security checks
4. Test after configuration changes

CBIT Best Practices¶

1. Choose appropriate frequencies (don't poll too fast)
2. Monitor critical resources (CPU, memory, disk)
3. Enable security monitoring (whitelist checks)
4. Review logs regularly for trends

FBIT Best Practices¶

1. Run in controlled environment (not production)
2. Ensure proper hardware setup (loopbacks, etc.)
3. Document pass/fail criteria clearly
4. Include in manufacturing checklist

Next Steps¶

• bit-learn Guide - Generate test configurations
• bit-inspect Guide - Inspect test details
• Running Tests - Execute tests
• Test Reference - Complete test catalog