Running Tests

Running Tests

This guide covers all the ways to run BIT tests and explains the different execution modes.

Execution Modes

One-Shot Mode

Run all tests once and exit. Useful for: - Initial system validation - CI/CD pipelines - Manual testing - Debugging

# Run all tests once
bit-manager -o

# Run with specific environment
BIT_CONFIG_PATH=/etc/bit BIT_TEST_PATH=/usr/local/lib/bit_manager bit-manager -o

Behavior: - Loads all PBIT and CBIT tests - Executes each test once - Displays results - Exits with status code (0 = all passed, non-zero = failures)

Service Mode (Default)

Run continuously as a system service:

# Enable service
sudo systemctl enable bit_manager

# Start service
sudo systemctl start bit_manager

# Check status
sudo systemctl status bit_manager

Behavior: - PBIT tests: Run once at startup - CBIT tests: Run continuously at configured intervals - Runs until stopped or system shutdown - Logs to files and systemd journal

Specific Test Mode

Run only specific tests by name:

# Single test
bit-manager -t pbit_cpu_usage -o

# Multiple tests
bit-manager -t pbit_cpu_usage -t cbit_memory_usage -t pbit_ethernet -o

# Mix PBIT and CBIT
bit-manager -t pbit_disk_usage -t cbit_disk_usage -o

Use cases: - Testing specific hardware components - Debugging individual tests - Validation after hardware changes - Focused monitoring

Environment Variables

Required Variables

BIT_CONFIG_PATH - Location of TOML configuration files - Default: Not set (must specify) - Example: /etc/bit or ./bit_manager/config

BIT_TEST_PATH - Location of test plugin .so files - Default: Not set (must specify) - Example: /usr/local/lib/bit_manager or ./target/debug

Setting Environment Variables

For single command: 

BIT_CONFIG_PATH=/etc/bit BIT_TEST_PATH=/usr/local/lib/bit_manager bit-manager -o

For session: 

export BIT_CONFIG_PATH=/etc/bit
export BIT_TEST_PATH=/usr/local/lib/bit_manager
bit-manager -o

System-wide (systemd service): Edit /etc/systemd/system/bit_manager.service: 

[Service]
Environment="BIT_CONFIG_PATH=/etc/bit"
Environment="BIT_TEST_PATH=/usr/local/lib/bit_manager"
ExecStart=/usr/bin/bit-manager

Command-Line Options

Basic Options

# Run all tests once
bit-manager -o
bit-manager --once

# Run specific test(s)
bit-manager -t test_name -o
bit-manager --test test_name -o

# Help
bit-manager -h
bit-manager --help

# Version
bit-manager -V
bit-manager --version

Examples

# Production: run as service
sudo systemctl start bit_manager

# Development: run once with local builds
BIT_CONFIG_PATH=./bit_manager/config BIT_TEST_PATH=./target/debug ./target/debug/bit-manager -o

# CI/CD: run critical tests only
bit-manager -t pbit_cpu_usage -t pbit_memory_usage -t pbit_disk_usage -o
EXIT_CODE=$?
if [ $EXIT_CODE -ne 0 ]; then
    echo "Critical tests failed"
    exit 1
fi

# Testing new test plugin
bit-manager -t my_new_test -o

Understanding Test Output

Console Output Format

2025-12-08 03:25:30.717 FBIT:[PASS] pbit_cpu_usage
2025-12-08 03:25:30.850 FBIT:[PASS] pbit_memory_usage
2025-12-08 03:25:31.123 FBIT:[PASS] pbit_disk_usage
2025-12-08 03:25:31.456 FBIT:[FAIL] pbit_gpu_loading
2025-12-08 03:25:31.789 FBIT:[PASS] pbit_ethernet

Format: - Timestamp (YYYY-MM-DD HH:MM:SS.mmm) - Log level (FBIT = Info) - Test status ([PASS] or [FAIL]) - Test name

Test Status

PASS ✓ - Test completed successfully - All checks passed - System meets expectations

FAIL ✗ - Test detected an issue - Threshold exceeded - Hardware missing or malfunctioning - Configuration mismatch

NotRun - Test hasn't executed yet - Test is disabled - Configuration error prevented execution

Logging

The project uses tracing as the logging backend and bridges existing log calls into it.

Configuration:

  • Default: Stdout logging is disabled to keep local runs quiet.
  • Enable Stdout: Opt-in with BIT_STDOUT_LOG=1 (the packaged systemd unit sets this so journald captures logs).
  • File Logging: Enable by setting BIT_LOG_FILE.
  • export BIT_LOG_FILE=1 -> Defaults to /var/log/bit/bit.log (daily rotating)
  • export BIT_LOG_FILE=/path/to/custom.log -> Logs to specified file
  • Verbosity: Control with RUST_LOG.
  • export RUST_LOG=debug
  • export RUST_LOG=info (default)

Performance: File logging uses a non-blocking background worker so application threads do not block on file I/O.

CI/Testing: For CI and test runs, you may want to disable file logging to avoid contention during heavy concurrent runs: 

export BIT_DISABLE_FILE_LOGGING=1

Test Execution Flow

PBIT Test Execution

  1. Load: Read configuration from $BIT_CONFIG_PATH/pbit_test_name.toml
  2. Initialize: Set up test parameters and expectations
  3. Execute: Run test logic once
  4. Validate: Compare results against thresholds/expectations
  5. Report: Return PASS or FAIL status

CBIT Test Execution

  1. Load: Read configuration including frequency setting
  2. Initialize: Set up monitoring parameters
  3. Schedule: Register for periodic execution
  4. Loop: Execute at configured intervals until stopped
  5. Check current state
  6. Compare against thresholds
  7. Log warnings/errors
  8. Sleep until next interval
  9. Report: Update status continuously

FBIT Test Execution

  1. Load: Read test-specific configuration
  2. Setup: Prepare test environment (files, connections)
  3. Execute: Perform hardware validation
  4. Verify: Check results (checksums, responses)
  5. Cleanup: Remove test artifacts
  6. Report: Return PASS or FAIL

Running Tests by Type

Running Only PBIT Tests

There's no built-in flag to run only PBIT tests, but you can specify them individually:

# List all PBIT tests
bit-inspect | grep "^pbit_" | cut -d: -f1 > pbit_tests.txt

# Run all PBIT tests
while read test; do
    bit-manager -t "$test" -o
done < pbit_tests.txt

Or use a script: 

#!/bin/bash
PBIT_TESTS="pbit_cpu_usage pbit_memory_usage pbit_disk_usage pbit_ethernet pbit_gpio"
for test in $PBIT_TESTS; do
    echo "Running $test..."
    bit-manager -t "$test" -o
done

Running Only CBIT Tests

Similarly for CBIT tests:

# Run specific CBIT tests
bit-manager -t cbit_cpu_usage -t cbit_memory_usage -t cbit_disk_usage -o

In service mode, CBIT tests run automatically at their configured intervals.

Running Only FBIT Tests

FBIT tests are typically run in manufacturing:

# Run all FBIT tests
bit-manager -t fbit_can_data -t fbit_gpio_data -t fbit_serial_data -t fbit_ssd -o

Exit Codes

bit-manager returns exit codes for scripting:

  • 0: All tests passed
  • Non-zero: One or more tests failed

Using Exit Codes in Scripts

#!/bin/bash
bit-manager -o
if [ $? -eq 0 ]; then
    echo "✓ All BIT tests passed"
    exit 0
else
    echo "✗ BIT tests failed"
    exit 1
fi

CI/CD Integration

# GitHub Actions example
- name: Run BIT Tests
  run: |
    export BIT_CONFIG_PATH=./bit_manager/config
    export BIT_TEST_PATH=./target/release
    ./target/release/bit-manager -o
  continue-on-error: false

Monitoring Test Execution

Real-Time Monitoring

Watch systemd service: 

# Follow logs
sudo journalctl -u bit_manager -f

# Follow with filtering
sudo journalctl -u bit_manager -f | grep FAIL

Watch log files: 

# Follow latest log
tail -f /tmp/bit-manager_INFO_*.log

# Watch for failures
tail -f /tmp/bit-manager_INFO_*.log | grep --color FAIL

Remote Monitoring with Zenoh

Use the built-in Zenoh publisher to stream results to other machines:

# Run the manager with Zenoh enabled
BIT_CONFIG_PATH=./bit_manager/config \
BIT_TEST_PATH=./target/release \
bit-manager --zenoh -o

On the observer host, run bit-monitor to subscribe to the bit/{hostname}/PBIT, CBIT, and FBIT topics:

# Optional: enable tracing logs if you want them in stdout
BIT_STDOUT_LOG=1 bit-monitor -hostname=<target-hostname>

bit-monitor prints each result to the console and, when BIT_STDOUT_LOG is set, also emits structured tracing logs (info on pass, error on fail) for journald or other collectors.

For information on implementing your own subscribers in other languages (Python, C++, etc.), see the Implementing Subscribers guide.

Scheduled Monitoring

CBIT tests run automatically. Check their intervals:

# View test frequencies
bit-inspect cbit_cpu_usage | grep "Run Frequency"
bit-inspect cbit_memory_usage | grep "Run Frequency"
bit-inspect cbit_disk_usage | grep "Run Frequency"

Debugging Failed Tests

Get Test Details

# See what the test checks
bit-inspect failing_test_name

# View full configuration
cat $BIT_CONFIG_PATH/failing_test_name.toml

Check Logs

# Search for specific test
grep "failing_test_name" /tmp/bit-manager_INFO_*.log

# View recent failures
grep "FAIL" /tmp/bit-manager_INFO_*.log | tail -20

# Check error messages
grep "ERROR" /tmp/bit-manager_INFO_*.log

Run Test Individually

# Run in isolation
bit-manager -t failing_test_name -o

# With verbose output
bit-manager -t failing_test_name -o 2>&1 | tee test_output.log

Check Permissions

Some tests need root or specific permissions:

# Try with sudo
sudo bit-manager -t failing_test_name -o

# Check if config file is readable
ls -l $BIT_CONFIG_PATH/failing_test_name.toml

# Check if test plugin exists
ls -l $BIT_TEST_PATH/libfailing_test_name.so

Common Execution Scenarios

Development Testing

# Quick build and test
cargo build && \
  BIT_CONFIG_PATH=./bit_manager/config \
  BIT_TEST_PATH=./target/debug \
  ./target/debug/bit-manager -o

Production Validation

# Run critical tests before deployment
CRITICAL_TESTS="pbit_cpu_usage pbit_memory_usage pbit_disk_usage pbit_ethernet"
for test in $CRITICAL_TESTS; do
    bit-manager -t $test -o || exit 1
done
echo "✓ System validated"

Scheduled Testing

Use cron for periodic validation:

# Add to crontab
# Run all PBIT tests daily at 3 AM
0 3 * * * BIT_CONFIG_PATH=/etc/bit BIT_TEST_PATH=/usr/local/lib/bit_manager /usr/bin/bit-manager -o >> /var/log/bit-daily.log 2>&1

Remote Testing

# Run tests on remote system
ssh user@remote-host "BIT_CONFIG_PATH=/etc/bit BIT_TEST_PATH=/usr/local/lib/bit_manager bit-manager -o"

# With sudo
ssh user@remote-host "sudo BIT_CONFIG_PATH=/etc/bit BIT_TEST_PATH=/usr/local/lib/bit_manager bit-manager -o"

Performance Considerations

Test Execution Time

  • PBIT tests: Typically 1-5 seconds each
  • CBIT tests: Similar per-execution, but run continuously
  • FBIT tests: Can be longer (disk I/O, network tests)

Resource Usage

  • CPU: Minimal when tests pass; higher during system checks
  • Memory: Low footprint (~10-50MB depending on active tests)
  • Disk I/O: Occasional reads (SMART data, logs)
  • Network: Minimal (interface status checks only)

Optimization Tips

  1. Adjust CBIT frequencies for less critical tests
  2. Disable unused tests in configuration
  3. Run FBIT tests separately (not in service mode)
  4. Use specific test mode instead of loading all tests

Troubleshooting

Tests Not Running

Check environment variables: 

echo $BIT_CONFIG_PATH
echo $BIT_TEST_PATH

Verify paths exist: 

ls -la $BIT_CONFIG_PATH/
ls -la $BIT_TEST_PATH/*.so

Service Won't Start

# Check service status
sudo systemctl status bit_manager

# View errors
sudo journalctl -u bit_manager -n 50

# Verify service file
cat /etc/systemd/system/bit_manager.service

# Reload and restart
sudo systemctl daemon-reload
sudo systemctl restart bit_manager

Permission Denied

# Run with sudo
sudo bit-manager -o

# Or set capabilities (for specific tests)
sudo setcap cap_net_admin=ep /usr/local/lib/bit_manager/libpbit_can.so

Next Steps

Edit this page on GitHub