Safety-Critical Software

NASA’s paper “The Power of 10 Rules” presents a disciplined coding approach designed to increase confidence in software correctness. In safety-critical systems, where failure can cause irreversible harm, this discipline matters: reducing ambiguity, constraining complexity, and making behavior easier to understand, analyze, and verify before defects can propagate into real-world incidents.

The core rules are simple and intentionally strict:

1. All code must be restricted to a simple control flow that never includes goto statements, setjmp or longjmp constructs, or direct or indirect recursion.
2. All loops must have a fixed upper bound on the number of iterations, regardless of the input data, making it trivially possible to prove statically that the loop cannot exceed a preset upper bound.
3. Dynamic memory allocation is never allowed after initialization.
4. No function should be longer than can be printed on a single sheet of paper, using one line per statement and declaration.
5. Code assertion density should average at least two assertions per function. Assertions must check for anomalous conditions that should never occur during normal execution. All assertions must be side-effect-free and defined as Boolean tests. When an assertion fails, an explicit recovery action must be taken, such as returning an error condition to the caller. Any assertion that a static checking tool can prove can never fail or can never hold violates this rule.
6. Declare all data objects at the smallest possible level of scope.
7. A calling function must always check the return value of non-void functions, and all called functions must check the validity of all parameters provided by the caller.
8. The use of the preprocessor must be limited to the inclusion of header files and simple macro definitions. Token pasting, variable argument lists (ellipses), and recursive macro calls are never allowed. All macros must expand into complete syntactic units. The use of conditional compilation directives must be kept to a minimum.
9. The use of pointers must be restricted. Specifically, no more than one level of dereference should be used. Pointer dereference operations may not be hidden in macro definitions or inside typedef declarations. Function pointers are never permitted.
10. All code must be compiled, from the first day of development, with all compiler warnings enabled in the most pedantic mode available. All code must compile without warnings. A daily check of all code with a static source code analyser must be performed, and all analyses must pass with zero warnings.

The stakes are not theoretical.

When safety-critical software fails, consequences are immediate and often irreversible. Notable failures include:

• The Therac-25 radiation therapy machine, where a race condition and weak interlock design contributed to lethal overdoses in the 1980s.
• The Ariane 5 failure in 1996, triggered by an overflow when converting a 64-bit floating-point value to a 16-bit signed integer, which crashed both guidance systems.
• The Boeing 737 Max crashes, where MCAS relied on faulty angle-of-attack input and repeatedly commanded nose-down trim, contributing to two fatal accidents.

Standards and certification make these practices enforceable.

Standards and certification processes are essential for correctness and reliable operation in safety-critical software. Domain-specific standards such as DO-178C (aerospace), IEC 62304 (medical devices), and ISO 26262 (automotive) define lifecycle controls across requirements traceability, design, coding, testing, and verification. Compliance is mandatory before systems can be certified for deployment in safety-critical contexts, and it depends on rigorous documentation, testing, and independent verification.

Declarative programming can further support this assurance model when applied in the right places. By describing intended state and constraints explicitly, declarative approaches reduce hidden control flow, make dependencies visible, and improve traceability from requirement to implementation artifact. This can strengthen reviewability, static analysis, and reproducibility, especially in build, configuration, and deployment layers. It is not a replacement for verification and testing, but it is a strong force multiplier for them.

Deterministic build environments are part of that same safety case. A system cannot be treated as verified if its build depends on ambient host state, undeclared tools, mutable references, or ad hoc environment variables. In safety-critical work, reproducibility is not convenience engineering; it is assurance engineering. If two approved build hosts can produce different binaries from the same revision, certification evidence weakens, root-cause analysis slows, and rollback confidence erodes.

Some workflows still require controlled impurity, such as hardware-bound vendor tooling or protected local credentials. When unavoidable, impurity must be explicit, justified, narrowly scoped, and recorded in build documentation. Production and certification builds should reject silent impurity and require documented exceptions with reviewer approval.

Practical build-integrity checklist:

• No undeclared host tools in the build path
• No mutable or floating input references for release builds
• No hidden environment-variable dependencies in evaluation or build logic
• No unverified network fetches during build
• No host filesystem mounts that bypass declared inputs
• Reproducibility verification across at least two independent hosts

A quick implementation comparison makes the difference concrete.

The impact of these rules is easiest to see by comparing a typical unsafe implementation with one designed for correctness.

#include <stdio.h>
#include <stdlib.h>

int *samples = NULL;
int idx = 0;

void process_sensor(void) {
    while (1) {
        int v;
        if (scanf("%d", &v) != 1) break;

        if (!samples) samples = malloc(sizeof(int) * 1000);
        samples[idx++] = v;

        if (idx > 16) {
            goto report;
        }
    }

    report:
    long sum = 0;
    for (int i = 0; i < idx; ++i) sum += samples[i];
    double avg = (double)sum / idx;
    printf("Avg: %f\n", avg);
}

int main() {
    process_sensor();
    free(samples);
    return 0;
}

#include <stdio.h>
#include <stdbool.h>
#include <stdint.h>

#define SAMPLE_COUNT 16

static int sensor_read(void) {
    int v;
    if (scanf("%d", &v) != 1) return INT32_MIN;
    return v;
}

#define CHECK(cond, action) do { if (!(cond)) { action; } } while (0)

int compute_average(const int buf[], size_t n, double *out) {
    CHECK(buf != NULL, return -1);
    CHECK(out != NULL, return -1);
    CHECK(n > 0, return -2);

    long sum = 0;
    for (size_t i = 0; i < n; ++i) {
        sum += buf[i];
    }
    *out = (double)sum / (double)n;
    return 0;
}

int main(void) {
    int samples[SAMPLE_COUNT];
    size_t count = 0;

    for (size_t i = 0; i < SAMPLE_COUNT; ++i) {
        int v = sensor_read();
        if (v == INT32_MIN) {
            break;
        }
        samples[count++] = v;
    }

    double avg;
    int rc = compute_average(samples, count == 0 ? 1 : count, &avg);
    if (rc != 0) {
        fprintf(stderr, "compute_average failed (rc=%d)\n", rc);
        return 1;
    }

    printf("Average over %zu samples: %.3f\n", count, avg);
    return 0;
}

gcc -Wall -Wextra -pedantic -o safe_avg safe_avg.c

Correct source code is necessary, but not sufficient. In safety-critical systems, the build and release pipeline must be as deterministic, auditable, and constrained as the software it produces.