Property-based testing for engineers who have not tried it yet

The Fragility of Example-Based Testing

In the TypeScript ecosystem, we spend a significant amount of our time refining the boundaries of our data. We leverage discriminated unions, exhaustive switches, and the never type to ensure that our business logic is as robust as the compiler allows. When we write unit tests, we tend to follow a familiar pattern: we define a specific input, we execute a function, and we assert that the output matches a hardcoded expectation. This is example-based testing. It is comfortable, it is intuitive, and unfortunately, it is often fundamentally insufficient.

Example-based testing suffers from a "confirmation bias" of the developer. When I write a test case for a function that parses a union type, I am testing the scenarios that I can visualize. I am testing the happy path, perhaps one or two edge cases like empty strings or null pointers, and then I move on. However, the compiler—and the runtime environment—does not care about what I can visualize. It only cares about the mathematical space of all possible inputs. If a function accepts a string, it accepts any string, not just the ones I remembered to put in my it() blocks. When we rely solely on specific examples, we are essentially performing a blind walk through our own code, hoping we haven't missed a cavernous gap in our logic.

Property-Based Testing as a Constraint System

Property-based testing (PBT) shifts the perspective from "does this specific input produce this specific output" to "what invariants must always remain true regardless of the input?" Think of it as applying the same rigor to your tests that we apply to our type definitions. If your discriminated union is defined with three members, your switch statement must handle all three or the compiler rejects the program. Property-based testing is the logical extension of this mindset into the runtime.

In a PBT framework, you define a generator that produces valid data according to your type specifications. You then define a property—a predicate that must return true for any input generated. The test runner then proceeds to bombard your function with hundreds or thousands of randomized inputs, searching for the specific combination of values that causes your invariant to break. If a failure occurs, the framework doesn't just report the error; it performs 'shrinking.' It attempts to find the smallest, most concise input that triggers the bug, stripping away the noise until you are left with the exact edge case you failed to account for.

Bridging Types and Runtime Reality

As someone who builds tooling to enforce exhaustive union handling, I often see developers struggle with the transition between static type safety and runtime data. We use tools like Zod or TypeBox to validate incoming data, but what happens after that? We assume that because the type is narrowed, the logic is sound. However, logic errors are rarely about type mismatches in production—they are about incorrect state transitions or failed assumptions about data bounds.

Consider a function that processes a numeric field within a discriminated union. You might have a type definition that allows any number, but your logic might implicitly assume that the number is non-negative or within a specific range. In an example-based test, you might test 0, 1, and 100. PBT, however, will inject -0.0000000001, Number.MAX_SAFE_INTEGER, and NaN. These inputs are often the ones that break production systems, precisely because they fall outside the 'happy path' of our manual test suites. By defining properties, we force ourselves to document the intended constraints of our functions in code, rather than relying on tribal knowledge or implicit assumptions.

Implementation Strategy: From Concepts to Code

Implementing PBT in a TypeScript project requires a shift in how we structure our test files. You are no longer just calling functions; you are describing the domain. Let us look at how this might be implemented using a hypothetical testing framework structure, similar to how one might handle complex data shapes in an exhaustive switch statement.

// Conceptualizing the property check for a complex union type
// We ensure that for any valid 'Action' object, the state transition is pure.

property('state transition preserves consistency', () {
  forall(arbitraryAction(), (action) {
    const initialState = getInitialState();
    const nextState = reducer(initialState, action);
    
    expect(validateState(nextState)).toBeTrue();
    expect(nextState.history.length).toBe(initialState.history.length + 1);
  });
});

When we use PBT, we are essentially writing a fuzzing harness. The power lies in the arbitraryAction() generator. This generator must be able to produce any possible variant of your union. If you add a new member to your union and forget to update your generator, the property tests will start failing. This creates a powerful feedback loop: your test infrastructure becomes as strictly tied to your type definitions as your exhaustive switch blocks.

The Ergonomics of Compiler Constraints

For the skeptic, the primary argument against PBT is the overhead. Setting up generators for complex object trees feels like extra work compared to writing a simple assert(a === b). However, consider the cost of an unhandled runtime exception in a production system. In my work with the ESLint plugin for exhaustive unions, I prioritize the 'shift left' mentality: if the developer can catch a bug at compile time, they should. If they cannot, they should catch it in the most automated, thorough way possible.

Property-based testing is the ultimate 'shift left' for runtime logic. It exposes the bugs that occur in the hidden corners of your state space. When I look at a codebase, I look for where the developer has relied on hope rather than constraints. Hope is not a strategy. Type systems are a strategy. Exhaustive checking is a strategy. Property-based testing is the final pillar that ensures that even when data reaches the runtime, it cannot violate the core invariants that keep your application stable.

Why We Must Embrace Randomized Testing

We live in a world of increasingly complex state management. React applications, distributed systems, and microservices are all susceptible to 'the edge case from hell'—the one that only happens once every million requests. By manually writing examples, we are statistically unlikely to hit those cases. By utilizing PBT, we treat our code as a formal system that must be verified against its own requirements.

# Example of a configuration set for a property-based test runner
# Ensuring that even extreme inputs are tested to maintain system safety

test_configuration:
  iterations: 1000
  shrinking: true
  seed: "dynamic"
  failure_log: "./test/failures/last_run.json"
  constraints:
    max_depth: 5
    allow_NaN: false
    numeric_range: [-10000, 10000]

This configuration isn't just about 'trying more stuff.' It is about defining the boundaries of what your system should realistically tolerate. If your application cannot handle a number greater than 10,000, your code should either reject it explicitly at the type boundary or handle it gracefully. PBT highlights exactly where those boundaries are poorly defined. It forces you to confront the reality of your data structures.

The Future of Robust TypeScript Tooling

As our industry continues to prioritize type safety, we need to bring that same level of scrutiny to our testing suites. We have built compilers and linters that effectively act as gatekeepers for our type definitions, ensuring we never miss a branch of a discriminated union. Now, we must adopt PBT to act as the gatekeeper for our logic.

It is time to move beyond the comfort of the static, predictable unit test. It is time to treat our logic with the same suspicion that the TypeScript compiler treats our types. Every time you write an exhaustive match, you are asserting: 'I have considered all known possibilities.' Property-based testing allows you to say: 'I have verified that even for the possibilities I did not know existed, my code remains correct.' This is the standard we should hold ourselves to in a modern, professional development environment. Stop relying on your own imagination to find bugs. Let the machine do the heavy lifting, and let your code stand up to the rigors of the unexpected.