Fuzzing: Breaking Things with Random Inputs

A Simple Fuzzer

This is a very simple fuzzer that is implemented in the Fuzzing: Breaking Things with Random Inputs book. Note that the output of running the fuzzer will appear after this Python source code segment. What do you notice about the output? While it be the same each time this code is run?

import random
def fuzzer(max_length: int = 100, char_start: int = 32, char_range: int = 32) -> str:
    """A string of up to `max_length` characters in the range [`char_start`, `char_start` + `char_range`)"""
    string_length = random.randrange(0, max_length + 1)
    out = ""
    for i in range(0, string_length):
        out += chr(random.randrange(char_start, char_start + char_range))
    return out
fuzzer()

'=43"&64!8=++ ?.54(\'/+!:51,&0$<&?'

Activity: What do you think the benefits as well as trade-offs are with fuzzing programs?

Bugs Fuzzers Find

Fuzzers help developers to find errors that they need to fix. Some programs come with a max length for inputs, and when it is triggered it is called buffer overflows.

• In programming languages without exceptions, errors are often indicated by special return codes. For example in C, getchar() reads characters from input and returns EOF when no more input is available.
• If a programmer writes a loop to read characters until a space is encountered and the input ends, getchar() will return EOF repeatedly.
• Since EOF is not equal to a space, the loop will never stop and will run indefinitely.
• If you give a random number input that might be too large, is less than what you want, or is negative fuzzing can run into termination issues.

Checking for Errors

Even though fuzzing find errors simply, if the input space is “more discrete” then there needs to be more checks. Buffer overflows are a specific example of an issue where programs can access any part of memory, including areas that are uninitialized or not meant to be accessed. This offers high performance and control but increases the risk of mistakes. There are tools available to detect these problems during runtime, and they work well with fuzzing to improve reliability. In addition to general checkers for programs, you can create specific checkers tailored to your program. Assertions are a key technique for early error detection by verifying function inputs and results. While assertions can impact performance, they can be turned off in production code, with critical ones remaining active. They are particularly useful for checking the integrity of complex data structures. There is also the mypy a static checker that can detect type errors once argument types are correctly declared.

Runners

import subprocess
from typing import Any, List, Tuple, Union


class Runner:
    """Base class for testing inputs."""

    # Test outcomes
    PASS = "PASS"
    FAIL = "FAIL"
    UNRESOLVED = "UNRESOLVED"

    def __init__(self) -> None:
        """Initialize"""
        pass

    def run(self, inp: str) -> Any:
        """Run the runner with the given input"""
        return (inp, Runner.UNRESOLVED)


class ProgramRunner(Runner):
    """Test a program with inputs."""

    def __init__(self, program: Union[str, List[str]]) -> None:
        """Initialize.`program` is a program spec as passed to `subprocess.run()`"""
        self.program = program

    def run_process(self, inp: str = "") -> subprocess.CompletedProcess:
        """Run the program with `inp` as input. Return result of `subprocess.run()`."""
        return subprocess.run(self.program,
                              input=inp,
                              stdout=subprocess.PIPE,
                              stderr=subprocess.PIPE,
                              universal_newlines=True)

    def run(self, inp: str = "") -> Tuple[subprocess.CompletedProcess, str]:
        """Run the program with `inp` as input. Return test outcome based on result of `subprocess.run()`."""
        result = self.run_process(inp)

        if result.returncode == 0:
            outcome = self.PASS
        elif result.returncode < 0:
            outcome = self.FAIL
        else:
            outcome = self.UNRESOLVED
        return (result, outcome)

Activity: How do you think the runner relates to fuzzing? What do you think this runner does?