Property-Based Testing for OSINT Parsers: Find the Bugs Your Examples Miss

A unit test proves the parser works on Example A. Example-based tests are great at capturing regressions — you fix a bug, write the test, it never comes back. What they are bad at is finding the bugs you never thought of. That is the job of property-based testing, and for an adapter that ingests scraped HTML from the open internet, it is not optional.

#The failure mode

You wrote a parser for Czech ARES results. You tested it against 20 real responses. Shipped. Six months later, an ARES result comes back with a Unicode diaeresis in an address field, and your parser crashes because your regex assumed ASCII. No example test would have caught this — you had no example with a diaeresis.

Property tests catch it because they generate inputs you would never write by hand.

#The shape

defmodule PrismaticOsint.Parsers.AresTest do
  use ExUnit.Case
  use ExUnitProperties

  property "parser never crashes on any string" do
    check all input <- string(:printable, max_length: 10_000) do
      assert {:ok, _} = safe_parse(input)
    end
  end

  property "round-trip: encode then decode is identity" do
    check all record <- ares_record_generator() do
      encoded = AresEncoder.encode(record)
      assert {:ok, ^record} = AresParser.parse(encoded)
    end
  end
end

The first property is a fuzz test masquerading as a property: the parser must never raise. The second is a real property: encoding and decoding are inverses.

#Generators are where the value is

A weak generator produces boring inputs. A strong generator produces weird ones:

def ares_record_generator do
  gen all name <- unicode_name_generator(),
          ico <- ico_generator(),
          address <- address_generator(),
          owners <- list_of(owner_generator(), min_length: 0, max_length: 20) do
    %AresRecord{name: name, ico: ico, address: address, owners: owners}
  end
end

def unicode_name_generator do
  gen all parts <- list_of(string(:utf8, min_length: 1, max_length: 20), length: 1..4) do
    Enum.join(parts, " ")
  end
end

Unicode strings find encoding bugs. Empty lists find “must have at least one” assumptions. Nested structures find recursion bugs. Every weakness in your assumptions becomes a failing property — with a minimal counterexample courtesy of shrinking.

#Shrinking is the killer feature

When a property fails, StreamData automatically shrinks the input to the smallest example that still fails. A 10,000-character crash becomes a 4-character crash becomes a 1-character crash — and now you know exactly which character broke the parser. No debugging archeology.

#The rule

Every pure parser in lib/ gets one example test per real-world format and at least one property.

The example tests are for regressions. The property tests are for the bugs that have not happened yet. You need both.

#Where to go next

• Academy: Storage Patterns — parsers live close to storage
• Glossary: Property-Based Testing, Testing, Unit Testing, Shrinking, ExUnit

Example tests find yesterday’s bugs. Properties find tomorrow’s.