Testing dynamic JSON

Testing web applications, in my experience, has involved a lot of JSON munging. Make a request, get a response, assert on JSON. But sometimes the key you're interested in isn't included. In fact, it may be best practice to leave these keys out if they would be null [ref]. It's fine, we can handle it, but it starts to look like this:

def test_some_response(response: dict):
    assert response.get("my_key") is None

The comfort that a test like this provides is misleading. The call to dict.get returns None by default if "my_key" isn't in there which is maybe what we wanted.

But, the same call will return None if "my_key" is in there and its value is None.

Consider how the test would behave if we threw each of these responses in:

r1 = {"my_key": None}
r2 = {"my_key": "A value."}
r3 = {"wrong_key": "A value."}

r1 passes, r2 fails, and r3 passes.

If your application cares about the difference between including a key with a null value and not including the key at all, then it will be important to design tests that can detect the inclusion or exclusion of a key and whether or not its value is null.

There is a difference between missing and null data.

As a tester, even if your SUT considers both cases equivalent (missing vs. null), this should still drive you crazy.

If you want to discern between missing and null you might try this for missing keys:

def test_my_key_missing(response):
    assert "my_key" not in response

And this for null keys:

def test_my_key_null(response):
    assert["my_key"] is None

Or you might open your third eye, use glom, and never look back.