Skip to content

Conversation

@RunDevelopment
Copy link
Contributor

I added a test that mutates the image files we already have and checks that decoders do not panic for them. Current mutations include:

  1. Truncations like image tests for too.
  2. An extension where is just appends garbage.
  3. Header mutations where certain header bytes are changed.

This would have found a panic in OBT that was fixed by #28.


Unfortunately, this test is currently a lot slower than it has to be because image only supports using plugins when reading files. This forces the test to write the modified files to disk for every mutation that we want to test. Writing to disk makes the test a lot slower than it has to be, but it's not too bad. It only takes 2 sec on my machine.

I made image-rs/image#2662 to address this API oversight. But even if a solution gets added, we probably won't be able to use the solution right away due to version compatibility.

@fintelia
Copy link
Contributor

This sounds like it is implementing fuzzing. To quote the readme:

Fuzzing is not a priority for this crate and decoders may panic or worse on malformed input. Please do not open issues for crashes found by fuzzing, unless they are memory safety violations, though PRs fixing them are welcome.

@RunDevelopment
Copy link
Contributor Author

Kinda? It qualifies as fuzzing in the sense that it uses (mostly) invalid inputs.

My understanding was that fuzzing isn't a priority to minimize maintenance burden. Random fuzzing can find crashes whenever, making the workload unpredictable. Worse, it's likely on the maintainer to triage and fix those crashes once discovered.

But those downsides don't really apply here. The mutations done here are deterministic. So if we start in a valid state (as we are now), they'll only find crashes when something changes. E.g. the implementation changes, new test files get added, dependencies get updated. And since changes are introduced via PRs, it should be on the PR author to fix the crashes they introduced or uncovered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants