VerifyJson object comparison without order validation (at least for IEnumerable)

[lfritz-seweurodrive]

Is the feature request related to a problem

Yes. Problem which are causing me issues when wanting to use this library.

When comparing JSON using Quibble, currently it is impossible to ignore the order for specific list. This is an issue, as whilst there are lists where the order must be exactly the same, for unordered list this renders comparison difficult.

Why this should be possible:

Imagine a situation where one part of the JSON File differentiates in order with each run, e.g.

(a,b,c)
(b,c,a)
(c,b,a)

Imagine a situation where one part of the JSON File contains a list where one element was removed, but the others unchanged, each with a set of nested properties e.g.

(a: ["mykey: akey", "somevalue: somevalue", ...]), (b: ["mykey: bkey", "somevalue: somevalue", ...]), (c: ["mykey: ckey", "somevalue: somevalue", ...])

when b is removed, I would expect there to be an error - signifying that only b was removed. I would no

Describe the solution

I would like for there to be a flag which can be set as a setting on the VerifierSettings, VerifySettings, Quibbly and finally as attribute.
It should signify, that the ordner is not important for the object being compared.

Alternatively, IEnumerable should by default be handled as unordered, as IEnumerable is an unordered set by dotnet definition per default.
IEnumerable makes no guarantee regarding ordering, whereas the implementation in VerifyTest and Quibbly - assume one was made.

Describe alternatives considered

I tried using a IDictionary, but this does not cover my use case, as it requires the presence of a unique key. And this in turn, defeats the point of having a unordered set. I have multiple elements with similar data, same class being serialized, just differences in e.g. position.

Additional context

I tried creating this for the Quibbly github repo, but the project has been setup in a way which prevents creation of GitHub issues.

[SimonCropp]

can you share a some code that illustrates the problem

[lfritz-seweurodrive]

Schematically, the case I am struggling with (can't just change all IEnumerables to IList with specific orders everywhere, legacy code-base).

// Order of returned rows in SQL without ORDER BY is undefined, just like with an IEnumerable in C#. The order is non deterministic, usually governed by the SQL Execution Query Plan 
var myEnumerable = RunDBQuery("SELECT a, b, c FROM banana")
var finalObject = doSomeAdditionalProcessing(myEnumerable)
printElements(finalObject)
Verifier.Verify(finalObject)

Or if you need something testable, highlighting the non-guaranteed order of the ienumerable interface:

using System;
using System.Collections.Generic;
using System.Linq;

public class RandomOrderExample
{
    private static readonly Random _random = new Random();

	// IEnumerable in C# does not provide any guarantees about ordering
    public static IEnumerable<T> GetWithoutSpecificOrder<T>(IEnumerable<T> source)
    {
        return source.OrderBy(x => _random.Next());
    }

    public static async Task Main()
    {
        var items = new List<string> { "A", "B", "C", "D", "E" };
        var itemsAsUnorderedEnumerable = GetWithoutSpecificOrder(items);
        var verifierSettings = new VerifySettings();
        verifierSettings.UseStrictJson();
        verifierSettings.DontSortDictionaries();
        verifierSettings.DisableDiff();
        verifierSettings.UseDirectory("example");
        verifierSettings.UseFileName("example");
        await Verifier.Verify(itemsAsUnorderedEnumerable, verifierSettings);
    }
}

[SimonCropp]

try this

public class RandomOrderExample
{
    static Random random = new();

    public static IEnumerable<T> GetWithoutSpecificOrder<T>(IEnumerable<T> source) =>
        source.OrderBy(x => random.Next());

    [Fact]
    public Task Foo()
    {
        var items = new List<string> { "A", "B", "C", "D", "E" };
        var enumerable = GetWithoutSpecificOrder(items);
        return Verify(enumerable)
            .OrderEnumerableBy<string>(_=>_);
    }
}

[lfritz-seweurodrive]

Thanks, I'm glad that there's a helper function for this. But if you introduce the order, it would make existing verify files fail and the issue of a collection without order (IEnumerable) being compared as if it had one, remains

[SimonCropp]

sorry. i dont understand what you are trying to achieve

[lfritz-seweurodrive]

I am basically pitching for a change towards the default behaviour of VerifyTests regarding IEnumerable, so:

• Using IEnumerable
• -> Order is not assumed to exist, as long as all elements are in collectionA && collectionB -> approval pass
• Using IList or IOrderedEnumerable
• -> Order is assumed to exist, all elements are in collectionA && collectionB && sameOrder(collectionA, collectionB) -> approval pass

Using .OrderEnumerableBy e.g., would also not fully remedy the issue in case of a unordered collection not having a element - as depending on where the element is, the implementation of the comperator would throw an error in the direction of "element at position Y has changed from A B, subsequent elements are wrong as they have all changed as well", rather than "error, element A is missing"

[SimonCropp]

happy to consider a pull request