Add comprehensive validator error message test suite #266
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…patibility This spec serves as a baseline for validator 1.0.76 error message format. It tests all major validator types to ensure error messages remain consistent after upgrading json-schema-validator to 1.5.8. All 20 tests pass with validator 1.0.76, confirming that the test expectations accurately capture the current error message behavior.
… suite Added ValidatorMessageFormatSpec with complete coverage of all JSON Schema Draft 4 validation keywords supported by json-schema-validator 1.0.76. ## Coverage (26 validators) - Numeric: minimum, maximum, multipleOf, exclusiveMinimum, exclusiveMaximum - String: minLength, maxLength, pattern - Array: minItems, maxItems, uniqueItems, items, additionalItems - Object: minProperties, maxProperties, required, additionalProperties, patternProperties, dependencies - Type: type, enum - Composition: allOf, anyOf, oneOf, not - Format: format ## Implementation Details - All test methods numbered sequentially (e1-e26) matching spec definition order - All assertions use beEqualTo with exact error messages for precise matching - No fuzzy contain() assertions - maximum confidence in error message stability This baseline test suite will be used to verify backward compatibility when upgrading from json-schema-validator 1.0.76 to 1.5.8.
oguzhanunlu
force-pushed
the
test-spec-on-old-validator
branch
from
e59aced to
58eeaa7
Compare
istreeter
approved these changes
Oct 31, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR adds a comprehensive test suite (
ValidatorMessageFormatSpec) to establish a baseline of error message behavior with json-schema-validator 1.0.76. This provides confidence that future validator upgrades maintain backward compatibility in error message formatting.Motivation
During the upgrade from json-schema-validator 1.0.76 to 1.5.8, we need to ensure that error messages remain consistent to avoid breaking changes for downstream consumers who may depend on specific error message formats.
Changes
ValidatorMessageFormatSpec: Comprehensive test suite covering all JSON Schema Draft 4 validation keywordsminimum,maximum,multipleOf,exclusiveMinimum,exclusiveMaximumminLength,maxLength,patternminItems,maxItems,uniqueItems,items,additionalItemsminProperties,maxProperties,required,additionalProperties,patternProperties,dependenciestype,enumallOf,anyOf,oneOf,notformatbeEqualTowith exact error messages (no fuzzycontainmatching) for maximum confidenceNext Steps
This test suite will be used to verify that the upcoming upgrade to json-schema-validator 1.5.8 maintains backward compatibility in error message formatting.
ref: https://snplow.atlassian.net/browse/PDP-2203