Skip to content

Conversation

@Grantimus9
Copy link
Contributor

So far, this PR implements the following tests to support each given requirement to enable registration_tags to accept both strings and references to other strings and references to other arrays, which will be flattened appropriately.

Requirement 1: Should Resolve Referenced Strings

Pattern: ["literal1", "$inputs.string_tag"] where string_tag = "some_string_value" → ["literal1", "some_string_value"]

Schema Parsing Tests (test_schema_parser.py)

  • test_parse_block_manifest_when_manifest_defines_list_of_strings_and_string_selectors (line 536)
    • Verifies manifest parsing creates both primitive and selector definitions for mixed arrays with List[Union[WorkflowParameterSelector(STRING_KIND), str]]
  • test_parse_block_manifest_when_manifest_defines_union_of_string_and_string_selector (line 591)
    • Tests non-array union of string or selector: Union[str, WorkflowParameterSelector]
  • test_parse_block_manifest_when_manifest_defines_mixed_selectors_and_step_output_selectors (line 795)
    • Verifies mixed arrays with multiple selector types (WorkflowParameter and StepOutput)

Parameter Resolution Tests (test_step_input_assembler.py)

  • test_unfold_parameters_when_array_contains_mixed_literal_strings_and_dynamic_selectors (line 346)
    • Tests runtime resolution of mixed array with one dynamic selector: ["static-tag-1", "dynamic-tag-value", "static-tag-2"]
  • test_unfold_parameters_when_array_contains_mixed_literals_and_multiple_dynamic_selectors (line 380)
    • Tests multiple dynamic selectors in one array are all resolved in their correct positions
  • test_unfold_parameters_when_array_selector_resolves_to_batch (line 450)
    • Tests batched selector where each batch element gets different resolved values
  • test_unfold_parameters_when_array_has_mixed_literals_and_batch_selectors (line 492)
    • Tests mixed literals with both batch and non-batch selectors (non-batch broadcasts, batch unfolds)

Block-Level Tests (test_v2.py)

  • test_run_sink_with_mixed_literal_and_dynamic_registration_tags (line 431)
    • Verifies the dataset upload block receives correctly resolved mixed array at runtime
  • test_manifest_parsing_with_mixed_literal_and_selector_registration_tags (line 496)
    • Tests manifest parsing accepts mixed array with both $inputs and $steps selectors
  • test_manifest_parsing_with_all_selector_types_in_registration_tags (line 543) - Case 3
    • Validates mixed literals and selectors pattern: ["literal", "$inputs.dynamic", "$steps.output.tag"]

Integration Tests (test_workflow_with_active_learning_sink.py)

  • test_active_learning_workflow_with_custom_tag_value_resolves_correctly (line 182)
    • End-to-end test verifying workflow executes successfully with custom tag values in mixed array

Requirement 2: Should Resolve Referenced Arrays (with Flattening)

Pattern: ["literal1", "$inputs.string_tag"] where string_tag = ["value1", "value2"] → ["literal1", "value1", "value2"]

Parameter Resolution Tests (test_step_input_assembler.py)

  • test_unfold_parameters_when_array_element_resolves_to_array_and_should_flatten (line 534)
    • Tests selector resolving to array preserves nested structure at unfold layer (flattening occurs elsewhere)
  • test_unfold_parameters_when_batched_array_element_resolves_to_arrays (line 568)
    • Tests batched selector where each batch element resolves to different arrays

Requirement 3: Should Resolve Literal Strings in Array (Baseline)

Pattern: ["literal1", "A string"] → ["literal1", "A string"]

Schema Parsing Tests (test_schema_parser.py)

  • test_parse_block_manifest_when_manifest_defines_only_list_of_strings (line 706)
    • Verifies baseline case with only literal strings creates primitive type definition without selectors

Block-Level Tests (test_v2.py)

  • test_manifest_parsing_with_all_selector_types_in_registration_tags (line 543) - Case 1
    • Validates manifest parsing for array containing only literal strings: ["tag1", "tag2", "tag3"]

Requirement 4: Should Resolve Just String

Pattern: reference_tags = "literal1" → ["literal1"]

Schema Parsing Tests (test_schema_parser.py)

  • test_parse_block_manifest_when_manifest_defines_union_of_string_and_string_selector (line 591)
    • Tests schema parsing for non-array union accepting either string or selector

Block-Level Tests (test_v2.py)

  • test_manifest_parsing_with_all_selector_types_in_registration_tags (line 543) - Cases 1-3
    • Implicitly validates that both string and array formats are accepted by the manifest

Requirement 5: Should Resolve Just Selector That's an Array

Pattern: reference_tags = $inputs.string_tag where string_tag = ["value1", "value2"] → ["value1", "value2"]

Schema Parsing Tests (test_schema_parser.py)

  • test_parse_block_manifest_when_manifest_defines_list_of_strings_and_list_selector (line 647)
    • Tests Union[List[str], WorkflowParameterSelector] where entire parameter can be selector to array
  • test_parse_block_manifest_when_manifest_defines_only_string_selector_list (line 745)
    • Tests array containing only selectors (no literals): List[WorkflowParameterSelector(STRING_KIND)]

Parameter Resolution Tests (test_step_input_assembler.py)

  • test_unfold_parameters_when_array_has_only_dynamic_selectors (line 419)
    • Tests array with only dynamic selectors are all properly resolved

Block-Level Tests (test_v2.py)

  • test_manifest_parsing_when_registration_tags_given_as_selector (line 158)
    • Tests entire registration_tags parameter as single selector: "$inputs.tags"
  • test_manifest_parsing_with_all_selector_types_in_registration_tags (line 543) - Case 2
    • Validates single selector reference pattern: "$inputs.tags"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants