[MILESTONE 1.3] Streaming Support with Pre-Routing

[saschabuehrle]

Overview

Implement streaming support for cascade models with intelligent pre-routing based on query complexity.

Tasks

• Implement _streamResponseChunks() method
• Implement _selectModelPreRoute() for complexity detection
• Add streaming error handling and fallback
• Inject pre-routing metadata for observability
• Write 15+ streaming tests

Acceptance Criteria

• Streaming works from both drafter and verifier
• Pre-routing achieves >90% accuracy
• Cost tracking works during streaming
• Error recovery functional
• Compatible with LCEL .stream() API

Tests Required

• Streaming tests: 15+
• Pre-routing accuracy tests
• Error handling tests

Estimated Duration

2-3 days

Part of

LangChain Integration Phase 1 - Milestone 1.3