Skip to content

Conversation

@zhangzhefang-github
Copy link

Summary

Fixes #34057 - Ensures that streaming mode includes llm_output field in LLMResult, fixing broken callback integrations.

Description

Previously, when using streaming mode (stream() or astream()), the LLMResult passed to on_llm_end callbacks was missing the llm_output field. This caused issues for callback handlers like Langfuse that rely on this field to extract metadata such as model names.

This PR adds llm_output={} to all streaming on_llm_end calls in both BaseLLM and BaseChatModel, ensuring consistency with non-streaming behavior.

Changes

  • Updated BaseLLM.stream() to include llm_output={} in LLMResult
  • Updated BaseLLM.astream() to include llm_output={} in LLMResult
  • Updated BaseChatModel.stream() to include llm_output={} in LLMResult
  • Updated BaseChatModel.astream() to include llm_output={} in LLMResult
  • Added unit test test_stream_llm_result_contains_llm_output() to verify the fix

Test Plan

  • ✅ All existing tests pass
  • ✅ New test verifies llm_output field is present and is a dict in streaming mode
  • ✅ Tested with GenericFakeChatModel using callback handler

@github-actions github-actions bot added fix core Related to the package `langchain-core` labels Nov 21, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Nov 21, 2025

CodSpeed Performance Report

Merging #34060 will not alter performance

Comparing zhangzhefang-github:fix/streaming-llm-output (b2abc0d) with master (525d5c0)1

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 13 untouched
⏩ 21 skipped2

Footnotes

  1. No successful run was found on master (ee3373a) during the generation of this report, so 525d5c0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

  2. 21 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Fixes langchain-ai#34057

Previously, streaming mode did not include the `llm_output` field in the
`LLMResult` object passed to `on_llm_end` callbacks. This broke integrations
like Langfuse that rely on this field to extract metadata such as model name.

This commit ensures that `llm_output` is always present in streaming mode by
passing an empty dict `{}` in all streaming methods (`stream` and `astream`)
for both `BaseLLM` and `BaseChatModel`.

Changes:
- Updated `BaseLLM.stream()` to include `llm_output={}` in LLMResult
- Updated `BaseLLM.astream()` to include `llm_output={}` in LLMResult
- Updated `BaseChatModel.stream()` to include `llm_output={}` in LLMResult
- Updated `BaseChatModel.astream()` to include `llm_output={}` in LLMResult
- Added test to verify `llm_output` is present in streaming callbacks
Fix test_event_stream_with_simple_chain to expect llm_output={} instead of llm_output=None in streaming mode, consistent with the fix for issue langchain-ai#34057
This commit comprehensively fixes issue langchain-ai#34057 where streaming mode
was returning LLMResult with llm_output: None instead of llm_output: {}.

Root cause: Multiple code paths were creating ChatResult/LLMResult
without explicitly setting llm_output={}, causing it to default to None.

Changes:
- chat_models.py: Added llm_output={} to cache retrieval paths (sync/async),
  generate_from_stream(), and SimpleChatModel._generate()
- llms.py: Added llm_output={} to SimpleLLM._generate() and _agenerate()
- fake_chat_models.py: Fixed all 4 fake model _generate() methods
- event_stream.py: Improved llm_output serialization in on_llm_end()
- test_runnable_events_v1.py: Updated test expectations

Tests:
- test_astream_events_from_model: PASSED ✓
- test_event_stream_with_simple_chain: PASSED ✓
- All linting checks: PASSED ✓
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Related to the package `langchain-core` fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming mode returns incomplete LLMResult (missing llm_output), breaking Langfuse callbacks

1 participant