fix(langchain): Wrap finish_reason in array for gen_ai span attribute#5666
Open
ericapisani wants to merge 1 commit intomasterfrom
Open
fix(langchain): Wrap finish_reason in array for gen_ai span attribute#5666ericapisani wants to merge 1 commit intomasterfrom
ericapisani wants to merge 1 commit intomasterfrom
Conversation
The gen_ai.response.finish_reasons attribute should be an array of strings per the semantic convention. Wrap the single finish_reason value in a list before setting the span data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Semver Impact of This PR🟢 Patch (bug fixes) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨Anthropic
Pydantic Ai
Other
Bug Fixes 🐛
Documentation 📚
Internal Changes 🔧Anthropic
Docs
Openai Agents
Other
🤖 This preview updates automatically when you update the PR. |
Contributor
Codecov Results 📊✅ 13 passed | Total: 13 | Pass Rate: 100% | Execution Time: 8.12s All tests are passing successfully. ✅ Patch coverage is 100.00%. Project has 14158 uncovered lines. Files with missing lines (1)
Generated by Codecov Action |
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Comment on lines
554
to
561
| finish_reason = generation.generation_info.get("finish_reason") | ||
| if finish_reason is not None: | ||
| span.set_data( | ||
| SPANDATA.GEN_AI_RESPONSE_FINISH_REASONS, finish_reason | ||
| SPANDATA.GEN_AI_RESPONSE_FINISH_REASONS, | ||
| [finish_reason], | ||
| ) | ||
| except AttributeError: | ||
| pass |
There was a problem hiding this comment.
Bug: The on_chat_model_end callback closes the span before on_llm_end can run, preventing the finish_reason from being set on chat model spans.
Severity: MEDIUM
Suggested Fix
The logic to set the finish_reason should be moved from on_llm_end to on_chat_model_end for chat model spans. This ensures the attribute is set before the span is closed and removed from the span_map.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: sentry_sdk/integrations/langchain.py#L554-L561
Potential issue: For chat models, LangChain invokes both `on_chat_model_end` and
`on_llm_end` for the same `run_id`. The `on_chat_model_end` callback calls `_exit_span`,
which closes the span and removes it from the internal `span_map`. Subsequently, when
`on_llm_end` executes, it cannot find the span for the given `run_id` and returns early.
This prevents the `GEN_AI_RESPONSE_FINISH_REASONS` attribute from being set on the span,
meaning chat model spans will be missing the `finish_reason` in production environments.
The existing tests may pass because the mocks do not replicate this dual-callback
behavior.
Did we get this right? 👍 / 👎 to inform future reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gen_ai.response.finish_reasons attribute should be an array of strings per the semantic convention. Wrap the single finish_reason value in a list before setting the span data.
Resolves #5664 and PY-2139