-
-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Open
Labels
Description
What happened?
When LiteLLM is used as a proxy for AWS Bedrock, it returns incorrect error responses, which makes it difficult to troubleshoot, because AWS SDKs (at least AWS GO SDK v2) cannot parse error responses and produce meaningless messages.
Request example:
This request returns an error because messages[].content[].document.source must be either bytes or s3Location.
URL:
POST <litellm_proxy_host>/bedrock/model/us.anthropic.claude-3-5-haiku-20241022-v1%3A0/converse
Body:
{
"messages": [
{
"content": [
{
"document": {
"format": "txt",
"name": "aws-bedrock-description",
"source": {
"text": "Amazon Bedrock is a fully managed service from Amazon Web Services (AWS) that provides access to a variety of foundation models (FMs) for building and scaling generative AI applications. It simplifies development by offering a serverless platform to experiment with, customize with your own data, and deploy AI models securely without managing infrastructure. Key features include the ability to create agents for complex tasks, use knowledge bases to connect apps to your data, and leverage tools for fine-tuning and responsible AI."
}
}
},
{
"text": "Describe the content of the attached document"
}
],
"role": "user"
}
]
}Expected response: the same that comes from AWS Bedrock API
Headers:
X-Amzn-Errortype: ValidationException:http://internal.amazon.com/coral/com.amazon.bedrock/
X-Amzn-Requestid: 49b75565-cc59-4b17-aae5-2bce605504e0
Body:
{
"message": "DocumentSource object at messages.0.content.0.document.source must set one of the following keys: bytes, s3Location."
}Actual response:
X-Amzn-Errortype, X-Amzn-Requestid are absent
Body has different structure:
{
"detail": {
"error": "{\"message\":\"DocumentSource object at messages.0.content.0.document.source must set one of the following keys: bytes, s3Location.\"}"
}
}Found while using AWS GO SDK v2 while printing errors.
Expected error message:
operation error Bedrock Runtime: Converse, https response error StatusCode: 400, RequestID: 49b75565-cc59-4b17-aae5-2bce605504e0, ValidationException: DocumentSource object at messages.0.content.0.document.source must set one of the following keys: bytes, s3Location.
Actual error message:
operation error Bedrock Runtime: Converse, https response error StatusCode: 400, RequestID: , api error UnknownError: UnknownError
Relevant log output
Are you a ML Ops Team?
No
What LiteLLM version are you on ?
v1.79.1
Twitter / LinkedIn details
No response