Skip to content

[Bug]: LiteLLM Proxy returns incorrect error response while proxying AWS Bedrock API #16819

@olte36

Description

@olte36

What happened?

When LiteLLM is used as a proxy for AWS Bedrock, it returns incorrect error responses, which makes it difficult to troubleshoot, because AWS SDKs (at least AWS GO SDK v2) cannot parse error responses and produce meaningless messages.

Request example:
This request returns an error because messages[].content[].document.source must be either bytes or s3Location.

URL:
POST <litellm_proxy_host>/bedrock/model/us.anthropic.claude-3-5-haiku-20241022-v1%3A0/converse
Body:

{
    "messages": [
        {
            "content": [
                {
                    "document": {
                        "format": "txt",
                        "name": "aws-bedrock-description",
                        "source": {
                            "text": "Amazon Bedrock is a fully managed service from Amazon Web Services (AWS) that provides access to a variety of foundation models (FMs) for building and scaling generative AI applications. It simplifies development by offering a serverless platform to experiment with, customize with your own data, and deploy AI models securely without managing infrastructure. Key features include the ability to create agents for complex tasks, use knowledge bases to connect apps to your data, and leverage tools for fine-tuning and responsible AI."
                        }
                    }
                },
                {
                    "text": "Describe the content of the attached document"
                }
            ],
            "role": "user"
        }
    ]
}

Expected response: the same that comes from AWS Bedrock API
Headers:

X-Amzn-Errortype: ValidationException:http://internal.amazon.com/coral/com.amazon.bedrock/
X-Amzn-Requestid: 49b75565-cc59-4b17-aae5-2bce605504e0

Body:

{
    "message": "DocumentSource object at messages.0.content.0.document.source must set one of the following keys: bytes, s3Location."
}

Actual response:
X-Amzn-Errortype, X-Amzn-Requestid are absent
Body has different structure:

{
    "detail": {
        "error": "{\"message\":\"DocumentSource object at messages.0.content.0.document.source must set one of the following keys: bytes, s3Location.\"}"
    }
}

Found while using AWS GO SDK v2 while printing errors.
Expected error message:

operation error Bedrock Runtime: Converse, https response error StatusCode: 400, RequestID: 49b75565-cc59-4b17-aae5-2bce605504e0, ValidationException: DocumentSource object at messages.0.content.0.document.source must set one of the following keys: bytes, s3Location.

Actual error message:

operation error Bedrock Runtime: Converse, https response error StatusCode: 400, RequestID: , api error UnknownError: UnknownError

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.79.1

Twitter / LinkedIn details

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions