[BUG] CodeInterpreterCallImageOutput.DeserializeCodeInterpreterCallImageOutput throws invalid url length

[shanepowell]

Describe the bug

On the line with new Uri(item.Value.GetString()) where the item.Value.GetString() returns: "data:image/png;base64,long base64 string"

i.e.:
if (item.NameEquals("url"u8))
{
imageUri = new Uri(item.Value.GetString());
continue;
}

I'm using OpenAI package version 2.7.0.

This using the gpt5.1 model with medium reasoning effort using the question:
Parse the time and city from the airport board shown in this image into a list.

I can provide the image if useful.

Steps to reproduce

This using the gpt5.1 model with medium reasoning effort against the responses API using the following question:
Parse the time and city from the airport board shown in this image into a list.

I can provide the image if useful. The image is sent the modal as a embedded image part after the text part above.

Code snippets

OS

Windows

.NET version

9

Library version

2.7.0

[jsquire]

Hi @shanepowell. Thanks for reaching out and we regret that you're experiencing difficulties. The behavior that you're inquiring about is a platform limitation with the Uri constructor, not something that the OpenAI client has insight into nor influence over. The limitation was removed in .NET 10 via dotnet/runtime/117287. For more context, please see BUG: MessageContent.FromImageUrl(Uri) incompatible with "data url" (#135)

In order to use the longer data URLs returned by the OpenAI service, the recommended mitigation is to run against .NET 10. If you're unable to do so, the best approach on .NET 9 would be to use the protocol overload for the operation and read the data URL directly a s a string. A protocol example for the code interpreter tool can be found in this comment and an example of parsing responses to extract a base64 image can be found in this comment.

[shanepowell]

That is not data coming from me, but the openai code interpreter tool via the responses api. So the bug is that the OpenAi library does not support embedded images that the openai code interpreter tool can produce.

[jsquire]

That is not data coming from me, but the openai code interpreter tool via the responses api. So the bug is that the OpenAi library does not support embedded images that the openai code interpreter tool can produce.

The OpenAI library is limited to what the underlying .NET types support. As discussed in the linked issue, the platform removed limitations for uri length in .NET 10, and the decision was made not to make a breaking change in the OpenAI library to move from Uri => string for that field.

[shanepowell]

As I see it, it's a openai library problem as it has to deal with the data returned from openai not a Uri object problem. That data looks to be either a "url" or a "embedded image" (they have that convention in other parts of there api). e.g. fill in a the optional Uri when it's actually Uri or a optional BinaryData when it's a embedded image. That embedded image the openai code interpreter tool is sending out to you should never be put into a Uri object as it's not a uri.