Consider making a number of properties on `ChatCompletionOptions` public

[eiriktsarpalis]

Confirm this is a feature request for the .NET library and not the underlying OpenAI API

• This is a feature request for the .NET library

Describe the feature or improvement you are requesting

I'm currently implementing a feature in Microsoft.Extensions.AI that maps OpenAI chat completion requests (as represented in the ChatCompletionOptions type) to the Microsoft.Extensions.AI model. This is to accelerate development of Copilot agents that must be able to accept completion requests in the OpenAI wire format.

One of the issues I've encountered is that a number of important properties in the ChatCompletionOptions class are currently marked internal. This includes the Messages property, the Stream property, and the Model property.

While I've been able to work around the issue temporarily using reflection, this isn't a viable solution longer term. Please consider making these (and other properties in the model) public so that your types can be used as DTOs in the context of agents implementing the OpenAI wire format.

Additional context

No response

[KrzysztofCwalina]

I think the right way to implement the abstractions is to use the protocol methods. They will not only give you access to all, otherwise internal, properties, but you will be able to avoid the cost of instantiating OpenAI model types and then copying their properties into the abstraction layer model types. You can read about the protocol methods here: https://learn.microsoft.com/en-us/dotnet/azure/sdk/protocol-convenience-methods?tabs=convenience-methods%2Csystem-clientmodel

[eiriktsarpalis]

Correct me if I'm wrong, but it appears protocol methods necessitate working with a client. In the case of agents though we require processing the wire format from the perspective of a server.

From our perspective, the cost of allocating the intermediate models is not a huge source of concern, we do that already on the client side of things. What we would want to avoid though is hardcoding/reimplementing the OpenAI wire format, as such being able to use the existing DTOs from the server side would be amazing.

cc @stephentoub

[KrzysztofCwalina]

Yeah, I did not mean to imply that the model allocations are something you should worry about. I was just saying that protocol methods give you all the flexibility including some additional perf benefits. I understand that the cost of implementing these abstractions would be much higher if you were to call the protocol methods, and so I am not proposing you do it whole sale. But would it be reasonable to do it just for the APIs where you need to access the additional properties? Also, are you aware that for output models (models returned from client methods), you can access all the "internal" properties using raw content?, i.e. take a look at ClientResult.GetRawResponse

[bbartels]

To add to the list it would be useful to get access to:

openai-dotnet/src/Custom/Chat/ChatCompletionOptions.cs

Line 49 in c046c34

internal InternalChatCompletionStreamOptions StreamOptions { get; set; }

Currently getting around it with:

public static class OpenAIHelper
{
    public static ChatCompletionOptions GetOptionsNotIncludingStreamUsage()
    {
        ChatCompletionOptions options = new ChatCompletionOptions();

        var streamOptionsPropertyInfo = typeof(ChatCompletionOptions).GetProperty("StreamOptions", BindingFlags.NonPublic | BindingFlags.Instance);

        var streamOptionsInstance = streamOptionsPropertyInfo!.GetValue(options);

        var streamOptionsType = streamOptionsInstance!.GetType();
        var includeUsagePropertyInfo = streamOptionsType.GetProperty("IncludeUsage", BindingFlags.Public | BindingFlags.Instance);
        includeUsagePropertyInfo!.SetValue(streamOptionsInstance, false);

        return options;
    }
}

Which isn't exactly a very maintainable hack 😅

My use-case is quite specific however, this is for an integration test that ensure our openai proxy correctly appends "streamOptions": { "includeUsage": true } if not supplied

[stimpy77]

These DTOs should be public just on the basis of testability. While I have no desire to write my own unit or integration tests with the OpenAI SDK, my own code which leverages and needs orchestration behavior testing cannot be tested with unit tests because we can't mock the chat completion services properly. This is a real nuisance. And to be clear, I am already using an abstraction: Semantic Kernel. But I have to route through an Open AI wrapper to track token counts which SemKer hides.

[joseharriaga]

@stimpy77: For the purposes of unit testing, you can use the OpenAIChatModelFactory to create mocks:
🔗 https://github.com/openai/openai-dotnet/blob/main/api/OpenAI.netstandard2.0.cs#L1535

[joseharriaga]

@bbartels: The reason why StreamOptions is internal is because we set it on your behalf whenever you call the CompleteChatStreaming or CompleteChatStreamingAsync methods of the ChatClient. In other words, it is internal because you don't have to worry about it. Given that, do you still need to test it?

[bbartels]

Yes, I do still need it. But as i said, my use-case is extremely niche. The reason I need it is because we built our own openai proxy internally that automatically adds "streamOptions": { "includeUsage": true } to the request that is then proxied to openai. And I wrote some integration tests to test that proxy, one of which does not set includeUsage to ensure that it still comes back with usage information. Hence why i am using reflection to disable it.
But can totally understand that this is an extraordinarily uncommon use-case, so may not be worth adjusting

[verdie-g]

Same here, I'm trying to build an OpenAI proxy. I can deserialize the ChatCompletionOptions fine but then I can't use it because many properties are internal. In addition to the internal properties mention before, I would also need to access the child classes of OpenAI.Chat.ChatResponseFormat to know if the format is text or JSON.

[verdie-g]

Here is the full list of properties I would need to access:

• ChatCompletionOptions.Messages
• ChatCompletionOptions.Model
• ChatCompletionOptions.Stream
• ChatResponseFormat.Kind
• InternalDotNetChatResponseFormatJsonSchema.JsonSchema
• InternalDotNetChatResponseFormatJsonSchemaJsonSchema.Description
• InternalDotNetChatResponseFormatJsonSchemaJsonSchema.Name
• InternalDotNetChatResponseFormatJsonSchemaJsonSchema.Schema
• InternalDotNetChatResponseFormatJsonSchemaJsonSchema.Strict
• ChatToolChoice._predefinedValue
• ChatToolChoice._function
• InternalChatCompletionNamedToolChoiceFunction.Name

Having public getters for those would allow building OpenAI endpoints using this library.

I'm currently using reflection as a work-around.