Skip to content

COMPATIBILITY rule fails with Protobuf schemas registered via Confluent API (ccompat/v7) using FileDescriptorSet format #6831

@vmartykhin

Description

@vmartykhin

Description

When APICURIO_RULES_GLOBAL_COMPATIBILITY: FORWARD is enabled, registering new Protobuf schema versions via Confluent-compatible API (/apis/ccompat/v7) fails with InternalServerError 0 Syntax error in :1:1: unexpected label: CiV.... The error occurs because Confluent .NET Protobuf serializer registers schemas as base64-encoded FileDescriptorSet instead of .proto text, and the compatibility check parser expects .proto source text.

This appears related to #2440, but occurs specifically when global compatibility rules are enabled.

Environment

  • Apicurio Registry version: 3.0.9 (using apicurio/apicurio-registry:3.0.9)
  • API: Confluent-compatible /apis/ccompat/v7
  • Client: Confluent.SchemaRegistry.Serdes.Protobuf (.NET) via KafkaFlow
  • Configuration:
    APICURIO_RULES_GLOBAL_COMPATIBILITY: FORWARD
    KAFKA_BOOTSTRAP_SERVERS: kafka:29092

Steps to Reproduce

  1. Configure Apicurio Registry with:

    environment:
      APICURIO_RULES_GLOBAL_COMPATIBILITY: FORWARD
  2. Register first Protobuf schema version using Confluent .NET client:

    middlewares.AddSchemaRegistryProtobufSerializer(
        new ProtobufSerializerConfig { 
            SubjectNameStrategy = SubjectNameStrategy.TopicRecord 
        }
    );
  3. First version registers successfully.

  4. Modify the .proto file (e.g., add a field):

    message SomeMessage {
        // ... existing fields
        bool NewField = 15;  // new field
    }
  5. Attempt to register the updated schema (produce a message with the new schema).

Expected vs Actual Behaviour

Expected

  • Compatibility check should succeed (adding a new optional field in Proto3 is forward-compatible).
  • New schema version should be registered successfully.

Actual

  • Registration fails with:
    InternalServerError 0 Syntax error in :1:1: unexpected label: CiVLYWZrYS9Qcm90b2J1Zi9jaGFyZ2luZy1zZXNzaW9uLnByb3RvEhJBdG9tLkVtc3AuU2Vzc2lvbnM...
    
  • The string starting with CiV... is a base64-encoded FileDescriptorSet.

Logs

System.Net.Http.HttpRequestException: [http://apicurio-registry:8080/apis/ccompat/v7/] InternalServerError 0 Syntax error in :1:1: unexpected label: CiVLYWZrYS9Qcm90b2J1Zi9jaGFyZ2luZy1zZXNzaW9uLnByb3RvE*
   at Confluent.SchemaRegistry.RestService.ExecuteOnOneInstanceAsync(Func`1 createRequest)
   at Confluent.SchemaRegistry.RestService.RequestAsync[T](String endPoint, HttpMethod method, Object[] jsonBody)
   at Confluent.SchemaRegistry.RestService.RegisterSchemaWithResponseAsync(String subject, Schema schema, Boolean normalize)
   at Confluent.Shared.CollectionUtils.ConcurrentDictionaryExtensions.GetOrAddAsync[TKey,TValue](ConcurrentDictionary`2 dictionary, TKey key, Func`2 valueFactory)
   at Confluent.SchemaRegistry.CachedSchemaRegistryClient.RegisterSchemaWithResponseAsync(String subject, Schema schema, Boolean normalize)
   at Confluent.SchemaRegistry.Serdes.ProtobufSerializer`1.SerializeAsync(T value, SerializationContext context)
   at KafkaFlow.ConfluentSerializerWrapper.InnerConfluentSerializerWrapper`1.SerializeAsync(Object message, Stream output, ISerializerContext context)
   at KafkaFlow.Middlewares.Serializer.SerializerProducerMiddleware.Invoke(IMessageContext context, MiddlewareDelegate next)
   at KafkaFlow.Producers.MessageProducer.ProduceAsync(String topic, Object messageKey, Object messageValue, IMessageHeaders headers, Nullable`1 partition)
   at KafkaFlow.Producers.MessageProducer.ProduceAsync(String topic, Object messageKey, Object messageValue, IMessageHeaders headers, Nullable`1 partition)
   at MediatR.NotificationPublishers.ForeachAwaitPublisher.Publish(IEnumerable`1 handlerExecutors, INotification notification, CancellationToken cancellationToken)

Metadata

Metadata

Assignees

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions