Skip to content

Microsoft.SemanticKernel.Connectors.Sqlite changes may break existing In-Memory connection #250

@paulio

Description

@paulio

In previous releases the client code was responsible for loading the Vector extension, this required opening a connection and invoking LoadExtension. More recent code has taken responsibility for this, and in doing so has hidden the underlying connection. The new code relies on Connection Pooling. However, if the database is an in-memory database then whilst the previous code would work, the later code breaks as the connection is lost after any action, e.g. CreateCollectionIfNotExistsAsync will create a new table and immediately lose it upon completion of the invoke. To use the new code with an In-memory database appears to require the client to maintain their own connection during the scope of any SK actions.
E.g.

connectionString = "Data Source=:InMemorySample:;Mode=Memory;Cache=Shared";
using (var connection = await GetConnectionAsync(connectionString))
{
    var collection = new SqliteVectorStoreRecordCollection<string, CustomClass>(connectionString, collectionName);
    await collection.CreateCollectionIfNotExistsAsync();
    var doesExist = await collection.CollectionExistsAsync();     
}

private static async ValueTask<SqliteConnection> GetConnectionAsync(string connectionString, CancellationToken cancellationToken = default)
{
    var connection = new SqliteConnection(connectionString);
    await connection.OpenAsync(cancellationToken).ConfigureAwait(false);
    connection.LoadExtension("vec0");
    return connection;
}

[Edit] I.e. this isn't necessarily a bug, but perhaps it could be called out somewhere in the documentation or samples?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions