Skip to content

Conversation

@gorbak25
Copy link
Contributor

@gorbak25 gorbak25 commented Oct 23, 2025

This PR adds UNNEST which is an table valued function for zipping multiple Rc<Vec<Value>> arrays together. Arrays are passed to sqlite using it's pointer passing interface. The array type accepted by UNNEST is the same which is accepted by rarray in rusqlite.

This makes it possible to make very efficient bulk queries in sqlite. For ex. bulk insertion:

INSERT INTO foo (col1, col2, col3, col4) 
SELECT value0, value1, value2, value3 
FROM unnest($1, $2, $3, $4)

or selecting/deleting a list of objects in a single statement:

SELECT * FROM foo WHERE foo.id IN (SELECT value0 FROM unnest($1))

This is very powerful as most patterns of

for params in parameters:
     sql.query(..., params)

Can get converted to some sort of single statement using unnest.

I've exposed this feature on our Postgres emulation. Unfortunately due to some limitations array parameters need to be strongly typed and for now at most 10 array arguments per unnest call are supported. For now 4 array types are available via postgres - text[], blob[], int[] and float[].

To use UNNEST via postgres make an explicit type cast, for ex:

INSERT INTO foo (col1, col2, col3, col4) 
SELECT value0, value1, value2, value3 
FROM unnest(CAST($1 AS int[]), CAST($2 AS float[]), CAST($3 AS test[]), CAST($4 AS blob[]))

UNNEST is very efficient if there is significant network latency between the client and database.

@gorbak25 gorbak25 requested a review from somtochiama October 23, 2025 15:31
@gorbak25 gorbak25 changed the title Implement UNNEST in sqlite Implement UNNEST in sqlite. Expose it via the postgres emulation. Oct 23, 2025
Copy link
Contributor

@somtochiama somtochiama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 .

@gorbak25 gorbak25 force-pushed the gorbak/unnest-vtab branch 2 times, most recently from 9b372c4 to 386d977 Compare October 29, 2025 13:41
@gorbak25 gorbak25 merged commit bc00215 into main Oct 29, 2025
6 checks passed
@gorbak25 gorbak25 deleted the gorbak/unnest-vtab branch October 29, 2025 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants