Skip to content

Conversation

@NickCrews
Copy link
Contributor

This is a repro to show that using a large memtable on postgres gives an error. (at least I think this should repro, I am still having trouble getting postgres running in my env)

On my computer, this happens somewhere around 8400 rows, but is inconsistent, sometimes 8400 rows works, sometimes it errors. I'm getting this when connected to a hosted serverless neon.tech postgres instance.

I get a warning: WARNING:psycopg:error ignored terminating <psycopg.Pipeline [BAD] at 0x3488be590>: the connection is lost, and then an error at

cursor.execute(create_stmt_sql).executemany(sql, table.to_pylist())
:

OperationalError: sending prepared query failed: SSL error: bad length
SSL SYSCALL error: EOF detected

I can repo even more simply with pgbackend._register_in_memory_tables(big_memtable), but I thought it would be good to only test the public API

Perhaps this is due to the executemany() getting called too many times? Should we batch the memtable in smaller chunks?

If you give me any pointers I can try to put up a fix, but I don't understand the root cause yet.

@github-actions github-actions bot added the tests Issues or PRs related to tests label Nov 11, 2025
@NickCrews NickCrews added the postgres The PostgreSQL backend label Nov 11, 2025
@NickCrews NickCrews marked this pull request as draft November 11, 2025 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

postgres The PostgreSQL backend tests Issues or PRs related to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants