Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use random identifier in DuckDB attachment name #205

Merged
merged 1 commit into from
Dec 25, 2024

Conversation

phillipleblanc
Copy link
Collaborator

This PR addresses a concurrency issue where multiple queries attempting to attach the same DuckDB database would fail with catalog write-write conflicts.

Problem

When multiple queries run concurrently and each tries to attach a database using the same attachment name (e.g., "attachment_0"), they create separate transactions that attempt to modify DuckDB's catalog simultaneously. DuckDB enforces strict serialization of catalog modifications to maintain consistency, causing errors like:
TransactionContext Error: Catalog write-write conflict on create with "attachment_0"

Example of the failing scenario:

Query 1: BEGIN
Query 1: Check if attachment_0 exists (sees it doesn't exist)
Query 2: BEGIN
Query 2: Check if attachment_0 exists (sees it doesn't exist)
Query 1: Try to create attachment_0
Query 2: Try to create attachment_0 -> CONFLICT! Transaction 1's change is not visible in Transaction 2's snapshot

Solution

Instead of having concurrent transactions compete to create the same attachment name, we now generate a unique identifier for each attachment. This means each transaction creates its own uniquely named attachment, avoiding catalog conflicts entirely.

@phillipleblanc phillipleblanc added the bug Something isn't working label Dec 24, 2024
@phillipleblanc phillipleblanc self-assigned this Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants