Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bring-your-own-pg-client™ (browser support) #782

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

gregnr
Copy link
Member

@gregnr gregnr commented Jun 17, 2024

Adds a PostgresMetaBase class that allows you to bring-your-own-pg-client™. Instead of depending on pg, the base class accepts custom query() and end() functions allowing you to choose how these methods are implemented.

The existing PostgresMeta class now extends PostgresMetaBase and implements query() and end() using pg exactly as before without breaking changes.

Why?

Browser support 🤓 ElectricSQL's pglite gives us a working Postgres instance in the browser (via wasm). Offering a platform-agnostic version of this lib means we can use it in any environment including the browser.

Example using PGlite

import { PGlite } from '@electric-sql/pglite'
import {
  PostgresMetaBase,
  PostgresMetaErr,
  PostgresTable,
  wrapError,
  wrapResult,
} from '@supabase/postgres-meta/base' // note '/base' - see below

const db = new PGlite()

// Implement `query` and `end` for PGlite
const pgMeta = new PostgresMetaBase({
  query: async (sql) => {
    try {
      const res = await db.query(sql)
      return wrapResult<any[]>(res.rows)
    } catch (error) {
      return wrapError(error, sql)
    }
  },
  // PGlite is single user/connection so doesn't require an end function
  end: async () => {},
})

const { data, error } = await pgMeta.tables.list({ includedSchemas: ['public'] })
// `data` contains public tables in PGlite 🚀

Other important notes

In order to make this platform-agnostic, all code needed to be pure TS (no native deps). We do 3 things:

  1. Introduce PostgresMetaBase class which doesn't import the pg dependency

  2. Create a new entrypoint base.ts with a package export at /base. This means you can import { PostgresMetaBase } from '@supabase/postgres-meta/base' without importing pg.

    Importing @supabase/postgres-meta will continue to use pg as before without breaking changes.

  3. Create custom loader for .sql files that live under ./src/lib/sql. Previously these were loaded using Node's fs API. Now they are imported directly like import tablesSql from './sql/tables.sql'.

    This custom import is accomplished by adding a lightweight bundler to the build step: tsup (esbuild bundler under the hood). So now tsup/esbuild run the build process instead of pure tsc. I did my best to make sure the dist outputs were consistent with previous tsc builds, but worth double checking (there are some differences, like bundling into single files). Also outputs both ESM and CJS outputs which should provide more flexibility to consumers.

All tests continue to pass. Only change needed was adding a custom vitest plugin to load .sql files, similar to what we do with tsup.

@gregnr gregnr requested review from a team as code owners June 17, 2024 03:52
@soedirgo
Copy link
Member

This is great! 💯

We're also slowly moving all pg-meta functionality to the main Supabase repo, which is pure-JS and is simply a SQL builder - would that be a better place for this?

There's a lot of work that still needs to be done for that though - feature parity, publishing the package somewhere, tests, the works.

@soedirgo
Copy link
Member

Or maybe we keep this feature here for now and switch to the pure-JS version once we reach parity? Wdyt?

@gregnr
Copy link
Member Author

gregnr commented Dec 10, 2024

@soedirgo wow sorry for the super long delay (I somehow had this repo's notifications muted 😬). Good to know about the migration to supabase/supabase (what was the motivation for this by the way?)

My use case for this was in database.build in order to grab table schema from PGlite as JSON and feed into the schema visualizer. @SaxonF will be doing something similar (in-browser) with new AI onboarding screens we are working on.

Do you think it should be as simple as migrating tables.sql to the new package? If so there's probably no need for this PR. Will the new package continue to be published as @supabase/postgres-meta?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants