Realistic data. No real people.

Installation

Homebrew

$ brew install subveilhq/tap/subveil
$ brew upgrade subveil

To create a target schema automatically, install the PostgreSQL client tools as well:

$ brew install libpq

Manual download

Download the release archive for your platform from your approved Subveil distribution channel. Archives are available for macOS, Linux, and Windows on AMD64 and ARM64. Verify the archive with checksums.txt before placing the binary on your PATH.

$ tar -xzf subveil_<version>_darwin_arm64.tar.gz
$ sudo mv subveil_<version>_darwin_arm64/subveil /usr/local/bin/subveil
$ subveil version

Build from source

$ tar -xzf subveil_<version>_<platform>_<arch>.tar.gz
$ cd subveil_<version>_<platform>_<arch>
$ make build
$ make test

The binary is written to bin/subveil. Go 1.23 or newer is required.

Quick start

Build with Go 1.23 or newer, provide three DSNs/secrets, create the target schema, then inspect and copy.

Build

$ cd subveil
$ make build

First copy

export SUBVEIL_SOURCE_DSN="postgresql://readonly@localhost:5432/app"
export SUBVEIL_TARGET_DSN="postgresql://developer@localhost:5432/app_staging"
export SUBVEIL_SEED="replace-with-a-secret-seed"
​
$ pg_dump --schema-only --no-owner "$SUBVEIL_SOURCE_DSN" | psql "$SUBVEIL_TARGET_DSN"
$ subveil init
$ subveil plan
$ subveil pull --dry-run
$ subveil pull

Cloud mode

Cloud mode is enabled when a project ID is configured and CLI credentials are available. It exchanges schema metadata, policy configuration, and signed receipt information, not row data.

Options

$ subveil pull --config subveil.yaml --cloud
$ subveil pull --config subveil.yaml --no-cloud

Use --cloud to require policy sync and audit submission. Use --no-cloud to disable cloud communication. --require-audit fails the pull when policy retrieval or receipt submission fails.

Target schema creation

--create-schema

When available, --create-schema uses pg_dump to create the target schema inside the pull transaction. The target must not already contain tables and the source and target must be different databases.

$ subveil pull --config subveil.yaml --create-schema
​
# Or create it manually:
$ pg_dump --schema-only --no-owner "$SUBVEIL_SOURCE_DSN" | psql "$SUBVEIL_TARGET_DSN"

Commands

subveil init

Introspect the source and write a generated configuration.

$ subveil init --dsn "$SUBVEIL_SOURCE_DSN" --schemas public,health --out subveil.yaml
wrote subveil.yaml
introspected 64 tables · 112 foreign keys

subveil plan

Validate the graph and show the planned load levels without writing rows.

$ subveil plan --dsn "$SUBVEIL_SOURCE_DSN"
level 0  public.tenants (1,204 rows)
level 1  public.users (8,420 rows), public.pets (42,891 rows)
level 2  public.appointments (109,230 rows)
cycle    public.users ↔ public.profiles · pass 2 required

subveil pull

Copy the planned rows into the target in one transaction.

$ subveil pull --config subveil.yaml --dry-run
pass 1  public.tenants       1,204 rows
pass 1  public.users          8,420 rows
pass 2  public.profiles       8,420 rows
summary 18 tables · 142,330 rows · masked 17 columns · target unchanged
​
$ subveil pull --config subveil.yaml --truncate
pass 1  ...
pass 2  ...
complete 142,330 rows

Create the target schema as part of the pull and copy tables in parallel:

$ subveil pull --config subveil.yaml --create-schema --jobs 4

subveil login

Authenticate this machine with the Subveil control plane. The CLI prints a code and opens /device in your browser; approve the request there and the token is stored locally. It is only needed for control-plane features such as policy sync and audit receipts.

$ subveil login
Open https://subveil.dev/device and enter code ABCD-1234
waiting for approval…
logged in

subveil logout

Delete locally stored credentials. This does not revoke an API key.

$ subveil logout
removed ~/.config/subveil/credentials.json

Configuration: subveil.yaml

Field reference

Annotated example

version: 1
​
source:
  dsn_env: SUBVEIL_SOURCE_DSN
target:
  dsn_env: SUBVEIL_TARGET_DSN
masking:
  seed_env: SUBVEIL_SEED
​
subset:
  roots:
    - table: public.tenants
      where: "id = 'acme'"
  full_tables:
    - public.currencies
  exclude_tables:
    - public.audit_events
​
tables:
  public.customers:
    columns:
      email: { transformer: email, domain: example.test }
      phone: { transformer: phone }
      profile: { transformer: jsonb, paths: ["$.address.street", "$.date_of_birth"] }
  public.patients:
    columns:
      name: { transformer: name }
      medical_record: { transformer: redact, value: "[REDACTED]" }

Transformers

Reference

JSONPath cheat-sheet

$.a.b       nested key
[n]         array index
[*]         every array element
.*          every child key
['key']     key with punctuation
..key       recursive key search

Each transformer section should be tested against representative values before a production-like refresh. For nested PII, use jsonb with explicit paths.

Unique columns

Subveil detects primary keys, unique constraints, and unique indexes. email and uuid are always unique-safe. name, address, and phone use unique-aware suffixes when required. Avoid redact on unique columns because a constant value cannot preserve uniqueness.

Ava Smith #3fa91c07
Oak St, Springfield, CA 12345 #3fa91c07
+1-555-012-3456 x48213907

How determinism works

HMAC-SHA256

For each supported value, Subveil computes HMAC-SHA256(seed, kind ‖ exact input). The same input produces the same output in every table and run, so joins on masked columns still match.

same seed + same kind + same exact input → same output
changing the seed → changes every masked output
NULL and empty strings → left unchanged
email and uuid → 48-bit token; input is case-sensitive

Unique safety

How subsetting works

Roots, children, and parents

roots ──down──► children
  ▲                 │
  └────── up ───────┘
​
roots select the slice; children follow their foreign keys;
parents are pulled when needed, but their other children are not.

Use full_tables for lookup data such as currencies. Use exclude_tables for data that should never be copied. With no roots, the default is a full copy.

Circular foreign keys

Subveil detects strongly connected components with Tarjan's algorithm. It tries these strategies in order:

1. null_then_update: insert a nullable edge, then back-fill it in pass 2.
2. defer_constraint: defer a DEFERRABLE constraint inside the target transaction.
3. replica_role: make the target constraint DEFERRABLE inside the transaction, then restore it.

pass 1  users       created_by = NULL
pass 1  profiles    user_id = users.id
pass 2  users       created_by = profiles.id
commit   cycle resolved; all foreign keys valid

Safety guarantees

• One target transaction: all rows commit, or none do.
• Source and target identity is checked and refused when equal.
• A non-empty target is refused unless --truncate is explicit.
• The source is read-only; only session temporary tables may be created.
• Reads use a REPEATABLE READ snapshot.
• Sequences are reset after loading.
• Row values never appear in logs or errors.

Recipes

Refresh staging nightly

name: Refresh staging
on:
  schedule:
    - cron: "0 2 * * *"
jobs:
  refresh:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: make build
      - run: subveil pull --config subveil.yaml --truncate

Sample tenants and nested PII

subset:
  roots:
    - table: public.tenants
      scale: 0.05
  full_tables: [public.currencies]
​
tables:
  public.users:
    columns:
      profile: { transformer: jsonb, paths: ["$.address.street", "$.dob"] }

For unique columns, choose a unique-safe transformer and review the generated configuration before the first pull.

Troubleshooting

Limitations & roadmap

• The target schema is not created by Subveil.
• Tables load sequentially for now.
• PostgreSQL only.
• Name-based PII detection is a hint, not a guarantee.