Configuration

A project has two files. puppi.config.ts is committed. It names the entry, the build output, and every connection’s type. bindings.local.json stays on your machine. It names the environment you develop against, and the host and secrets for today.

puppi dev reads bindings.local.json. The engine never reads that file. An environment that is already running gets the same facts from puppi env bind and puppi env secret.

puppi.config.json is not read. The build expects defineConfig() exported from puppi.config.ts.

Write puppi.config.ts

import { defineConfig } from "@puppi/sdk";

export default defineConfig({
  entry: "src/pipeline.ts",
  out: "dist",
  defaultStorage: "warehouse",
  connections: {
    warehouse: { type: "postgres" },
  },
});

Field	Default	Meaning
entry	src/pipeline.ts	The pipeline module.
out	dist	Where bundle.js and manifest.json are written. puppi-env.d.ts is always written next to the config.
defaultStorage	none	Where an asset writes when it does not set storage. A string is one connection for every role. A map may set source, raw, model, and destination.
connections	{}	Connection name to a { type } object or a connector module.

defaultStorage roles other than those four are a build error. An asset’s storage field wins over the default. examples/hello uses one string, "warehouse".

entry and out are read by evaluating the config module, so they may be computed. The clients registered while that module loads are discarded. The bundle’s own load is what registers them for the executor.

Declare a connection

A built-in connection is a type the executor constructs:

connections: {
  warehouse: { type: "postgres" },
  lake: { type: "filesystem" },
}

Built-in types are postgres, filesystem, s3, and http.

A catalog connector is the module. See How connectors work.

import * as stripe from "@puppi/adapter-stripe";

connections: { stripe }

Optional fields on a { type } connection, or overrides when you need a narrower binding than the adapter allows:

Field	Meaning
config_schema	Map of config field to a type string. Wins over the adapter’s schema.
secret_fields	Secret field names. Wins over the adapter’s list.
clientModule	Module specifier for Client in puppi-env.d.ts, such as "./src/clearbit.js". A catalog module defaults to @puppi/adapter-<type>.

puppi build fills config_schema and secret_fields from the registered client, or from an installed @puppi/adapter-<type>. A type that nothing describes, and that does not set both fields itself, is a warning. The engine rejects that deployment.

A client declared in the pipeline uses adapter() from @puppi/sdk with type, Client, and optional configSchema, secretFields, and clientModule.

Write bindings.local.json

examples/hello is the shape puppi dev and puppi run --local read:

{
  "environment": "hello",
  "bindings": {
    "warehouse": {
      "config": {
        "host": { "env": "PGHOST", "default": "127.0.0.1" },
        "port": { "env": "PGPORT", "default": 5432 },
        "database": { "env": "PGDATABASE", "default": "engine_dev" },
        "user": { "env": "PGUSER", "default": "postgres" }
      },
      "secret_refs": { "password": "warehouse_password" }
    }
  },
  "secrets": {
    "warehouse_password": { "env": "PGPASSWORD", "default": "postgres" }
  }
}

A value is a literal or { "env": "NAME", "default": <value> }. The default’s type decides the variable’s type: a numeric default binds a number. For puppi dev, values are resolved where the engine runs. For puppi run --local, they are resolved in the process that runs the command. A variable that is set wins over the default.

puppi init writes this file from the config fields and secret fields of the chosen adapter.

Environment variables

Variable	Used by
PUPPI_ENV	Commands that need an environment, when --env is omitted.
PUPPI_ENGINE_URL	Engine base URL. Default http://127.0.0.1:4000.
PUPPI_API_TOKEN	Bearer token for /api/v1.
PUPPI_EXECUTOR_JS	Path to the built local harness, when it is not installed in the project and not found in this checkout.

Declare an asset

Declare assets in the entry module. The key is a string you choose, usually group/name.

Shared fields:

Field	Meaning
key	Identity. Required.
schema	Column contract. A { columns: [{ name, type }] } list, or a Standard Schema (Zod, Valibot, ArkType) whose output is an object. The manifest stores the column list either way.
storage	Connection this asset writes to. Wins over defaultStorage.
schedule	When the engine runs this asset on its own. See below.
deps	Extra asset keys the literal scan cannot see.
connections	Extra connection names the literal scan cannot see.
partitioning	{ by: "none" } (default), { by: "time", grain: "day" | "hour" }, or { by: "key", column }. An edge is rejected when both ends are partitioned and the specs differ. An unpartitioned end is compatible with either.
checks	schema_contract and custom. See Quarantine.
group	A value returned by group({ name, visibility }). Omitted assets use the implicit default group.
visibility	group, project, or public. Can only widen the group’s visibility.

Column type strings the contract compares are string, integer, float, boolean, json, and null, with |null for nullable. A Standard Schema number becomes float. Declare integer columns as integers (z.int()), because a whole-number value is observed as integer.

Source

source({
  key: "demo/customers",
  schema: Customer,
  connection: "stripe",
  mode: "full_refresh",
  run: async (ctx) => {
    await ctx.emit([]);
  },
});

Field	Meaning
run	Required. (ctx) => Promise<void>.
connection	The system this source reads. Omit it when the producer does not call one, as examples/hello does. Storage is storage or defaultStorage, a separate field.
mode	full_refresh (default) or append.
cursor	Required when mode is append. The watermark column.

full_refresh stores a complete copy. append stores a delta in one table and needs a watermark.

Model

A model is produced by run. Put SQL inside that function with ctx.query, the way the quickstart does.

The SDK also accepts sql: (ctx) => string in place of run. The executor refuses that form and reports that the SQL producer is not in this slice.

model({
  key: "core/customer",
  schema: Customer,
  run: async (ctx) => {
    const upstream = ctx.input("demo/customers");
    const { rows } = await ctx.query(`select * from ${upstream}`);
    await ctx.emit(rows);
  },
});

Field	Meaning
run	(ctx) => Promise<void>. This is the producer the executor calls.
mode	table (default), view, or incremental.
cursor	Required when mode is incremental. The upstream asset key the delta is over.
connection	Optional. Most models use defaultStorage.

table is a full rebuild. incremental stores a delta. view is its own shape: puppi rollback refuses it.

Destination

destination({
  key: "warehouse/customer",
  from: "core/customer",
  connection: "warehouse",
  write: "replace",
});

Field	Meaning
from	The asset this destination delivers. Added to deps.
connection	Required. Where it delivers.
write	Required. merge, append, or replace. Stored on the asset.

A destination has no run. All three write values are delivered the same way: the destination reads the upstream named by from and emits those rows.

Group

import { group, source } from "@puppi/sdk";

const core = group({ name: "core", visibility: "project" });

source({ key: "core/customer", group: core, run: async () => {} });

visibility on the group is group, project, or public. The default group is project until you declare group({ name: "default", visibility }).

Schedule an asset

schedule: { every: "15m" }
schedule: { cron: "0 9 * * mon", timezone: "America/New_York" }

every is <n>m (1–59), <n>h (1–23), or <n>d (1–31). cron is five fields. Names such as mon and jan are accepted. Sunday is 0 or 7. A bad expression fails puppi build. timezone is checked by the engine at deploy.

Use ctx

ctx is supplied by the executor. The bundle does not include it.

Member	Meaning
input(key)	Physical table name of an upstream asset. The key is a string literal. This is the read that becomes a dependency.
query(sql, params?)	Runs SQL against the asset’s storage. The type argument is your assertion about the select list.
connection(name)	The client for a bound connection. The name is a string literal.
emit(rows)	Buffers rows for this materialization. Type-checked against schema. The schema contract runs after the write.
checkpoint(state)	In-progress state for a retry. Cleared when the producer finishes. The committed watermark lives on the materialization.
checkpointState	The checkpoint from a previous attempt, or null.
cursorFrom	Where an append source or incremental model should resume, or null.
log(message, level?)	A line on the attempt. Levels: debug, info, warn, error.
telemetry	span, event, and gauge for the attempt’s trace.
processPages({ fetch, transform, write })	Fetch, transform, and write one page at a time, with a checkpoint after write resolves.

A custom check receives a smaller context: rows, output, query, and log. It has no input().