setting-up-a-data-warehouse-source

Brings a company's data in from somewhere else — Stripe, Hubspot, a Postgres database, hundreds of others — so it sits next to the product data, and does it without passwords ever going through the chat.

WHEN TO USE

SCOPE

USE FOR

Bring in a table that lives only in the billing system and nowhere else.
Pick which tables sync and how often, so a huge table isn't synced needlessly.
Diagnose why a data connection is failing, like a wrong password or blocked host.
Set up near-real-time updates for Postgres or Stripe instead of periodic syncs.

This is a saved copy of the skill's SKILL.md instructions. It may differ from the upstream file or the version you install.

In this document (9)

Setting up a data warehouse source

Use this skill when the user wants to connect an external data source to PostHog's data warehouse for the first time.

Default to the one-step flow: data-warehouse-source-setup validates credentials, discovers every table, enables them with sensible sync defaults (incremental where possible), and creates the source in a single call — no schemas array to assemble. For credentials, hand the user a secure browser link with data-warehouse-source-connect-link instead of collecting secrets in chat. Only drop to the manual wizard → db-schema → create flow when the user wants to hand-pick which tables sync or set non-default sync types per table.

When to use this skill

The user wants to connect a new source: "connect Stripe", "import my Postgres orders table", "sync Hubspot contacts"
The user isn't sure what source types PostHog supports
The user has credentials but doesn't know how to structure the schemas payload
The user wants guidance on which sync method to pick per table

Available tools

Tool	Purpose
`data-warehouse-source-connect-link`	Preferred for credentials — get a secure browser/OAuth link so the user authenticates without pasting secrets in chat
`data-warehouse-source-setup`	Preferred to create — one call: validate creds, discover tables, apply sync defaults, create the source
`external-data-sources-wizard`	Discover which source types exist and what fields each needs (advanced flow)
`external-data-sources-db-schema`	Validate credentials and list tables with available sync methods per table (advanced flow)
`external-data-sources-create`	Advanced create — requires a `schemas` array built from the db-schema response
`external-data-sources-check-cdc-prerequisites-create`	Postgres CDC pre-flight check (optional, only for Postgres CDC)
`external-data-sources-webhook-info-retrieve`	Check if a source supports webhooks and whether one has been registered
`external-data-sources-create-webhook-create`	Register a webhook with the external service after source creation
`external-data-sources-update-webhook-inputs-create`	Supply the signing secret manually when auto-registration failed
`external-data-sources-list`	After creation, confirm the source is listed and see its initial status
`external-data-schemas-list`	See per-table sync status once the source is created

Pre-flight: credential gotchas that cause most failures

Surface these before collecting credentials — they're the top reasons setup fails on the first try. Validating against them up front avoids burning credential prompts on retries.

The host must be reachable from PostHog's network. localhost, 127.0.0.1, and private/RFC-1918 hosts (10.x, 192.168.x, 172.16–31.x) are rejected — PostHog runs the connection from its own infrastructure, not the user's machine. Serverless/managed Postgres (Neon, Supabase, RDS behind strict rules) often also needs PostHog's egress IPs allowlisted first. If the DB isn't publicly reachable, route to the browser deep-link (data-warehouse-source-connect-link) or an SSH tunnel rather than collecting credentials that can't validate.
Supabase is Postgres — don't collect it twice. Use the Session pooler connection, not the direct host (the direct host is IPv6-only). The pooler host looks like aws-0-<region>.pooler.supabase.com, the username must be postgres.<project-ref>, and the port is 6543 (not 5432). The password is the database password (Settings → Database), which is distinct from the anon/service_role JWT keys and from the Supabase account password. If SUPABASE_URL is in the project env, derive the project ref from db.<ref>.supabase.co to pre-fill these instead of asking the user to guess.
Many SaaS sources need a specific key type or plan — get the right one before the create call fails:
- Stripe — a restricted key (rk_live_…), not the standard secret key (sk_live_…).
- RevenueCat — a v2 secret key (sk_…) with the read scopes enabled.
- Sentry — an internal-integration token, not a DSN and not a personal auth token.
- Convex — requires the Professional plan.
- Twilio — API Key SID + Secret, not the account auth token.
- Mailchimp — the key carries its datacenter suffix (key-usX).
- For send-only services (Resend, Mailgun), the key already in the project env is often restricted; the warehouse import needs a full/read-access key.
Never pass an unresolved secret reference. If a credential field is still a {"secretRef": ...} object, PostHog can't resolve it — the create/db-schema/setup calls reject it with a clear error. Resolve it to the real value first, or collect credentials via data-warehouse-source-connect-link and pass the resulting credential_id.

Recommended: one-step setup

Most setups should use this path — it avoids the most common failures (skipping db-schema, malformed schemas, missing required fields).

Discover the source type and its fields (optional): external-data-sources-wizard lists every source type and the credential fields each needs. Use it to know what to ask the user for; skip it if the source type is obvious.
Collect credentials securely: call data-warehouse-source-connect-link({source_type}). It returns a connect_url to a minimal connect page rendering the source's full connection form — the user authorizes via OAuth or enters credentials there, whichever the source offers (the response's auth_method tells you which to expect). The page validates the details against a live connection and stashes them encrypted in a temporary store — it does NOT create the source. After the user confirms they're done, find the stored credential id via data-warehouse-stored-credentials-list (filter by source_type, newest first; the page also shows the id to the user) and pass {"credential_id": <id>} to setup. Stored credentials are single-use — deleted as soon as setup consumes them — and expire after 24 hours.

Never ask the user to paste raw database passwords, API keys, or OAuth tokens into the chat.
Create in one call: data-warehouse-source-setup({source_type, payload, prefix}). The server validates credentials, discovers all tables, enables them with sync defaults (incremental where a tracking column exists, else append, else full_refresh — never CDC), sets created_via=mcp, and creates the source. The payload carries a credential reference ({"credential_id": ...} or the OAuth integration id key) — or inline credentials for headless automation; no schemas array is needed. On success you get the new source id; call external-data-schemas-list to show the user what was enabled and how each table will sync.

Notes specific to this path:

All discovered tables are enabled. That's intended (incremental defaults keep ongoing cost low), but flag row counts for very large tables and offer the advanced flow if the user wants to sync only a subset.
Webhooks are auto-registered for sources that support them (currently Stripe). Check the webhook key in the setup response: on success, webhook-capable tables sync in real time and webhook-only tables (e.g. Stripe Discount) are enabled too; on failure (e.g. the API key can't create webhooks), tables keep the polling defaults and webhook-only tables stay disabled — relay the webhook.error to the user and offer Step 6 to register manually. If webhook.pending_inputs is non-empty, collect those values and submit via external-data-sources-update-webhook-inputs-create. CDC is never chosen automatically; use the advanced flow + CDC steps for near-real-time Postgres.
Inline credentials in payload still work for headless/automation, but prefer the connect-link handoff above.

Advanced: hand-pick tables (three-step flow)

Use this when the user wants to choose exactly which tables sync or set non-default sync types. Don't try to shortcut to external-data-sources-create — you need the db-schema response to build a valid schemas payload.

         ┌────────────────────┐
         │ 1. wizard          │  What source types exist? What fields does each need?
         └────────┬───────────┘
                  ▼
         ┌────────────────────┐
         │ 2. db-schema       │  Validate creds. List tables + available sync methods per table.
         └────────┬───────────┘
                  ▼
         ┌────────────────────┐
         │ 3. create          │  Send source_type + credentials + schemas[] to actually create.
         └────────────────────┘

Workflow

Step 1 — Discover the source type

Call external-data-sources-wizard with source_type set to the kind(s) you need (comma-separated, e.g. Postgres,Stripe). The unfiltered response describes every supported source and is hundreds of KB — large enough to blow your context budget. Only omit source_type when you genuinely need to enumerate every available type, and when you do, pass fields: ['*.name', '*.caption'] so the response carries just the names instead of every source's config field definitions. The response is a dict keyed by source type. Each entry describes:

name — the canonical source_type string you'll pass to later calls (e.g. "Postgres", "Stripe", "Hubspot").
caption — human-readable description.
fields — the config fields needed (host, port, database, api_key, client_id/secret, ...). Each has name, type (input, password, switch, select, file-upload), and required.
featured, unreleasedSource — use to gauge readiness. Skip sources marked unreleasedSource: true unless the user explicitly asked for a preview.

Match the user's request to a source. If they said "Postgres", look up Postgres. If they said something ambiguous like "database", present the top relevant matches (Postgres, MySQL, MongoDB, BigQuery, Snowflake, Redshift) and let them pick.

For OAuth-based sources (Hubspot, Salesforce, Google Ads), the wizard entry hints at an OAuth flow. These typically need the user to authorize in the PostHog UI rather than pasting credentials — explain this and direct them to the source setup page rather than trying to collect tokens in chat. OAuth is about authentication, not about how data flows; OAuth sources still use polling bulk sync, not webhooks.

Gather the required credentials from the user. Never ask for more fields than the wizard entry says are required — asking for an unnecessary port when the source doesn't need one confuses users.

Step 2 — Validate credentials and discover tables

Call external-data-sources-db-schema with source_type plus all credential fields. This does two things at once:

Validates the credentials against the live source. Returns 400 with a message if anything is wrong (bad host, wrong password, permission denied). Show the error verbatim — it's often actionable ("password authentication failed for user 'x'").
If valid, returns an array of table entries. Each entry:

{
  "table": "orders",
  "should_sync": false,
  "rows": 1_250_000,
  "incremental_available": true,   # can do sync_type=incremental
  "append_available": true,        # can do sync_type=append
  "cdc_available": true,           # can do sync_type=cdc  (null = not enabled for team)
  "supports_webhooks": false,      # can do sync_type=webhook for real-time push
  "incremental_fields": [          # candidates: usually updated_at, created_at, id
    {"field": "updated_at", "type": "datetime", "label": "updated_at", ...},
    {"field": "created_at", "type": "datetime", ...},
    {"field": "id", "type": "integer", ...}
  ],
  "detected_primary_keys": ["id"],
  "available_columns": [{"field": "id", "type": "integer", "nullable": false}, ...],
  "description": "..."
}

Present this to the user. Don't dump the raw JSON — summarize: which tables were found, row counts, and the default sync method recommendation per table (see sync-type decision guide).

Step 3 — Confirm per-table sync configuration

For each table the user wants to sync, pick a sync_type. See the sync-type decision guide for detailed rules, but the short version is:

Small / dimension tables (<50k rows, no natural ordering column): full_refresh — simple and always correct.
Large tables with an updated_at / modified_at: incremental — much cheaper per sync.
Append-only immutable tables (logs, events): append if available — preserves history.
Postgres with CDC enabled and you need near-real-time: cdc — requires primary keys and Postgres prerequisites.
Sources that support webhooks (currently Stripe): for near-real-time ingestion set sync_type: "webhook" on the tables where supports_webhooks: true, then register the webhook as a post-create step (see step 6 below). Tables that don't support webhooks on the same source still need a bulk sync_type.

For each schema that will use incremental/append/cdc, you also need:

incremental_field — which column to track for high-water-mark ordering. Pick from the incremental_fields list returned by db-schema. Prefer updated_at over created_at (updated_at catches late-arriving updates; created_at misses them). For integer-only tables, use the monotonically increasing primary key.
incremental_field_type — must match the chosen field's type (datetime, timestamp, date, integer, numeric, objectid).
primary_key_columns — required for CDC. Use detected_primary_keys from db-schema.

Step 4 — Pick a good prefix

The source's prefix is prepended to table names in HogQL. Tables end up as {prefix}_{table_name}.

Default to the source type lowercased if there's only one source of that type: stripe, postgres.
If the user already has a Postgres source, pick something distinguishing: postgres_prod, postgres_analytics.
Use lowercase, underscore-separated. The prefix becomes part of every HogQL query the user writes.

Confirm the prefix with the user before creating — changing it later is possible but renames every table.

Step 5 — Create the source

Call external-data-sources-create with:

{
  "source_type": "Postgres",
  "prefix": "postgres_prod",
  "payload": {
    "host": "...",
    "port": "5432",
    "dbname": "...",
    "user": "...",
    "password": "...",
    "schema": "public",
    "schemas": [
      {
        "name": "orders",
        "should_sync": true,
        "sync_type": "incremental",
        "incremental_field": "updated_at",
        "incremental_field_type": "datetime",
        "primary_key_columns": ["id"]
      },
      {
        "name": "users",
        "should_sync": true,
        "sync_type": "full_refresh"
      },
      {
        "name": "audit_log",
        "should_sync": false
      }
    ]
  }
}

Rules for the schemas array:

Every table returned by db-schema should be included, even ones the user doesn't want (set should_sync: false). Tables the user didn't mention default to should_sync: false.
sync_type is required only when should_sync: true.
incremental_field / incremental_field_type must be present when sync_type is incremental or append.
primary_key_columns must be present when sync_type is cdc.

On success you'll get back a source with a new id. The first sync is triggered automatically.

Step 6 — Register a webhook (only when any schema is `sync_type: "webhook"`)

Webhook-type schemas don't start receiving data just by existing — the external service needs to know where to POST events, and PostHog needs to know how to verify them. This is a second call after source creation, not part of the external-data-sources-create payload. Do this before telling the user the setup is complete, otherwise they hear "syncs are running" while the push channel is still unregistered.

Only needed when at least one schema on the source has sync_type: "webhook" and should_sync: true. Currently only Stripe implements this flow; for everything else skip this step.

Before calling create-webhook, check external-data-sources-webhook-info-retrieve({id}). If it already returns exists: true, do NOT call create-webhook again — each successful call registers a new external endpoint and would result in duplicate deliveries.

Call external-data-sources-create-webhook-create({id}). PostHog:
- creates the HogFunction that will receive webhook POSTs,
- builds a schema_mapping from external event types to PostHog schema ids,
- calls the source's API (e.g. Stripe) to register the webhook URL and subscribe to the relevant events,
- on Stripe, auto-captures the signing_secret and stores it securely.
Returns {success, webhook_url, error}. On success report the webhook_url to the user for their records — but they don't need to paste it anywhere; registration is already done.
If success: false with a permissions error like "API key doesn't have permission to create webhooks":
- The HogFunction is still created, just disabled.
- Ask the user to create the webhook manually in the source's dashboard using the returned webhook_url.
- Have them copy the signing secret from the source's webhook settings.
- Call external-data-sources-update-webhook-inputs-create({id}, {inputs: {signing_secret: "whsec_..."}}) to store it. The HogFunction picks it up and verifies incoming payloads.
Verify with external-data-sources-webhook-info-retrieve({id}). A healthy webhook has exists: true, external_status.status: "enabled", and no error.

Webhooks are supplementary to bulk sync. The first load of a webhook-enabled schema is still done via polling (initial_sync_complete flips to true when done); after that, the webhook becomes the primary ingestion path. A webhook schema will still have a sync_frequency that schedules a periodic bulk refresh as a safety net. This is expected — not something to "fix".

Step 7 — Confirm and explain what happens next

After creation (and, for webhook schemas, after Step 6):

Call external-data-schemas-list to show the user the initial state.
Explain: every enabled schema enters Running, then moves to Completed when the first sync finishes. First syncs can take anywhere from seconds to hours depending on row count — a multi-million-row table is fine, just slow.
Tell them how to query: SELECT * FROM {prefix}_{table_name} LIMIT 10 in HogQL.
Offer to check back in a few minutes to confirm the initial syncs succeeded.

CDC setup for Postgres (optional, when requested)

If the user wants near-real-time replication from Postgres:

Before calling db-schema, run external-data-sources-check-cdc-prerequisites-create with their Postgres creds. It returns {valid, errors[]} listing anything missing (wal_level, replication slot, publication, permissions).
If valid: false, present the errors and ask the user to fix on the Postgres side. Don't try to create a CDC source that will immediately fail.
Once prerequisites pass, proceed to db-schema and create. Set sync_type: "cdc" on the tables that need it, and include primary_key_columns for each (CDC requires them).

Important notes

Always validate creds with db-schema before create. The create endpoint will accept invalid creds and then fail asynchronously — the source appears in the list with status Error and no tables. Skipping the validation step just pushes the failure into the background.
Present the table list before creating. Large databases may have hundreds of tables. Don't auto-select them all — row counts and relevance matter for billing. Let the user opt in explicitly.
Don't invent schemas. Every entry in the schemas array must correspond to a real table from the db-schema response. You can't "also add an orders table" unless db-schema found one.
A new source syncs to the PostHog warehouse. That is the default and it needs no setup: a source nobody configured destinations for writes here and nowhere else. Some projects can also send a source's tables to their own database (external-data-sources-destinations-retrieve shows where a source writes, when the feature is on for the project). Setting that up is a separate step and never something to do as part of creating a source — adding a destination to a source that has already synced leaves it holding none of the history until the tables are resynced, and a resync bills every destination the source writes to.
Prefix is load-bearing. It's part of every HogQL query the user will ever write against these tables. Pick something short, descriptive, and not already taken.
Prefer the secure connect-link for any credentials. Use data-warehouse-source-connect-link so the user authenticates in their browser — the connect page renders the source's full connection form (OAuth and credential options alike) and stores the result without creating the source. Don't collect OAuth tokens or database passwords in chat; pass the credential_id reference to setup — source creation always happens through setup, not the UI. (An already-connected OAuth integration can also be passed directly via its id key, e.g. {"hubspot_integration_id": 123}.)
Webhooks are a separate step after create. Setting sync_type: "webhook" on a schema doesn't register the webhook — the create-webhook call does. Always follow create → create-webhook → webhook-info for webhook-type schemas, and never leave a webhook schema dangling without registration (it just won't receive events).
Webhook support is source-specific and sparse. Currently only Stripe implements WebhookSource. Don't promise webhooks for Hubspot, Salesforce, or Postgres — they'll use polling sync.
Row counts drive billing. Warehouse syncing is metered by rows synced. A chatty 500M-row events table synced hourly is very different from a 10k-row dimension table synced daily. Flag large tables and offer longer sync frequencies (sync_frequency: "24hour") as the default.

This skill ships inside the PostHog AI Plugin. Listed package contents include 3 slash commands, 1 agent, hooks and an MCP server. Installed components and skill versions may vary by AI tool.

Install just this skill

Run this outside your AI tool, using Node.js 22.20.0 or newer. Skills CLI 1.7.2 skips skill selection; prompts cancel installation without an interactive terminal. If choosing Project when asked about installation scope, open your project's folder before running this.

Install checklist

Check selected tools if a picker appears; Claude Code may be unchecked under Additional agents. ↑/↓ moves, Space toggles selections, Enter confirms.
Without a tool picker, check Installing to: for your intended tool. Stop if it names a different tool.
If asked, choose Symlink (Recommended) for shared files or Copy to all agents for separate copies; review the summary before confirming installation.
Install the find-skills skill? defaults to Yes, adding another skill for all projects. Choose No if you don't want that extra installation.

Skills CLI 1.7.2 sends install data to skills.sh by default, including the source, skill names, target tools and file details; GitHub sources are reported only when GitHub confirms the repository is public, while other remote sources can also be reported. Confirmed-public GitHub sources also trigger a security-audit request. Before installing, set the environment variable DO_NOT_TRACK=1 or DISABLE_TELEMETRY=1 to disable both.

npx skills add posthog/ai-plugin --skill "setting-up-a-data-warehouse-source" --full-depth

After installing for Claude Code, open it in the installation project if you chose Project; Global makes the skill available across projects on this computer. Once Claude Code loads the skill, its description helps Claude decide when to use it.

If installation created a top-level skills folder that did not exist when your Claude Code session started, run /reload-skills in that session. Run it again after later changes in that folder.

MCP server required

The skill's steps call for tools from the PostHog MCP server. Installing just this skill doesn't add that connection; set it up in your AI tool if it isn't there yet: PostHog MCP setup. Installing the whole plugin instead includes the connection settings.

Requirements for the PostHog MCP server

Connecting is free; sign in with your PostHog account.
Some tools use AI internally and may incur PostHog AI charges. Those tools require AI data processing to be enabled in your organization settings.

Install the whole plugin

This route installs the posthog plugin package selected for your AI tool.

Claude Code

Add PostHog's marketplace, then install the plugin

Run both lines in a terminal. They install PostHog's posthog plugin: this skill, PostHog's other skills, and a connection to PostHog's hosted MCP server.

claude plugin marketplace add PostHog/ai-plugin

02
```
claude plugin install posthog@posthog
```

Installation and loading

The plugin installs for you in every project and loads the next time you start Claude Code, or after /reload-plugins in an open session.
To connect your PostHog account, start claude, type /mcp, select plugin:posthog:posthog and follow the browser prompts.

Good to know

The plugin includes PostHog's session-capture hook; PostHog's README says it sends sessions to LLM Analytics only when POSTHOG_LLMA_CC_ENABLED=true and POSTHOG_API_KEY are both set.
If you use the separate posthog-telemetry plugin, PostHog says to keep only one of the two, because both register the same hook.
On a self-hosted PostHog, this route's MCP connection won't reach your instance: the plugin's config is fixed to https://mcp.posthog.com/mcp, which routes to PostHog's US or EU region. POSTHOG_MCP_URL works only for the Gemini CLI extension.

What's inside

Files included with this skill.

View on GitHub

Same repository

adding-warehouse-person-properties
Sync columns from a synced data warehouse table onto PostHog person or group properties, so warehouse data becomes usable anywhere person and group properties already work: feature flag targeting, cohorts, insight filters and breakdowns, surveys, session replay filters, workflows, and the person profile. Use when the user wants to "add a person property from my warehouse", "enrich people with Stripe/Postgres/Salesforce data", "put ARR or plan tier on my persons", "target a feature flag by a warehouse column", "sync warehouse columns onto groups or organizations", or wants to inspect, backfill, disable, or debug an existing warehouse-backed person or group property.Read full description
analyzing-expensive-users
Analyze the most expensive users in AI observability and explain why they cost so much. Use when the user asks about top spenders, expensive users, per-user LLM cost, user-level cost drivers, or patterns behind high AI observability spend.Read full description
analyzing-experiment-session-replays
Analyze session replay patterns across experiment variants to understand user behavior differences. Use when the user wants to see how users interact with different experiment variants, identify usability issues, compare behavior patterns between control and test groups, or get qualitative insights to complement quantitative experiment results. Also covers pairing the observed behavior with a linked survey when the user wants qualitative feedback beyond what recordings show.Read full description
assessing-heatmaps
Assesses what a page's heatmap is telling you and recommends concrete changes. Pulls click / rageclick / scroll-depth data for a URL, names the hot elements by cross-referencing autocapture events on the same page, and can create a saved heatmap the user opens in PostHog, then summarizes the behavior and proposes improvements. TRIGGER when: user asks what a heatmap shows, why people aren't clicking something, where users rage-click, how far they scroll, what to change on a page based on heatmap/click data, or to 'analyze/assess/review the heatmap' for a URL. DO NOT TRIGGER when: the user only wants to create a saved heatmap screenshot with no analysis (use heatmaps-saved-create directly), or is asking about session replay in general (use investigating-replay).Read full description
auditing-endpoints
Audit every endpoint in a PostHog project for staleness, failed materialisations, and unused materialised versions. Use when the user asks "what endpoints can I clean up?", "are any of my endpoints broken?", "which materialised versions are still being called?", or wants a one-shot cleanup pass over the Endpoints product. Produces a prioritised report grouped by issue type, with recommended actions but does not modify anything without explicit confirmation.Read full description
auditing-experiments-flags
Audit PostHog experiments and feature flags for configuration issues, staleness, and best-practice violations. Read when the user asks to audit, health-check, or review experiments or feature flags, check flag hygiene, or verify experiment setup.Read full description