Skip to main content

Architecture overview

Pylon is CPython 3.14 compiled to wasm32-wasip2 as a componentized wasm binary, plus a small ecosystem of capability components ("caps") that supply the systems surface CPython would normally get from libc + a real OS — TLS via OpenSSL, SQL via SQLite / DuckDB, compression, hashing, crypto primitives, IDs, and more.

This document is the "how it fits together" reference for contributors and reviewers. If you are looking for a step-by-step guide to add or fix a cap, see Add a cap. If you want the elevator pitch, see What is pylon?.

The layered picture​

    ┌─────────────────────────────────────────────────────────────────┐
    │  User program                                                    │
    │  (Python source, executed by CPython under wasmtime)             │
    └──────────────────────────────┬──────────────────────────────────┘
                                   │
    ┌──────────────────────────────▼──────────────────────────────────┐
    │  python.wasm  (CPython 3.14 built-ins + statically linked C exts)│
    │                                                                  │
    │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌───────┐│
    │  │ _ssl_capa-   │  │ _duckdb_cap  │  │ _crc32c_dyn- │  │  ...  ││
    │  │ bility (C)   │  │      (C)     │  │ link_cap (C) │  │       ││
    │  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  └───────┘│
    │         │ WIT imports declared by python.wasm                    │
    └─────────┼──────────────────┼──────────────────┼─────────────────┘
              │                  │                  │
      wac plug composition (compose-time)   compose:dynlink/linker (runtime)
              │                  │                  │
    ┌─────────▼──────────────────▼──────────────────▼─────────────────┐
    │  pylon.wasm  (single composed component, ~115 MB)                │
    │                                                                  │
    │   openssl-comp │ ducklink-core │ zstd-wasm │ ... 70+ cap plugs   │
    │                                                                  │
    │   + dynlink providers resolved at runtime by composectl          │
    └──────────────────────────────┬──────────────────────────────────┘
                                   │
    ┌──────────────────────────────▼──────────────────────────────────┐
    │  composectl                                                      │
    │  (content-addressed blob store, trust store, plan-JSON loader,   │
    │   embedded in packaging/launcher-rs/pylon-launch)                │
    └──────────────────────────────┬──────────────────────────────────┘
                                   │
    ┌──────────────────────────────▼──────────────────────────────────┐
    │  wasmtime (WASI Preview 2 host)                                  │
    └─────────────────────────────────────────────────────────────────┘

Left-to-right: python.wasm (produced by CPython's own Tools/wasm/wasi build) contains CPython plus every cap C extension. Each C extension declares WIT imports for its cap interface (e.g. openssl:component/tls). Caps are dynamically loaded at runtime by composectl — the launcher hands pylon.wasm and a plan JSON to composectl, which fetches cap components from its content-addressed blob store on first use and resolves each dynlink import. Swapping a provider is a plan edit, not a rebuild.

For convenience, the stock pylon.wasm distribution embeds a set of hot-path caps via wac plug at compose-time (openssl, sqlite, ducklink, zstd, arrow — anything where invoke overhead would dominate or that holds C handles across calls). This is a packaging choice, not architecture: you can build a custom pylon.wasm with a different embed set — scripts/compose-python-component.sh takes WITH_*=0 flags per cap. See The three linkage modes.

The four-file lifecycle​

     (1)                    (2)                    (3)                    (4)
  ┌──────────┐  wac plug  ┌────────────┐         ┌──────────────┐   pylon-launch
  │python.wasm│ ────────▶ │ pylon.wasm │ ──────▶ │  plan JSON   │ ─────────────▶ instantiate
  │  ~ 50 MB │            │ ~ 115 MB   │         │              │
  └──────────┘            └────────────┘         └──────────────┘
      ▲                                                 ▲
      │                                                 │
   CPython                                     build-pylon-dynlink-plan.py
   + cap C exts                                (also stages .compose/blobs/)
  1. python.wasm — CPython 3.14 built via Tools/wasm/wasi build, with every cap C ext statically linked. Declares WIT imports for every cap interface it consumes. Path: deps/cpython-3.14/cross-build/wasm32-wasip2/python.wasm.
  2. pylon.wasm — output of scripts/compose-python-component.sh. wac plugs the embed-set of caps (hot-path convenience) into python.wasm. Everything else stays a plan-JSON reference and is fetched by composectl at runtime. Path: build/pylon.wasm (renamed from python.composed.wasm in commit feee530e; older docs may still say the old name). The embed set is controlled by WITH_*=0 env flags in the compose script — build a custom pylon.wasm with any subset embedded.
  3. Plan JSON — plans/python-pylon-dynlink.json. Written by scripts/build-pylon-dynlink-plan.py. Declares component digests, bindings, and dynlink capabilities. Only referenced components need to be present as blobs in .compose/blobs/; extras are ignored.
  4. pylon-launch — Rust binary at packaging/launcher-rs/target/release/pylon-launch. Reads the plan, trusts each digest, resolves runtime bindings, hands the composed graph to wasmtime, and calls run.run().

The user-facing launcher (packaging/bin/pylon) is a thin shell wrapper that resolves the bundled pylon.wasm relative to its own location and delegates. From the tarball / brew / pip channel: pylon → pylon-launch → composectl → wasmtime.

Where names come from​

Naming is load-bearing — the same word tends to appear in five places across the two-sided artifact, and the small conventions below prevent "which name goes where?" from being a recurring paper cut.

WhatConventionExample
C extension module name_<name>_cap (underscore-prefixed)_duckdb_cap, _ssl_capability
User-facing Python name<name> (no underscore, no _cap)duckdb, ssl
This repo's cap directorycpython-ext/_<name>_cap/cpython-ext/_duckdb_capability/
Sibling repo~/git/<name>-wasm/~/git/duckdb-wasm/
WIT package name<name> (no -wasm)duckdb:component/database
Composed cap artifact<name>.component.wasm or similarducklink_core.wasm

Two adjacent-but-distinct concerns show up often enough to name:

  • Linkage mode is invisible above the facade. Do not export names like duckdb_dynlink or crc32c_dynlink from user code. If a cap moves from compose-static to runtime-dynlink (or vice versa), the Python-visible name should not change. Historic _dynlink module names still exist but were closed off from user-facing imports in commits 28d2c4f3 and fb396eb3. The private underscored C ext stays available for probes.
  • _capability vs _cap. Older caps use the long form (_ssl_capability, _sqlite_capability, _duckdb_capability); newer ones use the short form (_crc32c_cap, _id_mux_cap). Both are fine — do not rename an existing cap for consistency's sake. New caps should use _cap unless there is a specific reason not to.

Next​