Skip to content
enclava-labsPublic

About

Run stateful confidential containers on top of kubernetes

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Enclava CAP

Enclava CAP is the open-source control plane core for running OCI images as confidential workloads on Kubernetes. It targets Kata confidential containers with AMD SEV-SNP and combines a user CLI, an API service, Kubernetes manifest generation, and an in-TEE init sidecar.

What Is Included

Path Purpose
crates/enclava-cli enclava CLI for login, app setup, deploys, config, ownership, and recovery
crates/enclava-api Axum API service for auth, apps, deployments, domains, workload artifacts, and deploy orchestration
crates/enclava-engine Kubernetes manifest rendering, apply, watch, cleanup, and validation logic
crates/enclava-init In-TEE sidecar that verifies policy, unlocks storage, prepares TLS state, and signals workload readiness
crates/enclava-wait-exec Small wrapper that blocks app and ingress processes until enclava-init is ready
crates/enclava-common Shared descriptor, image-reference, validation, encoding, and crypto helpers
crates/enclava-snp SNP-attested TLS channel to a TEE (evidence, AMD KDS, collateral policy), shared with the hosted PaaS
deploy/api Minimal Kubernetes overlay for the CAP API service
scripts and runbooks Release manifest rendering, smoke-test helpers, and operator runbooks

How It Works

CAP deploys immutable OCI images by digest and validates the deployment before it reaches Kubernetes:

  • the CLI creates or reads local app config and signs a deployment descriptor;
  • CAP validates the descriptor, image digest, signer identity, org keyring, and generated policy artifacts;
  • the engine renders the Kubernetes resources for the confidential workload;
  • enclava-init runs inside the guest, verifies the runtime trust chain, opens encrypted app/TLS volumes, and releases runtime state only after verification.

This repository is useful for developing the CAP runtime, API contracts, CLI flows, and confidential-workload deployment machinery. A real deployment still requires the surrounding platform services described in DEPLOYMENT.md.

Install

macOS / Linux (musl-static binaries, checksum + cosign verified):

curl -fsSL https://raw.githubusercontent.com/enclava-labs/cap/main/scripts/install.sh | bash

Windows (PowerShell):

irm https://raw.githubusercontent.com/enclava-labs/cap/main/scripts/install.ps1 | iex

Installs the latest release to ~/.enclava/bin (%USERPROFILE%\.enclava\bin on Windows) and adds it to your PATH. Set ENCLAVA_VERSION=vX.Y.Z for a specific release, ENCLAVA_INSTALL_DIR to change the destination. Other platforms: cargo install --locked --git https://github.com/enclava-labs/cap enclava-cli.

CLI Flow

After authenticating against a CAP API, the normal app flow is:

enclava login
enclava init
enclava create --signer-subject <cosign-subject>
enclava deploy --image <registry>/<image>@sha256:<digest>
enclava status
enclava key backup --out enclava-recovery.json

New apps and deployments are password-only. Auto-unlock creation, enable requests and enclava auto-unlock enable fail with auto_unlock_disabled; the guest never fetches its owner seed automatically. After guest replacement, use enclava unlock --app NAME --password-file PATH with the existing storage password.

For a legacy auto-mode app, run enclava auto-unlock disable --app NAME --image REPO@sha256:DIGEST --password-file PATH from its original enclava.toml workspace before redeploying. This preserves the owner seed and volumes, but cannot revoke a previously copied auto-sealed seed.

For encrypted workload logs, use the current CLI and the tenant's log private key:

enclava logs --follow --log-private-key-file /path/to/log-key.x25519

Approve apps:logs with enclava login --approve-logs if the session lacks it. Successful follow streams have no total deadline; initial headers, error bodies and ordinary API requests remain bounded. Blank-line keepalives preserve idle streams without printing fake log messages. Each encrypted record is size-bounded before decoding.

For an encrypted-state-backed spool, restart recovery removes an incomplete EOF from the writer's bounded spool before appending new frames, preserving complete retained records. Abrupt guest loss can still lose unflushed newest logs. Already malformed, newline-terminated records are not silently discarded; preserve diagnostic evidence rather than clearing the spool or its PVC.

Images may be digest-pinned or tagged (enclava deploy --image <registry>/<image>:<tag>). The CLI resolves tags to a content digest before signing; CAP independently resolves the image and checks that the digests agree. The signer subject should match the identity that signs the image, such as a GitHub Actions keyless cosign subject.

CLI tag resolution uses direct HTTPS requests with TLS verification and no redirects or system HTTP proxies. Registry and permitted bearer-auth endpoints cannot resolve to loopback, private, link-local, metadata, or CGNAT addresses, including IPv4-compatible and IPv4-mapped IPv6 addresses. CLUSTER_POD_CIDR and CLUSTER_SERVICE_CIDR add comma-separated IPv4/IPv6 CIDRs to the blocklist. Literal IP destinations are checked against the same ranges. Digest-pinned references skip the CLI registry lookup, including when registry access requires an HTTP proxy. Bearer-auth failures do not print realm URLs or query credentials.

The CLI also contains hosted-template commands used by Enclava-hosted API mode:

enclava template list
enclava template deploy --name shell --ssh-public-key-file ~/.ssh/id_ed25519.pub
enclava template ssh-command --name shell --wait

Those commands are convenience flows for hosted templates. They are not required for the core manual deploy path.

An interrupted first template deployment can be retried with the same command. For CAP's single-replica automatic rollouts, applying changed init data replaces only an unready Pod owned by a ledger-recorded StatefulSet and still using the old init data. Deletion checks the live StatefulSet generation and the Pod UID and resource version, retains PVCs, and uses normal graceful termination. The CLI requires the signed deployment's attested launch identity both while waiting for bootstrap and on the client that submits the ownership claim.

If a deploy or template rollout fails terminally at startup (for example a guest start error), the CLI reports the typed failure and exits promptly instead of timing out; run enclava status --app <name> for the recorded detail.

Apps awaiting required confidential teardown retain their public routes even if every deployment has failed or a failed delete restored the app's prior status. Routes are removed after teardown completes; deleting a failed app does not bypass its storage or ownership checks.

If an app delete errors after its namespace deletion begins, the app's namespace mutation fence stays armed instead of risking name reuse. The periodic fence reconciler releases it automatically once Kubernetes reports the namespace gone — no manual lease edits — while a live or still-Terminating namespace keeps it fenced. Other stranded kubernetes_namespace fences remain an operator path; see runbooks/kubernetes-mutation-fence-recovery.md.

Local Development

Requirements:

  • Rust 1.88 or newer;
  • Docker Compose for the local API/PostgreSQL stack;
  • pkg-config and OpenSSL development headers (libssl-dev), required even for a standalone cargo install enclava-cli;
  • libcryptsetup development headers, clang, and libclang for the full workspace, including enclava-init.

On Debian or Ubuntu:

sudo apt-get install -y pkg-config libssl-dev libcryptsetup-dev clang libclang-dev

Start the development API:

docker compose up --build
curl http://localhost:3000/health

The compose stack is for development only. It uses local PostgreSQL and ephemeral API signing/session keys.

Common checks:

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace -- --test-threads=1

The workspace test run already includes every package's doctests, so a separate cargo test --doc is redundant. The suite must stay serialized: API fixtures share global PostgreSQL provider fences, and enclava-cli, enclava-init, and enclava-wait-exec tests mutate process-global environment or init surface paths.

Do not use --all-features as the default workspace test command. Some feature combinations intentionally fail, such as prod-strict with debug/test-only features.

When local native dependencies are unavailable, run targeted checks:

cargo test -p enclava-api --lib -- --test-threads=1
cargo test -p enclava-cli
cargo test -p enclava-engine
cargo test --workspace --exclude enclava-init -- --test-threads=1

Database-backed API library tests require a dedicated local PostgreSQL superuser, as in CI: isolated fixtures create/drop databases, and schema-guard tests mutate system catalogs. Run the full suite serially because remaining fixtures share global provider fences. Never point these tests at a persistent environment.

API integration tests need PostgreSQL:

DATABASE_URL=postgresql://enclava:enclava@localhost:5432/enclava \
  cargo test -p enclava-api --test integration_test

Deployment

The checked-in deploy/api overlay is a minimal starting point for the CAP API. Production deployments must provide durable key material, PostgreSQL, digest-pinned platform images, Trustee/KBS access, policy-signing service access, DNS credentials when CAP manages tenant DNS, and tightly scoped Kubernetes permissions.

See DEPLOYMENT.md for the runtime environment and production configuration checklist.

Security

Security-relevant behavior is built into both the API and runtime path:

  • release builds reject debug bypass flags;
  • platform sidecars and workload images are expected to be digest-pinned;
  • workload images are verified against configured signer identity;
  • private tenant-image pulls require authentication inside the TEE; host registry credentials alone do not authorize them. Never release a platform registry token to a tenant guest;
  • deployment descriptors, org keyrings, generated policies, and signed policy artifacts are bound together before deploy;
  • enclava-init verifies the in-TEE trust chain before releasing app secrets or mounted state.

See SECURITY_REVIEW.md for the current security snapshot.

More Documentation

License

MIT. See LICENSE.

About

Run stateful confidential containers on top of kubernetes

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages