RISC-V Multi-Tenancy in Varnish

A Varnish Module (VMOD) that enables ultra-fast multi-tenancy using ephemeral RISC-V virtual machines. Each tenant can execute custom logic to configure HTTP requests and responses with VCL-like capabilities, with VM creation/destruction overhead of less than 1 microsecond.

Key Features

• Blazing Fast: ~1μs overhead per request compared to native VCL
• Sandboxed Execution: Isolated RISC-V VMs for each tenant
• Developer-Friendly: Write tenant logic in C++ or Rust with a VCL-like API
• Ephemeral VMs: VMs are created and destroyed per-request for zero state leakage
• VCL Integration: Seamlessly integrates with existing Varnish configurations
• JSON/XML Support: Built-in JSON and XML parsing, validation

Use Cases

• Multi-tenant CDNs: Give each customer programmable edge logic
• Request customization: Per-tenant header manipulation, URL rewriting, and routing
• Dynamic backends: Programmatically select backends based on custom logic
• Edge computing: Run lightweight computations at the edge with microsecond latency

Quick Start

Check out the demo VCL and example tenant program to see it in action.

Example: Tenant Program

Here's a minimal example of what tenant code looks like:

#include <api.h>
namespace varnish = api;

static void on_recv(varnish::Request req) {
    // Manipulate request headers
    req.append("X-Hello: url=" + req.url());

    // Return a JSON response
    forge(varnish::Cached, [] (auto bereq, auto beresp) {
        nlohmann::json json;
        json["message"] = "Hello from RISC-V!";
        return varnish::response{200, "application/json", json.dump()};
    });
}

int main() {
    varnish::wait_for_requests(on_recv);
}

Example: VCL Configuration

sub vcl_init {
    riscv.embed_tenants("""{
        "customer1.com": {
            "filename": "/path/to/tenant_program"
        }
    }""");
    # Loads the programs. Required, or no tenant is loaded.
    riscv.finalize_tenants();
}

sub vcl_recv {
    riscv.fork("customer1.com");
    riscv.run();
}

Note: If you have a RISC-V compiler installed, you can use source files directly as the filename for a tenant, and it will be compiled, cached and loaded automatically: "filename": "/my/source.cpp". The compiler will be detected from the compiler detection script.

The same is true for Rust projects. Point "filename" to your Rust project and make sure you have a 64-bit RISC-V compiler installed. It will build and load the rust program automatically.

Config files

A tenant's config_files setting maps logical names to files that the program can read during initialization:

"customer1.com": {
    "filename": "/path/to/tenant_program",
    "config_files": {
        "settings": "/etc/varnish/customer1/settings.json",
        "denylist": "/etc/varnish/customer1/denylist.txt"
    }
}

In main(), api::config_file("settings") returns the file's contents, or std::nullopt if no file is configured under that name. This API is available only during initialization.

The VMOD watches the files' parent directories with inotify. When a file changes, it reloads the tenant as it would for a live update: a new instance runs main() with the updated contents, then replaces the running instance. Requests already in progress finish on the old instance. If a file cannot be read or main() fails, the tenant keeps its current program and logs the error.

Files can be updated in place or replaced by renaming a new file over the existing one. A VCL reload reuses a tenant if its program and config files are unchanged.

Rust example program

Have a look at the Rust main.rs. You will need a full Linux-compatible RISC-V cross compiler, which is usually provided by your distro package repositories. For example, on Ubuntu the package is called gcc-14-riscv64-linux-gnu and the correct linker value for the Rust cargo config is riscv64-linux-gnu-gcc-14.

In order to be able to boot the Rust program you should build the VMOD with 64-bit and C-extension support:

./build.sh --64 --C

Once this is done, point your VCL config to the path containing your project, and it will build and run it:

Info: Child (...) said >>> [rusty.com] RISC-V Rust example program started

An example program in Rust:

fn on_client_request(_a: u32, _b: u32) {
	add_req_header("X-Hello", &get_url());

	forge(CachingDecision::Uncached, |_a: u32, _b: u32| {
		BackendResponse {
			status: 200,
			content_type: "text/plain",
			body: "<p>hello from Rust 2</p>".as_bytes().to_vec(),
		}
	});
}

pub fn main() {
	wait_for_requests(on_client_request);
}

JWT example programs

HS256 JSON Web Tokens are verified and issued by jwt.cpp and rust-jwt, using the native base64 (api::base64_encode/decode, varnish::base64_encode/decode) and SHA-256 system calls. Both register functions that VCL calls:

set req.http.X-JWT-Sub = riscv.call("jwt_verify", req.http.Authorization);
if (req.http.X-JWT-Sub == "") {
	return (synth(401));
}

Selecting backends

A tenant can specify which VCL backends its program may select. During VCL initialization, finalize_tenants() resolves each configured backend by name. If any backend is missing, vcl_init fails:

"api.example.com": {
	"filename": "/path/to/program",
	"group": "test",
	"backends": ["origin", "images"]
}

The program can then select a backend by name or by its index in the configured list:

static void on_client_request(Request req)
{
	if (req.url().starts_with("/img/"))
		req.set_backend("images");
}

Backend indices remain stable for the lifetime of the tenant. A program can use api::find_backend() during initialization to look up an index, then pass that index to req.set_backend(int) when handling requests. Alternatively, req.set_backend("images") looks up and selects the backend in a single call.

Each tenant supports up to 32 backends. Configured backends occupy the first slots; backends added for individual requests with riscv.add_backend() use the remaining slots. If all 32 slots are occupied by configured backends, riscv.add_backend() returns -1.

Name resolution supports backends declared in VCL with backend name { ... }. On Varnish Enterprise, it does not support directors created by VMODs such as directors, goto, and udo.

Outgoing HTTP requests

On Varnish Enterprise, a tenant can make HTTP and HTTPS GET and POST requests with the ECALL_FETCH system call. They are run by libcurv, the HTTP engine that varnishd links and starts (the one behind vmod_http), so several requests can be in flight at once while the program carries on. The program then waits for any or all of them. A tenant may only reach the URLs its fetch allow list permits; without one, every request is refused:

"api.example.com": {
	"filename": "/path/to/program",
	"group": "test",
	"fetch": {
		"allow": ["https://api.example.com/v1/", "http://10.0.0.5:8080/"],
		"max_requests": 8,
		"timeout_ms": 5000,
		"connect_timeout_ms": 2000,
		"max_request_body": 1048576,
		"max_response_body": 16777216,
		"max_redirects": 0,
		"allow_insecure": false
	}
}

Each allow list entry specifies a scheme, host, port (using the scheme's default if omitted), and path prefix. Both the entries and the program's request URLs are parsed by curl. End the prefix with a trailing slash unless it should match any path beginning with that prefix. URLs are rejected if a server could interpret their paths as traversing above the allowed prefix, including paths containing encoded parent-directory segments (%2e%2e), ..;, or backslashes.

Programs cannot set headers that select the virtual host or control message framing, such as Host, Content-Length, Transfer-Encoding, and Connection. max_requests applies per task, timeout_ms limits the duration of each request including the redirects it follows, and allow_insecure controls whether requests may disable TLS certificate verification. The other values shown are the defaults.

Redirects are disabled by default. Set max_redirects to a positive value to enable them and limit the number of redirects followed per request. The maximum supported value is 20. Each redirect target must be permitted by the allow list and uses the TLS files configured in its matching entry. For 301, 302, and 303 redirects, POST requests are converted to GET requests, with the body and Content-Type header removed. For 307 and 308 redirects, the method, body, and headers are preserved. The Authorization, Proxy-Authorization, and Cookie headers are removed when a redirect changes the scheme, host, or port. If a redirect cannot be followed because the redirect limit has been reached, the target is not allowed, or timeout_ms has elapsed, the redirect response is returned and the reason is logged as a VCL_Error. Redirects do not count against max_requests.

An HTTPS entry can also be an object specifying TLS files for its origin. Use ca_file to verify servers with a private CA or a self-signed certificate, or client_cert and client_key to configure mutual TLS:

"allow": [
	"https://api.example.com/v1/",
	{
		"url": "https://10.0.0.5:8443/internal/",
		"ca_file": "/etc/varnish/tenants/x/internal-ca.pem",
		"client_cert": "/etc/varnish/tenants/x/client.pem",
		"client_key": "/etc/varnish/tenants/x/client.key"
	}
]

TLS files must use PEM format, and their paths must be absolute. The sandboxed program cannot access these files or select alternative files, so private keys remain outside the sandbox. ca_file replaces the system CA certificates for that entry. You can omit client_key if client_cert also contains the private key.

Entries with TLS files must have a path ending in a slash. Entries for the same URL cannot specify different TLS files. If multiple entries allow a request URL, the entry with the longest matching path prefix determines which TLS files are used. Requests that disable certificate verification (FETCH_FLAG_INSECURE) are rejected for origins configured with TLS files.

The Varnish worker process reads the TLS files for each new connection, so the files must be readable by the worker's user. File changes apply to subsequent connections without a reload. With libcurl versions older than 7.83.1, requests using TLS files disable connection and TLS session reuse to prevent connections from being shared across different TLS configurations.

int32_t h[3] = {
	api::fetch_get("https://api.example.com/v1/a", "Accept: application/json"),
	api::fetch_get("https://api.example.com/v1/b"),
	api::fetch_post("http://10.0.0.5:8080/log", body, "Content-Type: text/plain"),
};
api::FetchResponse r[3];
api::fetch_wait(h, r, FETCH_FLAG_ANY); // at least one is done: its handle is now -1
api::fetch_wait(h, r);                 // all of them are done
if (r[0].ok() && r[0].status() == 200)
	use(r[0].body(), r[0].header("Content-Type"));

The whole response is handed over when a request finishes: the system call allocates its body and headers on the program's native heap, with the exact sizes, and FetchResponse frees them. See ECALL_FETCH in syscalls.h for the system call itself.

Per-viewer streaming manifests

manifest.cpp personalizes HLS and DASH manifests for each viewer while Varnish caches a single copy of each manifest and segment. A session starts with a manifest request that carries an entitlement from the subscriber's authentication service. riscv.deliver_transform() signs every supported URI in the manifest for that session before delivery, so the only way to fetch a segment is through a URL from that viewer's manifest:

sub vcl_recv {
	if (!riscv.fork("streaming")) {
		return (synth(503));
	}
	set req.http.X-Client-IP = client.ip;
	set req.http.X-Token = riscv.call("authorize", req.url);
	if (req.http.X-Token !~ "^[A-Za-z0-9_.-]+$") {
		return (synth(403));
	}
	if (req.url ~ "^/live/[^/]+/(low|high)/seg[0-9]+\.m4s") {
		set req.http.X-Variant = riscv.call("watermark", req.http.X-Token);
	}
	set req.url = regsuball(req.url, "(?<=[?&])(token|auth)=[^&#]*&?", "");
	set req.url = regsub(req.url, "[?&]$", "");
	if (req.http.X-Variant ~ "^[AB]$") {
		set req.url = regsub(req.url, "^(.*/)([^/?]+)", "\1" + req.http.X-Variant + "/\2");
	}
}
sub vcl_backend_fetch {
	unset bereq.http.X-Token;
}
sub vcl_deliver {
	if (req.url ~ "(?i)\.(m3u8|mpd)(\?|$)" && resp.status == 200) {
		if (!riscv.deliver_transform("personalize", req.http.X-Token)) {
			return (synth(503));
		}
		set resp.http.Cache-Control = "private, no-store";
	}
}

The tenant reads two config files and reloads when either changes:

"streaming": {
    "filename": "/path/to/program/cpp/manifest.cpp",
    "config_files": {
        "settings": "/etc/varnish/streaming/settings.json",
        "revocations": "/etc/varnish/streaming/revoked.txt"
    }
}

{
    "keys": {"2026-11": "url-signing-secret", "2026-10": "previous-secret"},
    "signing_key": "2026-11",
    "entitlement_keys": {"auth": "entitlement-secret"},
    "subscriber_key": "subscriber-pseudonym-secret",
    "watermark_key": "watermark-secret",
    "session_lifetime": 21600,
    "segment_lifetime": 120,
    "bind_network": {"ipv4": 24, "ipv6": 48}
}

• Entitlements. A manifest request without a token requires an HS256 JWT in the auth parameter, signed with one of the entitlement_keys. If multiple entitlement keys are configured, the JWT's kid header must identify the signing key. The sub and exp claims are required; nbf is checked if present. An optional path claim restricts access to manifest paths that start with that prefix. The tenant logs Session <id> for subscriber <sub> (VCL_Log) for each new session.
• Tokens. The format is <key id>.<subscriber>.<session>.<expires>.<HMAC>. The subscriber pseudonym is the first 12 bytes of an HMAC of sub using subscriber_key. The session ID consists of 12 random bytes. Both are base64url-encoded. The HMAC covers <key id>.<subscriber>.<session>.<expires>\n<network>\n, padded with \n to a multiple of 64 bytes, followed by the path. This padding allows the token prefix to be hashed once per manifest. Each URI is resolved against the manifest URL (and DASH BaseURL elements) and signed for that exact path, so a token works for only that segment, rendition, and channel. Absolute URLs are signed for their path. Paths with dot segments, empty segments, percent escapes, or backslashes are rejected because origin normalization could map them to a different file. For manifests, authorize returns the token, and personalize verifies it against req.url before using it to sign URIs.
• Lifetimes. Manifest URIs (variant playlists, Location, PatchLocation, xlink:href, and similar references) expire with the session (session_lifetime, at most 10 years). Segment URIs in live playlists expire after segment_lifetime or when the session expires, whichever comes first. Each playlist refresh signs them again. Segment URIs in VOD playlists (ending in #EXT-X-ENDLIST) and static MPDs remain valid until the session expires, allowing playback to resume after a pause within that period.
• Key rotation. Each token identifies the key used to sign it and is verified with the matching key in keys. To rotate keys, add a new key and set it as the signing_key. Remove the old key once session_lifetime has elapsed since the switch.
• Revocation. revocations lists subscriber IDs (the sub claim) and session IDs, one per line. Blank lines and lines starting with # are ignored. Revoked subscribers cannot start new sessions. Every request associated with a revoked subscriber or session receives a 403 response, including requests from existing sessions.
• Network binding. With bind_network, tokens are valid only from the client network that started the session: the address in X-Client-IP, masked to the configured prefix length. The HMAC includes the network in a form such as 4/24/c00002 (address family, prefix length, and masked address bytes in hex). Shorter prefixes allow a wider range of addresses; longer prefixes may interrupt playback for mobile clients whose addresses change.
• Watermarking. watermark returns A or B for the requested path and the token's session, allowing VCL to serve one of two forensically watermarked copies of each segment. Each session has its own sequence of variants, and both variants remain cached. The sequence in a restream can identify the session so it can be revoked. To compute the variant, let d = HMAC-SHA256(watermark_key, path) and i = big-endian(d[0..3]) % 96. Using zero-based indices, take bit i % 6 of the numeric base64url value of session character i / 6, then XOR it with the low bit of d[4]. A result of 0 selects A; 1 selects B. The client cannot choose the variant because the token is signed for the path without the variant.
• DASH SegmentTemplate URIs are expanded by the player, so they can't be signed per URL. They use <key id>.<subscriber>.<session>.<expires>.<P>.<S>.<HMAC>, which signs the P bytes before the first $, the S bytes after the last $, and the number of / between them in the resolved path. For video/$RepresentationID$/seg_$Number$.m4s the token is valid for /vod/movie/video/ + one directory + anything + .m4s. Template tokens are never valid for manifests.
• HLS: A token is added to URI lines and URI="..." attributes (not X-ASSET-URI and other attributes ending in URI) in tags such as EXT-X-MAP, EXT-X-MEDIA, EXT-X-KEY, and EXT-X-PART. skd: and data: URIs remain unchanged. Master playlists include the session ID in a #EXT-X-SESSION-DATA tag with DATA-ID="com.varnish.viewer-id".
• DASH: A token is added to URI attributes in SegmentTemplate, SegmentURL, Initialization, RepresentationIndex, and BitstreamSwitching, to xlink:href on Period, AdaptationSet, SegmentList, and EventStream, as well as to Location, PatchLocation, and ContentSteering values and BaseURL values that refer to files. Directory BaseURL values remain unchanged because their query strings are discarded when relative URIs are resolved against them. URIs are signed for the first BaseURL of an element, so alternative directory BaseURL elements with a different path are removed; alternatives that differ only in host are kept. The token is appended to existing query strings with &.
• Any service with a key can issue tokens with HMAC-SHA256 truncated to 128 bits and base64url-encoded (see mac() in the test).
• Signing costs one HMAC per URI. On a Varnish Enterprise test host (Ryzen 9 7950X, asmjit), personalizing a 49 KB VOD playlist with 1,800 segments takes about 0.9 ms, about 2.6% longer than with the previous version's shorter tokens. Verifying a segment token takes about 1 µs longer. Each HMAC is a single host call (api::HmacSha256, using OpenSSL SHA-256).

The runnable demo in demo/manifests.vcl uses the included sample streams and A/B segment variants. It uses vsthrottle to limit new sessions per subscriber and requests per session, and logs viewer <subscriber> <session> for each request. Integration tests are in tests/manifest_cpp.vtc. This example requires Varnish Enterprise.

JavaScript example program

Tenant logic can also be written in plain JavaScript, powered by the embedded QuickJS-ng engine. No compiler or external toolchain is required: the QuickJS program is embedded in the VMOD. Point "filename" at a .js file:

sub vcl_init {
    riscv.embed_tenants("""{
        "customer1.com": {
            "filename": "/path/to/customer1.js"
        }
    }""");
    riscv.finalize_tenants();
}

The VMOD runs the embedded program with the file as its last argument. A reload loads the file again if it has changed, and live_update_file() with a .js file replaces the script. After changing program/cpp/js.cpp, rebuild the embedded program with make -C program blob (Zig 0.17.0 and llvm-objcopy) and commit program/js.elf.

The source can also be passed as a string directly in your VCL via add_main_argument, with the tenant filename pointing at a js binary built from program/cpp/js.cpp:

sub vcl_init {
    riscv.embed_tenants("""{
        "customer1.com": {
            "filename": "/path/to/js"
        }
    }""");
    riscv.add_main_argument("customer1.com", """
function on_recv(req) {
    req.set("X-Tenant: customer1");
    if (req.url == "/health") return ["synth", 200];
    return ["pass"];
}
""");
    riscv.finalize_tenants();
}

sub vcl_recv {
    if (!riscv.fork(req.http.Host)) {
        return (synth(403));
    }
    riscv.run();
    if (riscv.want_result() == "synth") {
        return (synth(riscv.want_status()));
    }
}

The available hooks mirror the VCL stages: on_recv, on_hash, on_synth, on_deliver, on_backend_fetch, on_backend_response. Each hook receives request/response objects with get, set, and unset methods for header manipulation. Return an array like ["synth", 200] or a string like "pass" to drive the VCL decision.

VCL tenants

A tenant can also be plain VCL. Point "filename" at a .vcl file and the VMOD compiles it to a tenant program at vcl_init. It uses the same compiler that Carapace uses for its policies, built as a RISC-V program and embedded in the VMOD. So the tenant's source is compiled inside the sandbox: a fresh VM per compile, an instruction budget, no filesystem and no sockets. A compile takes about 10 ms. No toolchain is needed.

vcl 4.1;

sub vcl_recv {
    if (req.url ~ "^/admin") {
        return (synth(403, "denied"));
    }
    set req.http.X-Tenant = "customer1";
    return (hash);
}

sub vcl_hash {
    hash_data(req.url);
    return (lookup);
}

sub vcl_synth {
    set resp.http.Content-Type = "text/plain";
    synthetic("no");
    return (deliver);
}

sub vcl_backend_response {
    set beresp.ttl = 1m;
}

The surrounding VCL forks the tenant in each subroutine and dispatches the return action. riscv.run() executes the tenant's hook for the current subroutine. riscv.want_result() returns the action string: "hash", "lookup", "pass", "miss", "fetch", "deliver", "synth", "error" or "abandon". It returns "" when the tenant has no hook for the subroutine or the hook did not return an action, in which case the surrounding VCL falls through to Varnish's built-in behavior. riscv.want_status() and riscv.want_reason() return the arguments of synth(...) and error(...). return (fail) calls VRT_fail on the subroutine.

tests/vcl_router.vcl is a complete surrounding VCL covering every subroutine. In short:

sub vcl_init {
    riscv.embed_tenants("""{
        "customer1.com": { "filename": "/etc/varnish/tenants/customer1.vcl" }
    }""");
    riscv.finalize_tenants();
}

sub vcl_recv {
    if (!riscv.fork(req.http.Host)) {
        return (synth(404));
    }
    riscv.run();
    if (riscv.want_result() == "synth") {
        return (synth(riscv.want_status(), riscv.want_reason()));
    } else if (riscv.want_result() == "pass") {
        return (pass);
    } else if (riscv.want_result() == "hash") {
        return (hash);
    }
}

sub vcl_hash {
    riscv.run();
    if (riscv.want_result() == "lookup") {
        return (lookup);
    }
}

# vcl_hit, vcl_miss, vcl_pass and vcl_deliver: the same, for their actions.

sub vcl_synth {
    if (riscv.active()) {
        riscv.run();
        if (riscv.want_result() == "deliver") {
            return (deliver);
        }
    }
}

sub vcl_backend_fetch {
    if (!riscv.fork(bereq.http.Host)) {
        return (error(503));
    }
    riscv.run();
    if (riscv.want_result() == "error") {
        return (error(riscv.want_status(), riscv.want_reason()));
    } else if (riscv.want_result() == "abandon") {
        return (abandon);
    }
}

# vcl_backend_response and vcl_backend_error: the same, for their actions.

Tenant VCL follows Varnish Enterprise VCL, with the same subroutines, variables, return actions and scope rules:

• Subroutines: vcl_recv, vcl_hash, vcl_hit, vcl_miss, vcl_pass, vcl_deliver, vcl_synth, vcl_backend_fetch, vcl_backend_response and vcl_backend_error.
• Return actions: hash, lookup, miss, fetch, pass, deliver, abandon, synth(status, "reason"), error(status, "reason") and fail. Custom-status synth works as usual, e.g. return (synth(750, "/elsewhere")) handled by a redirect in vcl_synth.
• hash_data() in vcl_hash. synthetic(), set resp.body and set beresp.body in vcl_synth and vcl_backend_error.
• Headers on req, bereq, beresp and resp. Writable: req.url, req.method, bereq.url, bereq.method, resp.status, resp.reason, beresp.status, beresp.reason. beresp.ttl, grace, keep, uncacheable, do_stream, do_gzip and do_gunzip. Read-only obj fields: obj.hits, obj.ttl, obj.status, etc. req.xid, req.restarts, req.esi_level, req.can_gzip, req.hash_always_miss, req.hash_ignore_busy, bereq.retries, bereq.uncacheable, the proto fields, server.hostname, server.identity, client.ip, now. req.cache_hit (non-standard) indicates whether vcl_deliver is serving a cache hit.
• Regular expressions, compiled by Varnish's regex engine, under Varnish's default pcre_match_limit (10000) and pcre_match_limit_recursion (20). A match that reaches a limit fails the subroutine instead of reading as "no match", so a deny rule cannot fail open. Each match is charged to the tenant's instruction budget. ACLs, user subroutines and include (confined to the tenant's directory).
• VMODs: std, str, digest, headerplus, cookieplus, urlplus. Lexical locals (var inside a sub).
• Request globals (var at the top level). The client side (vcl_recv through vcl_deliver and vcl_synth) shares one copy. The backend fetch starts from the initializers, not from the client-side values, because it runs in a separate fork. Pass values to the backend on bereq headers, as in Varnish.
• Statistics: static var NAME: INT stat [counter|gauge|max|min] ["description"]; declares a Varnish counter named RISCV.<tenant>.<name>, visible in varnishstat. See below.

The following are denied by the compiler and enforced again at runtime by the VMOD:

• req.http.Host is read-only. The surrounding VCL uses it to select the tenant on both client and backend sides; allowing writes would route the request to another tenant. To set the origin Host header, use bereq.http.Host in vcl_backend_fetch.
• Cache key isolation: the VMOD prepends the tenant name to every cache key before vcl_hash runs, preventing cross-tenant cache reads or poisoning.
• Restarts, retries, pipe, purge, vcl_purge, vcl_pipe, vcl_init, vcl_fini, bans, backends, directors, storage selection, timeouts and beresp.do_esi (whose includes could reference another tenant's origin). These are reserved for the surrounding VCL.
• Plain static var (non-stat), because each request runs in a fresh fork with no state carried between requests.
• Writes to framing headers (Content-Length, Transfer-Encoding, etc.).
• Variables used outside their allowed subroutine scope. req.* belongs to the client side and bereq.* to the backend side, as in Varnish.

Statistics

import std;

static var blocked: INT stat "Requests the tenant refused";
static var inflight: INT stat gauge;
static var largest_body: INT stat max "Largest Content-Length seen";

sub vcl_recv {
    set var.inflight += 1;
    if (req.url ~ "^/admin") {
        set var.blocked += 1;
        return (synth(403, "denied"));
    }
}

sub vcl_deliver {
    set var.inflight -= 1;
}

sub vcl_backend_response {
    set var.largest_body = std.integer(beresp.http.Content-Length, 0);
}

$ varnishstat -1 -f 'RISCV.*'
RISCV.customer1.com.blocked                     17         0.00 Requests the tenant refused
RISCV.customer1.com.inflight                     0          .   VCL-declared gauge inflight
RISCV.customer1.com.largest_body            524288          .   Largest Content-Length seen

Each fork initializes the static variable to its declared value. After each subroutine, the VMOD reads the variable and applies the delta to the tenant's counter. counter (the default) and gauge add the difference. max and min retain the highest or lowest value seen. A subroutine observes only its own request's delta, not the running total. min has no initializer and reads INT64_MAX until a request sets a lower value. A gauge that decrements below zero wraps, because Varnish counters are unsigned.

A tenant may have at most 64 statistics. The cap counts every name a tenant has published since varnishd started, because a counter is never freed and all tenants share the same shared memory. A reload that would add a name past the cap is refused, and the running program stays. Counters belong to the tenant, not to the program binary. They persist across riscv.live_update_file() and VCL reloads, and live until varnishd exits. Redeclaring a name with a different kind is refused. A changed description is logged; the original description is kept.

A .vcl file that fails to compile leaves the tenant without a program: riscv.fork() returns false and the compiler diagnostics are logged as Error records. riscv.live_update_file() also accepts .vcl files; a failed compile leaves the running program unchanged.

Forks take their dirtied pages from the task workspace, so give tenants some room, e.g. -p workspace_client=128k. vcl/README.md has the compiler's layout and how to rebuild it.

Benchmarks

Performance comparison showing the minimal overhead of RISC-V VMs:

RISC-V ephemeral VMs configuring the request:

$ ./wrk -c1 -t1 -L http://127.0.0.1:8000/riscv
Running 10s test @ http://127.0.0.1:8000/riscv
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.62us    2.53us 266.00us   98.65%
    Req/Sec    84.51k     0.94k   86.88k    72.73%
  Latency Distribution
     50%   11.00us
     75%   12.00us
     90%   12.00us
     99%   15.00us
  184992 requests in 2.20s, 63.98MB read
Requests/sec:  84083.68
Transfer/sec:     29.08MB

A regular Varnish cache hit, with equivalent work to RISC-V above:

$ ./wrk -c1 -t1 -L http://127.0.0.1:8000/varnish
Running 10s test @ http://127.0.0.1:8000/varnish
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.70us    7.30us 703.00us   99.75%
    Req/Sec    92.33k     2.12k   94.84k    88.46%
  Latency Distribution
     50%   10.00us
     75%   11.00us
     90%   11.00us
     99%   14.00us
  238839 requests in 2.60s, 215.02MB read
Requests/sec:  91862.25
Transfer/sec:     82.70MB

We can see that the single-threaded overhead from ephemeral VMs configuring the request is only ~1 microsecond.

Ubuntu Linux 6.14.0-33-generic, AMD Ryzen 9 7950X

Building the VMOD

Requirements:

sudo apt install build-essential cmake g++

Open-source Varnish:

./build.sh

Enterprise Varnish:

./build.sh --enterprise

Custom RISC-V compiler

For Rust (and some other languages) you will need a full 64-bit Linux cross-compiler, in which case you should ignore this step and install one from your distro packaging. Yes, you do have a RISC-V compiler, and no I don't know the incantation to install it for you.

A custom compiler is needed to make the most efficient C++ guest programs. Clone the RISC-V GNU toolchain, and build it like so:

./configure --prefix=$HOME/riscv --with-arch=rv32g_zba_zbb_zbc_zbs --with-abi=ilp32d
make

After completion, expose the compiler by adding ~/riscv/bin to PATH. Verify with:

$ riscv32-unknown-elf-g++ 
riscv32-unknown-elf-g++: fatal error: no input files

Again, this step is not necessary. Do not do this unless you want to flex.

Running

cd demo
./run.sh

It will automatically build a basic program, however the filepath might be wrong on your system. Please edit the Demo VCL with your path to the example program.