Signed traffic and evidence bundles

Two things nobody else gives you: monitoring traffic your firewall can trust without an IP allowlist, and a signed record of what each run proved.

Signed traffic

On plans that include it, every HTTP request the runner's browser makes during a check, the page itself and each script, image, font and API call it loads, carries three headers from RFC 9421 HTTP Message Signatures, the scheme Cloudflare calls Web Bot Auth:

Signature-Agent: "https://flarely.app"
Signature-Input: sig1=("@authority" "signature-agent");created=…;expires=…;keyid=…;tag="web-bot-auth"
Signature: sig1=:…:

Each signature is fresh, covers the host it is sent to, and expires after a minute. A verifier fetches our public keys from https://flarely.app/.well-known/http-message-signatures-directory, checks the signature, and knows the request came from Flarely and nowhere else. Nothing in the signature identifies your account; only that the traffic is ours.

Signing is on for every run on Team, including runs started from the CLI, the GitHub Action, MCP, schedules and deploy hooks, and for the check generator's browsing. Free and Solo runs are unsigned. Tests you write need no changes. Only requests made through the test's browser context are signed; a second context you open yourself with browser.newContext() is not.

Allowing Flarely in Cloudflare

With Bot Management or Super Bot Fight Mode, add flarely.app under Security, Bots, Signed agents. Cloudflare then verifies every request against our directory and treats it as a signed agent, which you can allow in a WAF custom rule or in your bot rules. No IP list, and nothing changes when we scale.

Without Bot Management, a custom rule that matches the header is a workable fallback. It proves presence of the header, not the signature, so pair it with a rate limit rather than skipping the WAF outright:

(http.request.headers["signature-agent"][0] eq "\"https://flarely.app\"")

Other firewalls

AWS WAF: a byte-match rule on the signature-agent header containing flarely.app, action Allow or Count. Verify cryptographically with the snippet below in a Lambda authoriser if you need proof rather than presence.
Akamai, Fastly, Vercel: same header match on signature-agent. Fastly's Compute and Vercel Middleware can run the verification snippet inline.

Verifying a signature yourself

Cloudflare's open source library verifies against the same directory. In any Node or edge runtime:

import { verify } from 'web-bot-auth';
import { verifierFromJWK } from 'web-bot-auth/crypto';

const directory = await fetch('https://flarely.app/.well-known/http-message-signatures-directory').then((r) => r.json());
const verifiers = await Promise.all(directory.keys.map(verifierFromJWK));

const result = await verify(request, {
  resolver: (candidate) => {
    const v = verifiers.find((k) => k.keyid === candidate.keyid);
    if (!v) throw new Error('not a Flarely key');
    return v;
  },
});
// result.keyid, result.created, result.expires

Keys rotate by appending a new one to the directory before the old one retires, so cache the directory for at most a day.

Evidence bundles

On Team, every finished run also gets an evidence bundle: a JSON manifest listing the check and its exact code hash, each test with its steps and outcome, the runner's Playwright and Node versions, and the SHA-256 of every trace, video and screenshot the run produced. The manifest is signed with the same key that signs our traffic.

It appears as the run's manifest artifact, on the run page as Evidence bundle, as [evidence] in the GitHub Action's PR comment, and at GET /v1/runs/:id/evidence on the API. Bundles are kept for as long as the run's other artifacts.

Verifying a bundle offline

A third party needs only Node 20 and the public key. Save the script below as verify-evidence.mjs; it fetches our directory unless you pass --jwks with a saved copy, checks the signature over the canonical manifest, and with --dir compares downloaded artifacts against their recorded hashes.

node verify-evidence.mjs evidence.json
node verify-evidence.mjs evidence.json --dir ./artifacts
node verify-evidence.mjs evidence.json --jwks flarely-keys.json

verify-evidence.mjs

#!/usr/bin/env node
/**
 * Verify a Flarely evidence bundle offline.
 *
 *   node scripts/verify-evidence.mjs evidence.json
 *   node scripts/verify-evidence.mjs evidence.json --dir ./artifacts   # also check file hashes
 *   node scripts/verify-evidence.mjs evidence.json --jwks keys.json    # use a saved key set
 *
 * Checks that the signature over the canonical manifest verifies with a
 * public key from Flarely's key directory (fetched unless --jwks is
 * given), and, with --dir, that every artifact file present under that
 * directory (named by its manifest key) matches its recorded SHA-256.
 * Exits non-zero on any failure. No dependencies beyond Node 20.
 */
import { readFile } from 'node:fs/promises';
import { createHash, webcrypto } from 'node:crypto';
import { join } from 'node:path';

const args = process.argv.slice(2);
const file = args.find((a) => !a.startsWith('--'));
if (!file) {
  console.error('usage: verify-evidence.mjs <evidence.json> [--dir <folder>] [--jwks <keys.json>]');
  process.exit(2);
}
const opt = (name) => { const i = args.indexOf(name); return i >= 0 ? args[i + 1] : undefined; };

const canonical = (v) => {
  if (v === null || typeof v !== 'object') return JSON.stringify(v);
  if (Array.isArray(v)) return `[${v.map(canonical).join(',')}]`;
  return `{${Object.keys(v).sort().map((k) => `${JSON.stringify(k)}:${canonical(v[k])}`).join(',')}}`;
};
const thumbprint = ({ crv, kty, x }) => createHash('sha256').update(JSON.stringify({ crv, kty, x })).digest('base64url');
const b64u = (s) => Buffer.from(s.replace(/-/g, '+').replace(/_/g, '/'), 'base64');

const bundle = JSON.parse(await readFile(file, 'utf8'));
const { manifest, signature } = bundle;
if (!manifest || !signature) { console.error('FAIL not an evidence bundle (needs manifest and signature)'); process.exit(1); }

let jwks;
if (opt('--jwks')) jwks = JSON.parse(await readFile(opt('--jwks'), 'utf8'));
else {
  const res = await fetch(signature.directory);
  if (!res.ok) { console.error(`FAIL could not fetch key directory ${signature.directory}: ${res.status}`); process.exit(1); }
  jwks = await res.json();
}
const key = (jwks.keys ?? []).find((k) => k.kty === 'OKP' && k.crv === 'Ed25519' && thumbprint(k) === signature.keyid);
if (!key) { console.error(`FAIL no published key matches keyid ${signature.keyid}`); process.exit(1); }

const publicKey = await webcrypto.subtle.importKey('jwk', { kty: 'OKP', crv: 'Ed25519', x: key.x }, { name: 'Ed25519' }, false, ['verify']);
const ok = await webcrypto.subtle.verify({ name: 'Ed25519' }, publicKey, b64u(signature.value), new TextEncoder().encode(canonical(manifest)));
if (!ok) { console.error('FAIL signature does not verify: the manifest was altered or signed by another key'); process.exit(1); }
console.log(`ok   signature verifies with key ${signature.keyid} published by ${manifest.issuer}`);
console.log(`     run ${manifest.run.id} on check "${manifest.check.name}" (${manifest.check.contentHash.slice(0, 12)}…) ended ${manifest.run.status}`);
console.log(`     ${manifest.tests.length} test(s), ${manifest.artifacts.length} artifact(s), generated ${new Date(manifest.generatedAt).toISOString()}`);

let failures = 0;
const dir = opt('--dir');
if (dir) {
  for (const a of manifest.artifacts) {
    let bytes;
    try { bytes = await readFile(join(dir, a.key)); } catch { console.log(`skip ${a.kind} ${a.key} (not present)`); continue; }
    const hash = createHash('sha256').update(bytes).digest('hex');
    if (a.sha256 && hash === a.sha256) console.log(`ok   ${a.kind} ${a.key} sha256 matches`);
    else { failures += 1; console.error(`FAIL ${a.kind} ${a.key} sha256 ${hash} != ${a.sha256 ?? '(unrecorded)'}`); }
  }
}
if (failures) process.exit(1);
console.log('Evidence bundle verified.');

The manifest is canonical JSON: keys sorted at every level, no whitespace. The signature is Ed25519 over those bytes, and signature.keyid is the RFC 8037 thumbprint of the public key that made it. Any edit to the manifest, or a bundle signed by anyone but Flarely, fails verification.