Add benchmark path matrix and safe local harness
Extend dosh-bench to benchmark the full attach path matrix and emit both raw per-iteration samples and summary statistics (count/min/median/p95/mean/max): - --cold-native: native cold auth terminal-ready (dosh_cold_native_ms) - --cached-ticket: cached attach-ticket fast path (dosh_cached_attach_ms) - --resume: UDP resume path (dosh_resume_ms; roaming-only, see docs) - --local-auth: self-contained local bootstrap, no SSH (dosh_local_attach_ms) - default: legacy SSH-bootstrap cold attach (dosh_attach_ms) Existing ssh-bootstrap/local-auth/mosh modes and all assert gates are preserved; legacy flags (--local-auth/--warm-cache/--no-cache) keep their prior behavior so the CI docker scripts run unchanged. Add --json (machine-readable raw samples) and --label. Add a table/JSON renderer and percentile-based summary stats. Add scripts/bench-local.sh: a safe, self-contained harness that builds release, spins up a throwaway dosh-server bound to 127.0.0.1 on a random free UDP port in a temp HOME (never port 50000, never a systemd unit), runs the matrix, prints results, sanity-checks native auth actually trusted the host, and cleans up. Point `make bench-local` at the new harness and add `make bench-local-json`; keep bench-docker-* targets intact. Add docs/BENCHMARKS.md: how to run each benchmark, metric meanings, methodology and caveats (resume is the roaming path, not cold reconnect; cite cached attach as the core claim per PUBLIC_READINESS.md), plus a real loopback sample table with machine/OS/sample-count and raw samples. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
.PHONY: build test fmt install bench-local bench-docker-ssh bench-docker-mosh
|
||||
.PHONY: build test fmt install bench-local bench-local-json bench-docker-ssh bench-docker-mosh
|
||||
|
||||
build:
|
||||
cargo build --release
|
||||
@@ -12,14 +12,16 @@ fmt:
|
||||
install:
|
||||
sh packaging/install.sh
|
||||
|
||||
# Safe, self-contained local benchmark matrix (native cold auth, cached attach
|
||||
# ticket, local-auth) on a throwaway server bound to 127.0.0.1 on a free port in
|
||||
# a temp HOME. Never touches the production server or UDP port 50000.
|
||||
bench-local:
|
||||
cargo build
|
||||
tmp="$$(mktemp -d)"; \
|
||||
HOME="$$tmp" target/debug/dosh-server serve >/tmp/dosh-bench-server.log 2>&1 & \
|
||||
pid="$$!"; \
|
||||
trap 'kill "$$pid" 2>/dev/null || true; rm -rf "$$tmp"' EXIT INT TERM; \
|
||||
sleep 0.5; \
|
||||
HOME="$$tmp" target/debug/dosh-bench --local-auth --server local --iterations 5
|
||||
sh scripts/bench-local.sh
|
||||
|
||||
# Same matrix, machine-readable JSON output (one object per metric with raw
|
||||
# samples). Useful for publishing or regression tracking.
|
||||
bench-local-json:
|
||||
DOSH_BENCH_JSON=1 sh scripts/bench-local.sh
|
||||
|
||||
bench-docker-ssh:
|
||||
sh scripts/ci-docker-ssh-bench.sh
|
||||
|
||||
Reference in New Issue
Block a user