Palav/dosh
1
0
Fork 0
Code Issues Pull Requests Actions 58 Packages Projects Releases 51 Wiki Activity

Expand native auth and benchmark gates

Browse Source
This commit is contained in:
DuProcess
2026-06-18 09:29:06 -04:00
parent 2835da76b0
commit 25d9a6aefa
13 changed files with 1036 additions and 178 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/BENCHMARKS.md
+47 -19
View File
@@ -93,11 +93,13 @@ Tunables: `scripts/bench-local.sh [ITERATIONS]` (default 20), `DOSH_BENCH_JSON=1
### `make bench-docker-ssh` / `make bench-docker-mosh` (`scripts/ci-docker-ssh-bench.sh`)
Builds one Ubuntu image with OpenSSH, `dosh-server`, `dosh-auth` (and Mosh for the
`-mosh` target), then runs `ssh_true_ms`, cold + ControlMaster `dosh_attach_ms`,
`dosh_cached_attach_ms`, and optionally `mosh_start_true_ms` against the same
container, key, and loopback network path. This is the gate CI enforces; it asserts
cold Dosh stays within 500 ms of SSH and that cached attach stays under a small
budget. See `README.md` "Develop" for the exact invocations.
`-mosh` target), then runs `ssh_true_ms`, cold + ControlMaster SSH-bootstrap
`dosh_attach_ms`, one-time `dosh trust` followed by native-cold
`dosh_cold_native_ms`, `dosh_cached_attach_ms`, and optionally
`mosh_start_true_ms` against the same container, key, and loopback network path.
This is the gate CI enforces; it asserts cold SSH-bootstrap Dosh stays within
500 ms of SSH, native-cold Dosh beats SSH mean by default, and cached attach stays
under a small budget. See `README.md` "Develop" for the exact invocations.
## Methodology and caveats
@@ -130,8 +132,8 @@ budget. See `README.md` "Develop" for the exact invocations.
## Sample results (loopback, self-contained)
Captured with `make bench-local` (`scripts/bench-local.sh 30`), all times in
milliseconds, 30 samples per metric.
Captured with `make bench-local` (`scripts/bench-local.sh 20`), all times in
milliseconds, 20 samples per metric.
- Machine: Intel Core i5-9500 @ 3.00 GHz, 6 cores
- OS: Ubuntu 24.04.4 LTS, Linux 6.8.0-124-generic, x86_64
@@ -141,27 +143,24 @@ milliseconds, 30 samples per metric.
| Metric | n | min | median | p95 | mean | max |
| --- | --- | --- | --- | --- | --- | --- |
| `dosh_cold_native_ms` | 30 | 8.10 | 9.01 | 10.40 | 9.18 | 10.82 |
| `dosh_cached_attach_ms` | 30 | 2.73 | 2.96 | 3.25 | 2.96 | 3.30 |
| `dosh_local_attach_ms` | 30 | 2.66 | 2.78 | 3.24 | 2.87 | 3.33 |
| `dosh_cold_native_ms` | 20 | 8.32 | 8.77 | 9.34 | 8.82 | 9.52 |
| `dosh_cached_attach_ms` | 20 | 2.83 | 3.23 | 3.75 | 3.24 | 3.77 |
| `dosh_local_attach_ms` | 20 | 2.84 | 3.21 | 3.96 | 3.28 | 3.98 |
Raw samples (ms):
```
dosh_cold_native_ms:
8.32, 8.23, 10.31, 10.37, 8.83, 8.81, 9.20, 9.04, 10.14, 8.98, 10.82, 9.87,
10.22, 10.42, 9.76, 9.09, 9.19, 8.94, 8.69, 8.67, 8.74, 9.33, 9.25, 8.92, 8.39,
8.55, 8.38, 9.81, 8.17, 8.10
8.82, 8.60, 8.69, 8.71, 9.33, 8.82, 8.90, 8.64, 8.39, 8.75, 8.79, 8.70, 8.64,
9.52, 8.72, 8.32, 9.06, 8.93, 8.96, 9.04
dosh_cached_attach_ms:
2.99, 2.99, 2.93, 2.87, 2.95, 2.80, 2.80, 2.90, 3.10, 2.73, 3.19, 2.80, 2.87,
2.82, 2.89, 3.03, 3.06, 3.21, 2.97, 2.83, 3.00, 2.82, 3.04, 2.97, 3.03, 3.24,
3.30, 2.73, 2.79, 3.25
3.04, 3.06, 2.98, 2.91, 2.96, 2.95, 3.05, 3.66, 3.37, 2.88, 2.83, 3.16, 3.49,
3.30, 3.77, 3.32, 3.49, 3.75, 3.56, 3.33
dosh_local_attach_ms:
3.05, 3.00, 2.84, 3.04, 3.33, 2.89, 2.76, 2.74, 2.73, 3.32, 3.14, 2.75, 2.83,
2.88, 2.99, 2.77, 2.75, 2.79, 2.69, 2.78, 2.75, 2.77, 2.75, 2.77, 3.03, 2.75,
2.94, 2.74, 2.82, 2.66
3.08, 3.96, 3.36, 3.30, 2.88, 2.99, 3.96, 3.98, 3.25, 2.84, 3.29, 3.59, 3.65,
3.47, 3.13, 2.92, 2.87, 2.91, 2.98, 3.18
```
Reading these numbers: cold native auth is ~9 ms because it pays the native
@@ -171,3 +170,32 @@ overhead floor. Over a real link, expect cached attach ≈ this floor + one netw
round trip — which is exactly the "near network RTT" target in the spec. There is
no SSH/Mosh baseline in this loopback table because those paths need an SSH server;
use `make bench-docker-ssh` / `make bench-docker-mosh` for the head-to-head.
## Sample results (Docker SSH/Mosh comparison)
Captured with `make bench-docker-mosh`, all times in milliseconds. This is the
same-container comparison gate: one generated key, one OpenSSH server, one
`dosh-server`, loopback-published TCP/UDP ports.
| Metric | n | min | median | p95 | mean | max |
| --- | --- | --- | --- | --- | --- | --- |
| `ssh_true_ms` (cold SSH baseline for native-cold gate) | 5 | 214.53 | 216.84 | 220.55 | 217.60 | 220.91 |
| `dosh_cold_native_ms` | 5 | 8.62 | 8.96 | 9.78 | 9.15 | 9.88 |
| `dosh_cached_attach_ms` | 10 | 8.41 | 8.83 | 9.52 | 8.90 | 9.53 |
| `mosh_start_true_ms` | 3 | 528.03 | 530.87 | 539.23 | 533.02 | 540.16 |
Raw samples (ms):
```
ssh_true_ms (native-cold gate):
216.84, 216.61, 214.53, 220.91, 219.09
dosh_cold_native_ms:
9.39, 8.88, 8.96, 8.62, 9.88
dosh_cached_attach_ms:
8.44, 8.41, 8.80, 8.48, 9.53, 9.51, 8.85, 8.69, 9.33, 8.96
mosh_start_true_ms:
540.16, 528.03, 530.87
```