# dosh - Dormant Shell Spec **Status:** Implemented Rust build with local and Docker SSH verification **Default language:** Rust, unless benchmarks prove the stack is the bottleneck **Binaries:** `dosh-server`, `dosh-client`, `dosh-auth`, `dosh-bench` **Helper mode:** `dosh-server auth` or `~/.local/bin/dosh-auth`, invoked by SSH with `-T` **Native v1 plan:** `docs/NATIVE_V1_SPEC.md` (substantially implemented; see its v1 status block) **Threat model:** `docs/THREAT_MODEL.md` --- ## 1. Product Shape dosh is a fast-attach remote terminal. It borrows the useful shape of mosh - UDP transport, roaming, and latency-tolerant terminal rendering - but optimizes a different first-order problem: getting the user back into an already-running terminal as quickly as possible. The daemon is resident. Sessions are named. A session owns one PTY and one authoritative terminal screen. Clients attach to that session over encrypted UDP. SSH is used for first trust establishment and as fallback when cached credentials are missing, expired, or rejected. Native Dosh auth is specified separately in `docs/NATIVE_V1_SPEC.md` as the path toward replacing the day-to-day SSH workflow on Dosh-installed servers. --- ## 2. Design Goals - Connection speed first: - UDP resume: one encrypted UDP request and one encrypted UDP reply. - UDP attach ticket: one encrypted UDP request and one encrypted UDP reply. - Warm SSH bootstrap: existing-ControlMaster SSH command latency plus one UDP RTT. - Cold SSH bootstrap: cold `ssh host` terminal-ready time plus at most one UDP RTT. - No daemon spawn, PTY spawn, shell startup, rc file execution, or MOTD on attach to an existing session. - Prewarm configured sessions at daemon startup, including `default` by default. - Encrypted UDP terminal data. - Single configurable UDP port. - Multiple clients attached to one session. - Optional read-only clients. - Named persistent sessions. - Reuse existing SSH key infrastructure. - Instrument connection timing from the first implementation. --- ## 3. Non-Goals - Replacing SSH as the first public-key trust mechanism. - Multi-user authorization or ACLs. - Windows support in v0. - Full mosh protocol compatibility. - Perfect local echo/prediction in the first MVP. - QUIC in v0. QUIC can be revisited if measurements show custom UDP is not enough. --- ## 4. Connection Speed Contract Measure terminal-ready time: elapsed time from launching `dosh ...` to first usable terminal render. Benchmarks must use the same host, network, key, DNS path, and SSH config. | Path | Acceptance gate | | --- | --- | | UDP resume | <= one measured UDP RTT + local render time | | UDP attach ticket | <= one measured UDP RTT + local render time | | Warm SSH bootstrap | <= `ssh host true` over existing ControlMaster + one measured UDP RTT | | Cold SSH bootstrap | <= cold `ssh host` terminal-ready time + one measured UDP RTT | | New session | Report separately; PTY/shell creation is expected | Required timing evidence: - Client stderr spans for credential lookup, SSH bootstrap, UDP resume, UDP ticket attach, and terminal-ready time. - `dosh-bench` samples for SSH `true`, Dosh attach, and optional ControlMaster-backed SSH `true`. - `make bench-docker-ssh` gates both cold SSH bootstrap and ControlMaster-backed SSH bootstrap against containerized OpenSSH plus resident `dosh-server`. --- ## 5. Fast Path Order The client always tries the cheapest path that is valid for the requested host/user/session/mode: 1. **UDP resume** - Requires cached `ClientId`, session key, server identity, and unexpired resume metadata. - Sends `ResumeRequest`. - Receives `ResumeOk` with a snapshot or diff. 2. **UDP attach ticket** - Requires cached attach ticket scoped to server identity, SSH username, session, mode, and expiry. - Sends `TicketAttachRequest`. - Receives `AttachOk` with session key, `ClientId`, and snapshot. 3. **SSH bootstrap** - Runs `ssh -T user@host ~/.local/bin/dosh-auth ...`. - Receives attach token, attach ticket, session key material, and server metadata. - Sends `BootstrapAttachRequest`. - Receives `AttachOk` with `ClientId` and snapshot. 4. **New session** - Same as attach, but if the session does not exist and is not prewarmed, the server creates PTY and shell before first paint. --- ## 6. Architecture ```text dosh-server config loader secret manager UDP socket on one port session table: HashMap optional prewarm of configured sessions auth helper mode for SSH bootstrap metrics/timing logger Session PTY master child process/shell terminal parser authoritative screen model scrollback ring monotonic output sequence client table: HashMap ClientState ClientId UDP endpoint mode: read-write | view-only session key id last acked sequence terminal size last seen timestamp dosh-client config loader local credential cache terminal raw mode UDP protocol engine SSH bootstrap runner reconnect state machine renderer ``` Server hot-path ownership should avoid locks on every broadcast. A single event-loop owner per session is preferred. Cross-thread designs are allowed only if benchmarked. --- ## 7. Security Model SSH is the first trust root and the recovery/bootstrap fallback. The v0 core relies on SSH for first authentication. Native v1 (`docs/NATIVE_V1_SPEC.md`) adds an opt-in native public-key login over the Dosh UDP transport that is tried before SSH, with its assets, attackers, and residual risks documented in `docs/THREAT_MODEL.md`. SSH remains the explicit fallback and the host-key trust bootstrap path. The UDP channel uses AEAD. Recommended default: `ChaCha20-Poly1305` for portable speed, with `AES-GCM` allowed when hardware acceleration is known to be available. The negotiated algorithm is recorded in the bootstrap response. All encrypted packets use: - Unique nonce per `(session_key_id, direction)`. - Monotonic packet counter. - Associated data containing protocol version, packet type, session name hash, client id when known, and sequence numbers. - Replay rejection using the packet counter window. Secrets: - `server_secret`: generated on first server start; stored mode `0600`. - `session_key`: random 256-bit key per client attachment, rotated on SSH bootstrap or ticket attach. - `attach_ticket_key`: derived from `server_secret` and rotated by server key epoch. No terminal bytes are sent outside AEAD after the attach handshake begins. --- ## 8. SSH Bootstrap Auth Client command: ```bash ssh -T user@host ~/.local/bin/dosh-auth \ --protocol 1 \ --nonce \ --session \ --mode \ --size x \ --client-version ``` `dosh-auth` must: - Not allocate a PTY. - Not start a shell. - Not run user shell rc files. - Read server config and secret directly. - Return one compact binary or base64url response on stdout. - Exit immediately. Bootstrap response fields: - `protocol_version` - `server_id` - `server_key_epoch` - `issued_at` - `expires_at` - `user` - `session` - `mode` - `terminal_size` - `attach_token` - `attach_ticket` - `attach_ticket_psk` - `session_key` - `session_key_id` - `udp_host` - `udp_port` - `aead_algorithm` `attach_token = HMAC-SHA256(server_secret, user || session || mode || terminal_size || client_nonce || issued_at || expires_at || session_key_id)`. The token TTL defaults to 30 seconds. Attach tickets default to 1 hour and are server-configurable. --- ## 9. Attach Tickets Attach tickets let a new client process attach without spawning SSH again. Ticket properties: - Server-sealed and authenticated by `attach_ticket_key`. - Paired with a client-held random `attach_ticket_psk` returned during SSH bootstrap. - Scoped to server identity, SSH username, session, mode, and key epoch. - Short-lived by default. - Revoked implicitly when server secret/key epoch changes. - Stored client-side with mode `0600`, along with `attach_ticket_psk`. Ticket attach does not prove fresh possession of the SSH private key. It proves recent possession of a server-issued credential. This is acceptable for speed, configurable, and can be disabled with `allow_attach_tickets = false`. Ticket attach flow: 1. Client sends `TicketAttachRequest` containing the sealed ticket, client nonce, and requested terminal size. 2. The request body is AEAD-encrypted with a key derived from `HKDF(attach_ticket_psk, client_nonce || "ticket-attach-request")`. 3. Server opens the sealed ticket, validates scope/expiry/key epoch, derives the same request key, and decrypts the request. 4. Server creates a fresh session key and `ClientId`. 5. `AttachOk` is AEAD-encrypted with `HKDF(attach_ticket_psk, client_nonce || server_nonce || "ticket-attach-ok")` and carries the fresh session key metadata plus first snapshot. 6. Subsequent terminal packets use the fresh session key, not the ticket PSK. --- ## 10. UDP Protocol UDP port defaults to `50000`. One socket handles all sessions and clients. Hot-path terminal packets use a fixed binary header: ```text magic 4 bytes "DOSH" version 1 byte 1 type 1 byte flags 2 bytes conn_id 16 bytes zero before client id is assigned seq 8 bytes sender packet sequence ack 8 bytes latest received peer sequence body_len 2 bytes body body_len bytes tag AEAD tag, length depends on algorithm ``` Packet types: | Type | Direction | Encrypted | Purpose | | --- | --- | --- | --- | | `BootstrapAttachRequest` | client -> server | token-authenticated | Attach after SSH bootstrap | | `TicketAttachRequest` | client -> server | ticket PSK | Attach with cached ticket | | `AttachOk` | server -> client | yes | Assign client id and send first snapshot | | `AttachReject` | server -> client | no terminal bytes | Reject and require SSH | | `ResumeRequest` | client -> server | yes | Resume known client | | `ResumeOk` | server -> client | yes | Endpoint updated; diff/snapshot follows | | `Input` | client -> server | yes | PTY input bytes | | `Resize` | client -> server | yes | Terminal size update | | `Frame` | server -> client | yes | Screen diff or PTY byte frame | | `Ack` | both | yes | Ack without payload | | `Ping` / `Pong` | both | yes | Keepalive and RTT | | `Detach` | client -> server | yes | Remove client, keep session | MTU target: - Default payload target: 1200 bytes. - Larger datagrams may be enabled only after path MTU discovery. - Snapshots larger than the target are chunked. Reliability: - Input packets are reliable and ordered per client. - Output frames are sequenced; clients ack rendered sequence. - Server retransmits unacked frames within a bounded window. - If a client falls too far behind, server sends a fresh snapshot instead of replaying unlimited diffs. --- ## 11. Sessions and PTYs Named sessions: ```bash dosh host # open a fresh generated session dosh --session work host # attach/create named persistent session dosh --session work --view-only host ``` Session behavior: - One PTY per session. - Sessions persist until killed or server exits. - If a session has zero clients, the PTY keeps running. - Plain client attaches use generated session names by default. - `--session ` intentionally reuses or shares a persistent session. - Configured sessions are prewarmed at daemon startup. - If a requested session does not exist: - `attach` creates it only when `create_on_attach = true`. - `new` always creates it and fails if it already exists. Resize policy: - One PTY means one size. - Read-write clients may resize. - View-only clients never resize. - Default policy: latest read-write resize wins. --- ## 12. Screen State Server maintains the authoritative terminal model: - Visible grid. - Cursor position and style. - Alternate screen. - Text attributes and colors. - Scrollback ring. - Monotonic output sequence. Initial attach: - Server sends a snapshot in the first UDP reply if it fits the packet budget. - If not, server sends a minimal first frame immediately and follows with chunks. Diffs: - Diffs are computed per client from that client's last acked rendered sequence. - Lagging clients may receive larger diffs or a full snapshot. - Diffs are preferred over raw PTY bytes for reconnect correctness. Encoding: - Hot terminal frames use fixed binary headers and compact binary payloads. - MessagePack is allowed only for non-hot control/list/config responses. - JSON is not used on the protocol path. --- ## 13. Multi-Client Model Default mode is shared input. Any read-write client can write to the session PTY. All clients see the same resulting screen. View-only mode: - Client suppresses local input. - Server rejects `Input` from view-only clients even if a malformed client sends it. - Promotion/demotion requires reconnect. Client timeout: - Clients are removed after `client_timeout_secs` without ack/ping. - The default is intentionally long enough for laptop sleep/resume, currently one day. Short timeouts make Dosh fall back to attach tickets or SSH after sleep. - Removing a client never kills the session. --- ## 14. Local Echo MVP local echo is conservative: - Printable keystrokes may be rendered optimistically only when the client is in a simple shell line-editing state. - Server output is always authoritative. - On mismatch, client replaces local prediction with server state. Full mosh-style predictive display is a later feature. It must not delay the first implementation of fast attach/resume. --- ## 15. Reconnect and Roaming Client detects possible disconnect when no server packet arrives for `reconnect_timeout_secs`. Reconnect order: 1. Send encrypted `ResumeRequest` to the configured host/port. 2. If accepted, update endpoint server-side and receive diff/snapshot. 3. If rejected, try attach ticket. 4. If ticket attach is rejected, run SSH bootstrap. The server matches resume by `ClientId` and session key id, not by source address. Successful resume updates the client's UDP endpoint. --- ## 16. Configuration Server config: `~/.config/dosh/server.toml` ```toml port = 50000 bind = "0.0.0.0" scrollback = 5000 auth_ttl_secs = 30 attach_ticket_ttl_secs = 3600 allow_attach_tickets = true client_timeout_secs = 86400 retransmit_window = 256 default_input_mode = "read-write" prewarm_sessions = ["default"] create_on_attach = true shell = "/usr/bin/zsh" sessions_dir = "~/.local/share/dosh/sessions" secret_path = "~/.config/dosh/secret" ``` Client config: `~/.config/dosh/client.toml` ```toml server = "user@example.com" update_repo = "https://git.palav.dev/Palav/dosh.git" update_port = 50000 ssh_auth_command = "~/.local/bin/dosh-auth" # ssh_port = 22 dosh_port = 50000 default_session = "new" reconnect_timeout_secs = 5 view_only = false predict = true predict_mode = "experimental" cache_attach_tickets = true credential_cache = "~/.local/share/dosh/credentials" ``` --- ## 17. Performance-First Stack Default implementation: ```text # server/client shared bytes chacha20poly1305 aes-gcm optional hmac hkdf sha2 rand serde toml # server mio or tokio # benchmark; single-thread hot path either way rustix # PTY/process/syscall wrappers where possible vt100 # authoritative terminal parser/model # client mio or tokio crossterm # raw terminal mode vt100 optional # only if client-side model is needed for prediction ``` Rules: - Benchmark `mio` vs single-thread `tokio` before committing to runtime. - Avoid locks on per-packet session broadcast. - Preallocate packet buffers. - Avoid serde on terminal frames. - Keep `dosh-auth` tiny and static where practical. - Optimize startup path before throughput. --- ## 18. MVP Scope MVP must include: - `dosh-server` daemon. - `dosh-auth` SSH helper mode. - `dosh-client`. - One UDP port. - Prewarmed `default` session. - SSH bootstrap attach. - Attach-ticket UDP attach. - Encrypted UDP channel. - UDP resume. - Raw terminal input/output. - Basic resize. - Timing instrumentation. MVP may defer: - Sophisticated predictive local echo. - Per-cell minimal diffs; raw frame plus snapshot fallback is acceptable initially if reconnect correctness is preserved. - Multi-session management commands beyond `attach`, `new`, `list`, and `kill`. --- ## 19. Verification Checklist A build is not done until these are demonstrated: - Cold attach timing compared against cold `ssh host`. - Warm attach timing compared against `ssh host true` with ControlMaster. - UDP resume completes without spawning SSH. - Existing session attach does not spawn PTY or shell. - Prewarmed `default` exists before first client. - Terminal data is encrypted on UDP. - Replay counters reject duplicate encrypted packets. - View-only clients cannot write to PTY. - Multiple clients see the same screen. - Client survives source port/IP change by resume. - Snapshot fallback repairs a lagging client. - `README.md` and `SPEC.md` remain consistent with implemented behavior. --- ## 20. Status Spec complete. The Rust implementation is present in this repository. Implemented: - Rust workspace and binaries: `dosh-server`, `dosh-client`, `dosh-auth`. - Server config and secret creation. - SSH/local bootstrap response generation. - HMAC bootstrap verification. - ChaCha20-Poly1305 encrypted UDP packets. - Fixed DOSH packet header. - Resident server with prewarmed named PTY sessions. - Authoritative server-side `vt100` terminal parser. - Full attach/resume snapshots from terminal screen state. - Per-client screen-state diffs for broadcast frames. - Raw terminal client attach. - UDP resume from cached client credentials. - Sealed attach-ticket UDP attach after server restart or unknown-client resume. - Client ACKs and server-side bounded pending retransmit window. - Sliding replay window for encrypted client packet counters. - View-only server-side input rejection. - Basic resize handling. - Timing output for bootstrap and terminal-ready. - `dosh-bench` benchmark harness for attach timing, SSH key/known-host options, and ControlMaster-backed SSH measurement. - Hardened user systemd unit. - Release install script. - Docker OpenSSH benchmark gate covering cold SSH bootstrap and ControlMaster-backed SSH bootstrap. - GitHub Actions CI for format, tests, release build, and Docker SSH benchmark gate. - Optional GitHub Actions remote benchmark job gated by repository secrets. - Auth/protocol tests. - Integration smoke tests for local attach, ticket attach after server restart, and view-only input rejection. - Integration tests for retransmit, resize, multi-client shared screen, and UDP endpoint roaming. Optional deployment evidence: - Configure `DOSH_BENCH_HOST`, `DOSH_BENCH_USER`, and `DOSH_BENCH_SSH_KEY` repository secrets to run the same benchmark against a real remote host in addition to the Docker OpenSSH gate.