303 lines
10 KiB
Markdown
303 lines
10 KiB
Markdown
# Testing & CLI Guide
|
||
|
||
This document covers the Portal TV / PortalCam security test suites and the
|
||
`portalkit-cli` tool used for pairing, control, and live MITM verification.
|
||
|
||
Related protocol design lives in [`portal-capability-test/README.md`](portal-capability-test/README.md)
|
||
(Authentication and Security Architecture) and [`mac2/README.md`](mac2/README.md).
|
||
|
||
---
|
||
|
||
## Overview
|
||
|
||
| Layer | Location | How to run | Scope |
|
||
|-------|----------|------------|-------|
|
||
| Android / JVM unit & integration | `portal-capability-test/` | `./run-tests.sh` | RFC 5054 vectors, SRP-6a math, channel binding, rate limits, padding, constant-time compare |
|
||
| Swift unit tests (`PortalKit`) | `mac2/PortalKit/` | `swift test` | Same crypto surface on the Mac client library |
|
||
| Live E2E + MITM | `tests/e2e_portal_test.py` | `python3 tests/e2e_portal_test.py` | Real Portal TV + `portalkit-cli` + rogue TLS proxy |
|
||
| CLI tool | `mac2/PortalKit` → `portalkit-cli` | see [portalkit-cli](#portalkit-cli) | Pair, status, control, MITM defense probe |
|
||
|
||
Hand-rolled SRP-6a is verified against **RFC 5054 Appendix B** test vectors on both
|
||
platforms (1024-bit group vectors from the RFC). Production pairing uses the
|
||
**RFC 5054 2048-bit** MODP group with SHA-256 hashing and TLS certificate channel
|
||
binding; those paths are covered by additional agreement / safety / MITM tests.
|
||
|
||
---
|
||
|
||
## Android / JVM test suite
|
||
|
||
### Layout
|
||
|
||
```
|
||
portal-capability-test/
|
||
├── run-tests.sh
|
||
├── src/com/portaltv/capability/
|
||
│ ├── PortalSrp.kt # server SRP-6a
|
||
│ └── PortalSrpClient.kt # client-side helpers used by tests
|
||
└── test/
|
||
├── android/util/Base64.java # JVM stub for android.util.Base64
|
||
└── com/portaltv/capability/test/
|
||
├── TestFramework.kt
|
||
├── TestRunner.kt
|
||
├── PortalSrpRfc5054Test.kt
|
||
├── PortalSrpMathTest.kt
|
||
├── PortalSrpSafetyTest.kt
|
||
├── PortalSrpRateLimitingTest.kt
|
||
├── PortalSrpChannelBindingTest.kt
|
||
├── PortalSrpPaddingTest.kt
|
||
├── PortalSrpConstantTimeTest.kt
|
||
└── PortalSrpIntegrationTest.kt
|
||
```
|
||
|
||
### Run
|
||
|
||
Requires `kotlinc` (Android Studio’s bundled Kotlin compiler is used if present)
|
||
and a JDK with `javac` / `java` on `PATH`.
|
||
|
||
```sh
|
||
cd portal-capability-test
|
||
./run-tests.sh
|
||
```
|
||
|
||
Expected summary (approximate counts):
|
||
|
||
```text
|
||
Total Suites: 8
|
||
Total Tests: 50
|
||
Passed: 50
|
||
Failed: 0
|
||
*** ALL 50 SRP-6a TESTS PASSED WITH 0 ERRORS ***
|
||
```
|
||
|
||
### What each suite covers
|
||
|
||
| Suite | File | Coverage |
|
||
|-------|------|----------|
|
||
| RFC 5054 Appendix B | `PortalSrpRfc5054Test.kt` | Official vectors for $k$, $x$, $v$, $A$, $B$, $u$, and premaster secret $S$ (client/server agreement + RFC match) |
|
||
| 2048-bit math | `PortalSrpMathTest.kt` | Group parameters ($N$, $g$, $k$), $K_{\text{client}} = K_{\text{server}}$, $M_1$/$M_2$, PIN variety, session entropy |
|
||
| Safety checks | `PortalSrpSafetyTest.kt` | Reject $A \bmod N = 0$, $B \bmod N = 0$, $u = 0$, malformed hex |
|
||
| Rate limiting | `PortalSrpRateLimitingTest.kt` | 3-attempt lifecycle ($3 \to 2 \to 1 \to 0$), session wipeout, recovery after one failure, expiry |
|
||
| Channel binding | `PortalSrpChannelBindingTest.kt` | Matching `tls_hash` succeeds; rogue / bit-flipped / empty hash rejected; forged $M_2$ rejected |
|
||
| Padding | `PortalSrpPaddingTest.kt` | `toPadded256` for zero, small, exact 256-byte, sign-byte, negative, oversized values |
|
||
| Constant-time compare | `PortalSrpConstantTimeTest.kt` | Equal / unequal / length-mismatch digests and keys |
|
||
| Protocol integration | `PortalSrpIntegrationTest.kt` | Simulated init → verify → bearer flow; MITM at HTTP layer; brute-force wipeout |
|
||
|
||
---
|
||
|
||
## Swift `PortalKit` unit tests
|
||
|
||
Pairing, TLS pinning, credential storage, and the high-level client live in the
|
||
`PortalKit` Swift package. `PortalCam.app` links this package; the CLI is the
|
||
same library’s executable target.
|
||
|
||
### Layout
|
||
|
||
```
|
||
mac2/PortalKit/
|
||
├── Package.swift
|
||
├── Sources/
|
||
│ ├── PortalKit/ # library (Crypto, TLS, Auth, Client, BigInt)
|
||
│ └── portalkit-cli/ # CLI executable
|
||
└── Tests/PortalKitTests/
|
||
├── Rfc5054VectorsTests.swift
|
||
├── Srp6aExchangeTests.swift
|
||
├── ChannelBindingTests.swift
|
||
├── SafetyChecksTests.swift
|
||
├── BigUIntPaddingTests.swift
|
||
├── ClientAndAuthTests.swift
|
||
└── SmokeTests.swift
|
||
```
|
||
|
||
### Run
|
||
|
||
```sh
|
||
cd mac2/PortalKit
|
||
swift test
|
||
```
|
||
|
||
Expected:
|
||
|
||
```text
|
||
Executed 33 tests, with 0 failures
|
||
```
|
||
|
||
### What each suite covers
|
||
|
||
| Suite | Coverage |
|
||
|-------|----------|
|
||
| `Rfc5054VectorsTests` | RFC 5054 Appendix B: $k$, $x$, $v$, $A$, $B$, $u$, client/server $S$ |
|
||
| `Srp6aExchangeTests` | 2048-bit end-to-end exchange, randomized sessions, wrong-PIN rejection |
|
||
| `ChannelBindingTests` | Matching cert hash succeeds; MITM / bit-flip / forged $M_2$ fail |
|
||
| `SafetyChecksTests` | Invalid salt, $B \bmod N = 0$, $u = 0$, empty PIN, bad $M_2$ |
|
||
| `BigUIntPaddingTests` | 256-byte padding, hex encode/decode, constant-time equality |
|
||
| `ClientAndAuthTests` | Credential storage, URL normalization, TLS challenge evaluation helpers |
|
||
| `SmokeTests` | Package / version sanity |
|
||
|
||
---
|
||
|
||
## Live E2E & MITM suite
|
||
|
||
### Prerequisites
|
||
|
||
- Portal TV reachable at `10.0.0.10:5654` with `com.portaltv.capability` running (HTTPS).
|
||
- `adb` authorized to the device (used to read the pairing PIN from logcat).
|
||
- Built CLI binary (see [Build portalkit-cli](#build-portalkit-cli)).
|
||
|
||
Default host and CLI path are configured at the top of `tests/e2e_portal_test.py`.
|
||
|
||
### Run
|
||
|
||
```sh
|
||
# from repo root
|
||
cd mac2/PortalKit && swift build -c release && cd ../..
|
||
python3 tests/e2e_portal_test.py
|
||
```
|
||
|
||
Expected:
|
||
|
||
```text
|
||
Ran 6 tests in ~13s
|
||
OK
|
||
```
|
||
|
||
### Scenarios
|
||
|
||
| # | Test | Asserts |
|
||
|---|------|---------|
|
||
| A | Status verification | `portalkit-cli status` → online; leaf cert SHA-256 present |
|
||
| B | Direct channel-binding defense | `portalkit-cli test-mitm` → server rejects; verdict `PASSED` |
|
||
| C | Live MITM TLS proxy | In-process proxy on `127.0.0.1:8888` presents a rogue cert; client binds $M_1$ to rogue hash; Portal returns **HTTP 401** |
|
||
| D | Pinned-cert hard-fail (all protected endpoints) | For each of `/control/state`, `/control/mode`, `/control/fixed`, `/control/desk`, `/control/events`, `/video.h264`, `/audio.aac`: client pinned to the genuine fingerprint must **abort at TLS** against the rogue proxy with **zero HTTP bytes** observed. Soft warnings / continued requests fail the test. Same for `portalkit-cli control` verbs (`state`, `mode`, `fixed`, `desk`). |
|
||
| E | Rate limiting | Three failed verifies → attempts $2 \to 1 \to 0$; fourth sees wiped session (`no_active_pairing`) |
|
||
| F | Legitimate pair + control | Read PIN from logcat → `pair` → pin cert → `control … mode Desk` → `{"ok":true}` → `control … state` shows Desk → restore `DefaultAuto` |
|
||
|
||
Scenario C is a real network MITM simulation (terminate TLS with a self-signed
|
||
rogue cert, forward upstream to the Portal over genuine TLS), not only an
|
||
in-memory hash swap.
|
||
|
||
---
|
||
|
||
## portalkit-cli
|
||
|
||
Command-line front end for `PortalKit`. Same pairing, pinning, and control
|
||
paths as `PortalCam.app`, without the UI or camera extension.
|
||
|
||
### Build portalkit-cli
|
||
|
||
```sh
|
||
cd mac2/PortalKit
|
||
swift build -c release
|
||
```
|
||
|
||
Binary:
|
||
|
||
```text
|
||
mac2/PortalKit/.build/release/portalkit-cli
|
||
```
|
||
|
||
Optional install:
|
||
|
||
```sh
|
||
cp .build/release/portalkit-cli /usr/local/bin/
|
||
```
|
||
|
||
Credentials are stored in the macOS Keychain when available, with a file fallback
|
||
at `~/.portalkit/credentials.json` (token + pinned cert SHA-256).
|
||
|
||
### Commands
|
||
|
||
#### `status <host>`
|
||
|
||
Probe service reachability and leaf certificate fingerprint; compare against any
|
||
stored pin.
|
||
|
||
```sh
|
||
portalkit-cli status 10.0.0.10:5654
|
||
```
|
||
|
||
#### `pair <host> [--pin <pin>]`
|
||
|
||
1. Connect over ephemeral HTTPS and capture leaf cert SHA-256.
|
||
2. `POST /auth/srp/init`.
|
||
3. Compute channel-bound $M_1$ from the PIN (prompted if `--pin` omitted).
|
||
4. `POST /auth/srp/verify`; verify $M_2$.
|
||
5. Persist bearer token + pinned fingerprint.
|
||
|
||
```sh
|
||
portalkit-cli pair 10.0.0.10:5654 --pin 123456
|
||
```
|
||
|
||
Read the live PIN from the TV (debug builds log it):
|
||
|
||
```sh
|
||
adb shell "logcat -d -s PortalService | grep 'SRP pairing started with PIN:' | tail -1"
|
||
```
|
||
|
||
#### `control <host> <command>`
|
||
|
||
Send camera control over **pinned** HTTPS with `Authorization: Bearer …`.
|
||
Mutations return `{"ok":true}`; `state` returns JSON camera state
|
||
`{"mode":"…","config":{…}}`. Errors are `{"error":"…","message":"…"}`.
|
||
|
||
```sh
|
||
portalkit-cli control 10.0.0.10:5654 state
|
||
portalkit-cli control 10.0.0.10:5654 mode Desk
|
||
portalkit-cli control 10.0.0.10:5654 mode DefaultAuto
|
||
portalkit-cli control 10.0.0.10:5654 fixed?x=0.5&y=0.5&scale=1.0
|
||
portalkit-cli control 10.0.0.10:5654 desk?tightness=0.5
|
||
```
|
||
|
||
Fails **hard** (non-zero exit, no request sent) if not paired, or if the
|
||
server cert no longer matches the pin.
|
||
|
||
#### `test-mitm <host>`
|
||
|
||
Initiate pairing, compute $M_1$ with an **altered** certificate hash, submit
|
||
verify, and expect Portal rejection (`Authentication failed (wrong PIN or MITM detected)`).
|
||
|
||
```sh
|
||
portalkit-cli test-mitm 10.0.0.10:5654
|
||
```
|
||
|
||
Successful defense prints `Verdict: PASSED`.
|
||
|
||
### Help
|
||
|
||
```sh
|
||
portalkit-cli --help
|
||
```
|
||
|
||
---
|
||
|
||
## Suggested local workflow
|
||
|
||
```sh
|
||
# 1. Crypto regressions (no device required)
|
||
cd portal-capability-test && ./run-tests.sh
|
||
cd ../mac2/PortalKit && swift test
|
||
|
||
# 2. Build CLI
|
||
swift build -c release
|
||
|
||
# 3. Deploy Portal APK (device required)
|
||
cd ../../portal-capability-test && ./deploy.sh
|
||
|
||
# 4. Live E2E including MITM proxy (device + ADB required)
|
||
cd .. && python3 tests/e2e_portal_test.py
|
||
```
|
||
|
||
---
|
||
|
||
## Notes
|
||
|
||
- **RFC vectors vs production group**: Appendix B uses the **1024-bit** RFC group
|
||
and the RFC’s hash construction for those vectors. Production and most unit
|
||
tests use the **2048-bit** group with SHA-256 and
|
||
$x = \mathrm{SHA256}(s \parallel \mathrm{PIN})$ (no identity string). Both are
|
||
intentional: vectors prove the modular arithmetic; 2048-bit tests prove the
|
||
deployed protocol.
|
||
- **Channel binding**: $M_1$ and $M_2$ include $\mathrm{SHA256}(\mathrm{leaf\_cert\_DER})$.
|
||
A TLS-terminating proxy that presents a different cert cannot complete pairing.
|
||
- **Rate limit**: three failed verifies per pairing session; session and PIN are
|
||
wiped at zero attempts remaining or after the 120s expiry window.
|