# Device telemetry (what the device app reports, and where it goes) What the Endpoint device app — the downloadable macOS agent — sends once a device is enrolled, how each report is stored, and how to get it back out. **This is not the Chrome proctoring extension.** The proctoring product has its own extension, its own API and its own docs. Everything below is about the device-monitoring product only. Enrollment and credentials — QR enrollment, `POST /endpoint/register`, API keys, the `endpoint.registered` webhook — are covered by the [Partner API doc](/docs/partners/partner-api); this doc starts where that one ends, at an enrolled device holding a `device_token`. ## The shape of it ``` Enrolled device (see the Partner API doc for how it got here) → wss:///cable (ActionCable WebSocket) X-Device-Token: (preferred auth) → subscribe {"channel":"EndpointChannel",...} ← welcome (effective config: blacklist, whitelist, work_hours) → report frames: screenshot, process_list, app_change, app_violation, browser_url, motion_detected, no_motion, monitoring_status stored as endpoint_events / endpoint_violations rows + screenshot files ← violation_acknowledged / app_list_update Operator, any time → /endpoint_console (dashboard, devices, screenshots) Partner backend, any time → GET /partner/api/v1/endpoints (enrollment state) → GET /partner/api/v1/endpoints/:id/events (telemetry pull) → GET /partner/api/v1/endpoints/:id/violations (telemetry pull) → GET /partner/api/v1/endpoints/:id/events/:event_id/screenshot ``` ## 1. Connection and authentication The device opens an ActionCable WebSocket to `wss:///cable` and authenticates the handshake with a request header: - **`X-Device-Token`** (preferred) — the `device_token` minted by `POST /endpoint/register` or `POST /endpoint/login`. An unknown token is rejected. - **`X-User-Id`** (legacy) — the scanned QR enrollment token, which the shipped macOS 1.6.0 client presents instead of the device token. It is accepted only when it matches an enrolled, non-disabled device; every acceptance writes a `[DEPRECATION]` log line, and the path will be removed once client 1.7 sends `X-Device-Token` (#166). | Handshake | Result | |---|---| | `X-Device-Token` matching a device | accepted; device marked online | | `X-Device-Token` matching nothing | rejected | | `X-User-Id` matching an enrolled, non-disabled device | accepted, deprecation-logged | | `X-User-Id` matching nothing, or a disabled device | rejected | | neither header | rejected | A device is marked `online` at connect and `offline` at disconnect. ## 2. The channel protocol Subscribe with the identifier `{"channel":"EndpointChannel","user_id":""}`. Identity comes from the authenticated connection, never from the identifier — the `user_id` field is carried for wire compatibility with the shipped client. On subscribe the server transmits a `welcome` config frame — the device's effective configuration, org settings merged under per-device overrides (device values win): ```json { "type": "welcome", "message": "Connected to Endpoint channel", "blacklist": ["Discord", "Slack"], "whitelist": [], "work_hours": { "enabled": true, "start_time": "9:00 AM", "end_time": "5:00 PM", "work_days": [1, 2, 3, 4, 5] } } ``` ### Report framing Reports are ActionCable `message` commands whose `data` field is a **JSON-encoded string** (not an object) carrying `{"action":"report","data":{...}}`: ```json { "command": "message", "identifier": "{\"channel\":\"EndpointChannel\",\"user_id\":\"\"}", "data": "{\"action\":\"report\",\"data\":{\"type\":\"app_change\",\"from_app\":\"Safari\",\"to_app\":\"Slack\",\"timestamp\":\"2026-08-28T12:00:00Z\"}}" } ``` The inner `data` object's `type` selects the telemetry type (section 3). ### Device → server: update_work_hours The device may also send `{"action":"update_work_hours","work_hours":{...}}` (same string-encoded framing). Only the keys `enabled`, `start_time`, `end_time` and `work_days` are accepted; they merge into the device's per-device override, which then wins over org settings in `effective_config`. ### Server → device frames | Frame | Payload | When | |---|---|---| | `violation_acknowledged` | `{"type":"violation_acknowledged","violation_id":42,"timestamp":"..."}` | echoed to the reporting device after each stored `app_violation` | | `app_list_update` | `{"type":"app_list_update","blacklist":[...],"whitelist":[...]}` | defined, but nothing in the server triggers it today — config changes reach a device via the `welcome` frame at its next connect | ## 3. Telemetry catalogue | Type | Cadence | Payload fields | Stored as | |---|---|---|---| | `screenshot` | every 600 s while monitoring | `timestamp`, `image_data`, `display_width`, `display_height` | `endpoint_events` row + bytes in `endpoint_screenshots` | | `process_list` | every 600 s | `timestamp`, `processes[]`, `total_count` | `endpoint_events` row | | `app_change` | on focus change | `from_app`, `to_app`, `timestamp` | `endpoint_events` row | | `app_violation` | on a blacklisted app | `app_name`, `bundle_id`, `timestamp` | `endpoint_violations` row | | `browser_url` | on browser URL change | `url`, `browser`, `timestamp` | `endpoint_events` row | | `motion_detected` | 60 s check cycle | `timestamp` | **no row** — updates last-activity only | | `no_motion` | after >900 s without motion | `duration`, `timestamp` | `endpoint_events` row | | `monitoring_status` | on start/stop | `status`, `timestamp` | device status flip + `endpoint_events` row | Unknown types are logged server-side and dropped without error. Every report, of any type, refreshes the device's `last_seen_at`. ### screenshot Sent every 600 seconds while monitoring is active. `image_data` is a base64 JPEG of the main display at 0.5 compression quality. Capture (both screenshots and the motion check below) pauses while a video-conferencing app is running and resumes when it exits. ```json { "type": "screenshot", "timestamp": "2026-08-28T12:00:00Z", "image_data": "", "display_width": 2560, "display_height": 1440 } ``` Stored as an `endpoint_events` row whose `event_data` holds the display dimensions and `file_size_bytes`, plus an `endpoint_screenshots` row holding the decoded JPEG bytes themselves. The bytes go to the primary database — not to the ingesting pod's local disk — so any web pod can serve them (#196; the app runs multiple dynos with separate ephemeral filesystems). A frame whose decoded payload exceeds 10 MB, or does not decode at all, is dropped with a server-side warning and no rows. ### process_list Sent every 600 seconds: every running application with a localized name. ```json { "type": "process_list", "timestamp": "2026-08-28T12:00:00Z", "processes": [ { "name": "Safari", "bundle_id": "com.apple.Safari", "pid": 4242, "is_active": true, "is_hidden": false, "is_terminated": false } ], "total_count": 1 } ``` ### app_change Sent when application focus changes (observed via workspace notifications, with a 30-second fallback poll). ```json { "type": "app_change", "from_app": "Safari", "to_app": "Slack", "timestamp": "2026-08-28T12:00:00Z" } ``` ### app_violation Sent when a blacklisted app becomes active. Stored as an `endpoint_violations` row (severity `medium`); the server echoes `violation_acknowledged` back to the device. ```json { "type": "app_violation", "app_name": "Discord", "bundle_id": "com.hnc.Discord", "timestamp": "2026-08-28T12:00:00Z" } ``` ### browser_url Sent when the active browser's URL changes. ```json { "type": "browser_url", "url": "https://example.com/page", "browser": "Safari", "timestamp": "2026-08-28T12:00:00Z" } ``` ### motion_detected The client hashes a screen capture every 60 seconds; a changed hash means motion. Updates the device's last-activity timestamp only — **no row is stored**. ```json { "type": "motion_detected", "timestamp": "2026-08-28T12:00:00Z" } ``` ### no_motion Sent when the screen has not changed for more than 900 seconds. `duration` is the elapsed seconds, as an integer. ```json { "type": "no_motion", "duration": 947, "timestamp": "2026-08-28T12:00:00Z" } ``` ### monitoring_status Sent when the user starts or stops monitoring. `"active"` flips the device `online`, anything else flips it `offline` (a `disabled` device is never touched); an `endpoint_events` row is stored either way. ```json { "type": "monitoring_status", "status": "active", "timestamp": "2026-08-28T12:00:00Z" } ``` ## 4. Storage and retention Three tables plus a file store: | Where | What | |---|---| | `endpoint_devices` | one row per device: hostname, serial_number, os_version, app_version, `status` (`registered` \| `online` \| `offline` \| `disabled`), `last_seen_at`, `last_ip`, per-device config overrides. `serial_number` is the hardware serial (IOKit `IOPlatformSerialNumber`) reported by client 1.7+ at registration/login; it is null for devices registered by client <= 1.6 until they re-register with 1.7, and a serial-less payload never clears a previously captured value. `last_ip` is the source IP of the most recent registration/login; the partner API derives an approximate `location` from it at read time via a local GeoIP lookup (country-level with the default DB-IP database, city-level if a City MMDB is configured through `GEOIP_DB_PATH`; null for private IPs or when unresolvable — see the [Partner API doc](/docs/partners/partner-api)). IP Geolocation by [DB-IP](https://db-ip.com) | | `endpoint_events` | one row per stored report (`screenshot`, `process_list`, `app_change`, `browser_url`, `no_motion`, `monitoring_status`) | | `endpoint_violations` | one row per `app_violation` | | `endpoint_screenshots` | one row per screenshot event: the JPEG bytes, `content_type`, `byte_size`. Bytes live in the primary database so every web pod can serve them; nothing is written to pod-local disk (#196) | If the org configures S3 in the console, `Endpoint::UploadScreenshotJob` moves each screenshot's bytes to that bucket under `endpoint/screenshots//` and then destroys the database blob — the only deletion anywhere in the pipeline. The console still serves offloaded screenshots by reading them back from that bucket. Retention is currently indefinite; this is not a contractual guarantee — a retention policy is tracked in #167. Until it lands, note the growth math: a monitored device produces one screenshot every 600 s at roughly 500 KB, so about **500 KB of database growth per device per 10 minutes** while monitoring is active. Orgs that configure S3 offload move that growth into their own bucket almost immediately; without offload it accrues in the primary database until #167's policy prunes it. Screenshots recorded before the database migration (#196) were files on one web pod's ephemeral disk. Any such file that survived is still served while it exists; once it is gone the event reports `available: false` and its bytes are unrecoverable. ## 5. Getting it back out The Endpoint console at `/endpoint_console` (session login) is the primary surface: - dashboard — device counts and recent events - device list, and a per-device page with recent events, violations and a screenshot gallery (`GET /endpoint_console/devices/:id/screenshot/:event_id` serves the image bytes, from the database or from the org's S3 bucket after offload) - disable / enable a device - org settings — blacklist, whitelist, work hours - QR enrollment The programmatic surface is the Partner API: `GET /partner/api/v1/endpoints` plus the `endpoint.registered` webhook cover enrollment state — including each device's `last_ip` and the approximate IP-derived `location`, which live on the device row there, not on individual events (documented in the [Partner API doc](/docs/partners/partner-api)) — and the telemetry pull routes below cover everything a device has reported (issue #165). All of them take the same `Authorization: Bearer ` header, share the same per-key 60 requests/minute throttle, and are structurally scoped: another partner's device or event answers a 404 `{"error": {...}}` envelope whatever parameters arrive. ### Events GET /partner/api/v1/endpoints/:id/events Authorization: Bearer Filters and pagination, all optional query parameters: | Parameter | Meaning | |---|---| | `event_type` | one of the telemetry catalogue's stored types (`screenshot`, `process_list`, `app_change`, `browser_url`, `no_motion`, `monitoring_status`) | | `occurred_from` / `occurred_to` | ISO8601 bounds on `occurred_at`, each side independent; a malformed timestamp is a 400 `invalid_parameter` envelope | | `page` / `per_page` | page number (default 1) and page size (default 100, max 500) | Events come in ascending id order — stable across requests, so pages never shuffle. `total_count` is the filtered total; keep fetching pages until `page * per_page >= total_count`. { "events": [ { "id": 91, "event_type": "screenshot", "occurred_at": "2026-08-28T12:00:00Z", "data": {"display_width": 2560, "display_height": 1440, "file_size_bytes": 48213}, "screenshot": { "available": true, "url": "/partner/api/v1/endpoints/42/events/91/screenshot" } }, { "id": 92, "event_type": "app_change", "occurred_at": "2026-08-28T12:00:30Z", "data": {"from_app": "Safari", "to_app": "Slack"} } ], "page": 1, "per_page": 100, "total_count": 2 } `data` is the stored `event_data` for the type, per the telemetry catalogue above. Screenshot events additionally carry a `screenshot` availability object — never a server filesystem path: | `screenshot` | Meaning | |---|---| | `{"available": true, "url": "/partner/api/v1/..."}` | the bytes are in the server's database; GET that path (with your API key) for them | | `{"available": true, "url": "s3://bucket/key"}` | offloaded to your configured S3 bucket — the bytes are already yours; read them there | | `{"available": false}` | no bytes anywhere (never captured, or a pre-#196 pod-local file is gone) | `available: true` with an API url is a serving guarantee, not a guess: the list applies exactly the checks the bytes route applies, so a pull of a listed-available screenshot cannot 404 (#196's secondary defect). ### Violations GET /partner/api/v1/endpoints/:id/violations Authorization: Bearer Same pagination contract (`page`, `per_page`, `total_count`, ascending id order). One row per `app_violation` report: { "violations": [ { "id": 7, "violation_type": "blacklisted_app", "app_name": "Discord", "bundle_id": "com.hnc.Discord", "severity": "medium", "occurred_at": "2026-08-28T12:00:00Z", "acknowledged": true, "acknowledged_at": "2026-08-28T12:00:01Z", "reviewed": false, "recorded_at": "2026-08-28T12:00:01Z" } ], "page": 1, "per_page": 100, "total_count": 1 } ### Screenshot bytes GET /partner/api/v1/endpoints/:id/events/:event_id/screenshot Authorization: Bearer | Status | Meaning | |---|---| | 200 | the JPEG bytes (`Content-Type: image/jpeg`), served from the database until offload | | 404 | `screenshot_not_found` envelope — the event has no screenshot bytes anywhere (routing/scoping misses are a `not_found` envelope) | | 410 | `screenshot_offloaded` envelope — the file was uploaded to your configured S3 bucket and deleted locally; the `s3://` URL is in the message and in `error.details.url`. The bytes are already in your bucket — read them there, this route will not proxy them | The 410 is deliberate: once `Endpoint::UploadScreenshotJob` has offloaded a screenshot to the partner's own bucket, the durable copy is the partner's, and re-serving it through this API would just proxy their own storage back at them. Screenshot bytes that have not been offloaded live in the primary database and survive restarts and are served from every web pod — the pre-#196 failure mode, where the bytes existed only on the one pod that held the device's WebSocket, is gone. Growth is bounded by the retention policy tracked in #167 (about 500 KB per device per 10 minutes of monitoring until then); configure S3 offload to own the durable copies immediately. One honest limitation: devices enrolled through the Partner API currently deliver **enrollment state only**. The shipped 1.6.0 client authenticates the WebSocket with its enrollment token as `X-User-Id`, and partner-owned device rows deliberately carry no such value — so their connections are rejected, no telemetry is stored, and their `status` stays `registered` (it never flips `online`/`offline`). This resolves when client 1.7 authenticates with `X-Device-Token` (#166). ## Traps - **The report frame is nested.** The channel reads `data.data` from the decoded frame; a flat `{"action":"report","type":"screenshot",...}` frame is silently ignored — no error, no row. - **`motion_detected` stores no row.** It only refreshes the device's last-activity timestamp; don't search `endpoint_events` for it. Absence of motion is what gets a row (`no_motion`). - **Offloaded screenshots disappear from the database by design.** After the S3 upload succeeds the `endpoint_screenshots` blob is destroyed; the `endpoint_events` row keeps the S3 location and the console reads the bytes back from the bucket. - **Unknown event types are dropped without error.** A typo'd `type` is logged server-side and discarded; the device gets no failure signal. - **The Linux client delivers none of this today.** It posts to an `/api/*` surface this server does not implement, so the telemetry above comes from the macOS app only. _Last verified: 2026-08-28 against main._ ## macOS security evidence and response macOS security reports are separate from activity capture and are collected outside work hours. `POST /endpoint/security_reports` uses the enrolled device's `X-Device-Token` header. The same report can be sent through the existing cable `report` action with `data.type = "security_report"`. Disabled devices and revoked owners cannot submit security evidence. Security observations are agent-reported, not hardware attestation or proof that a policy was enforced. Schema version 1 has a UUID `report_id`, UTC `observed_at`, `platform: "macos"`, agent/OS versions, per-check `posture` state/source/detail, installed `applications`, periodic `processes`, truncation indicators and explicit `capabilities`. Optional `malware_scan` describes the managed sample inbox or locally approved folders, with opaque scope labels, coverage limits, engine/signature versions, a stable scan observation timestamp (preserved between scheduled scans), definition freshness, file/error counts and SHA-256 detection evidence. Optional `endpoint_security` describes the separately entitled sensor's runtime state, receipt-timestamped process/persistence notifications and loss counters. Its informational persistence rules identify LaunchAgent/LaunchDaemon creation or replacement; they do not classify all such activity as malware or block it. Reports are at most 512 KiB, 1000 installed applications, 2000 processes, 100 scan detections and 128 sensor events. Collection excludes process arguments, environment variables and user paths. The server discards fields outside this contract and filters report bodies from application parameter logs. A shared, device-locked quota permits 24 new reports per hour across HTTP and cable. Repeating the same UUID and evidence is idempotent; reusing a UUID with changed evidence fails. The console and the owning partner's `/partner/api/v1/endpoints` response show the latest observed report. Organization CSV export is `GET /endpoint_console/devices/security_export`. Unknown, unavailable, stale and policy-changed evidence never receives a passing posture verdict. CSV values are protected against spreadsheet formula evaluation. Organization administrators can append a versioned security evidence policy with a minimum numeric macOS version, required MDM observation and a 15–1440 minute freshness window. Policy changes retain the requester's identity and time. Reports pin the evaluated policy version, and a new policy requires fresh device evidence. The default freshness window is 120 minutes. Policy evaluation does not install OS updates, enable FileVault or enroll a device into MDM. Administrators can request `refresh_security_report`, `scan_managed_samples` or `scan_approved_folders`. The last action is available only when a recent device report advertises locally approved folders; the server cannot select or change those folders. The agent polls `GET /endpoint/security_commands`, receives a UUID/action/expiry, then uses `PATCH /endpoint/security_commands/:id` to acknowledge `completed` or `failed`, with a short enum `detail` and a stored `report_id`. Commands expire in 15 minutes. Completion requires fresh evidence from that device; successful scan completion also requires the requested scope, at least one scanned file, no errors, no skipped files, complete reported scope and fresh definitions. Repeated identical results are accepted, but terminal results cannot be changed. No remote path, shell command, isolation, erase or arbitrary quarantine action is accepted. With separate, enrollment-bound local consent, administrators can also request `quarantine_managed_sample` with `{source_report_id, sha256}` or `restore_managed_sample` with `{quarantine_id, sha256}`. References are opaque UUIDs and SHA-256 values; no remote path is accepted. Quarantine requires a recent, complete, fresh-definition managed-inbox detection. The agent rescans and verifies the single eligible regular file before moving it. Restore requires an existing confirmed quarantine for the same device and refuses destination collisions. These operations never target locally approved general scan folders. The shared authorizer permits four response requests per rolling hour and 24 per day under the device lock, including requests that subsequently fail or expire. A report may contain at most 20 `response_receipts`, each binding command, action, source report (quarantine), SHA-256, quarantine UUID, terminal status, short detail, observation time and `reconciled` boolean. Both successful and failed acknowledgments require a matching persisted receipt. Result audit rows are immutable. Successful PATCH responses include the command `id` and terminal `status`; legacy command envelopes omit response `arguments` entirely. GET returns at most ten commands, prioritizing unexpired requests, then pending managed-response requests expired within 30 days for journal reconciliation. An expired request cannot initiate a new file mutation. A late completion requires an original receipt observed before expiry, or an explicit `reconciled: true` receipt recording when the agent verified an already executed action. This is retrospective evidence, not a fabricated execution timestamp. Local journal I/O uncertainty leaves the request pending and reports `managed_response` posture `error` with `response_reconciliation_required`. Security report bodies, including inventory and sensor snapshots, are stored in Endpoint's database for 30 days and purged hourly. This is durable storage, not a pass-through promise. Policy versions and response request/result audit rows remain until the owning device/organization is deleted; they contain identifiers and status, not raw file contents or recovery keys. Existing activity/screenshot retention is separate and is still tracked by issue #167.