The edge system: sensing, synchronization, and on-device reasoning.
A synchronized depth + vision + audio + biosignal capture system with on-device AI compute. The full credibility pipeline runs on the device — including the Cortex reasoner — with a 3-second end-to-end delay. No network required.
Four synchronized sensors, one time base
Every stream is time-stamped to correlate every facial micro-movement with a syllable of speech and the beat frequency of the heart.
| Sensor | Purpose |
|---|---|
| ToF Depth Indirect time-of-flight | Sub-mm facial muscle movement @ 0.5 m |
| RGB Global Shutter High-resolution colour | Face, landmarks, pose, action-unit classification |
| mmWave Radar Millimetre-wave | Non-contact heart & respiration rate |
| Beamforming Mic Array Multi-element | Transcript, prosody, turn-taking |
What the device sees, hears, and measures.
Micro-movement
An indirect time-of-flight camera captures facial muscle displacement with sub-millimeter resolution at 0.5 m — the only way to pick up suppression, lid-tightening, and dimpler cues that never show on RGB alone.
Vision
A global-shutter colour sensor produces distortion-free frames for face detection, landmarking, pose estimation, and action-unit classification. Global shutter preserves geometry during fast motion.
Audio
Multi-element array with spatial beamforming isolates the subject's voice from room noise. Downstream: prosody features, turn-taking, speaker identification, and time-stamped transcript.
Heart rate & respiration
Millimetre-wave radar resolves chest-wall motion for respiration rate. Paired with rPPG from the RGB feed, the system produces contactless heart rate — no chest strap, no electrodes.
Invisible illumination
Four near-IR VCSEL emitters flood the subject with invisible infrared for ToF operation — Class-1 rated, so the subject perceives no light, no laser, no discomfort.
Synchronization
Hardware-anchored time stamps across all sensor streams. Downstream fusion can correlate a 40–200 ms action unit with a syllable and a heartbeat — frame-for-frame, no drift.
The full credibility pipeline, running on the device.
Face → landmarks → pose → AUs
Face detection, landmark localization, pose estimation for frontalization, image normalization, and an action-unit classifier covering the full set of decision-relevant facial cues.
Transcript, prosody, speaker ID
Multi-language speech-to-text, prosodic feature extraction, and speaker-embedding identification — all time-aligned with the vision chain.
Cortex reasoner, on-edge
Action units, prosody, transcript, and biosignals converge at the Cortex reasoner running locally on the on-device NPU — returning a score and a rationale every ~3 seconds.
~3 s end-to-end delay
Budgeted to preserve natural turn-taking in a live interview. The interviewer sees the score and next-question suggestion before they would have formulated their follow-up.
Zero-transmission mode
Operate fully offline — sensors, processing, inference, and the interviewer UI all run locally. Recordings and metadata are written only to the local NVMe SSD. Classified-environment ready.
Rationale attached to every score
Cortex attaches the biometric and linguistic evidence that drove each credibility score — required for 2026 AML governance and federal audit trails, exported with the session.
Learns from every session
Cortex improves in real time — no retraining cycle, no catastrophic forgetting. Model updates propagate in-place; the device you field in 2028 will outperform the one shipped in 2027.
Multi-language, multi-cohort
The speech chain supports multiple languages out of the box. Cortex adapts to new demographics and cohorts on-the-fly without a retraining cycle.
Signed + encrypted session artifacts
Every session bundle is cryptographically signed and encrypted at rest. Chain-of-custody metadata is part of the export — tampering is detectable.