Evaluation partitions are isolated by actor where identity metadata supports it.
Technology · System architecture
Multimodal behavioral dynamics, engineered as a real system.
NDMS combines synchronized facial and vocal representations with quality-aware fusion, cross-modal discordance modeling, uncertainty estimation and longitudinal context. The goal is not to infer hidden truth from a face or voice; it is to build a calibrated, user-controlled measurement stack whose limits remain visible. [R1–R5]
System mapRuntime + research boundary
01CaptureFace + voice→
02EncodeTemporal features→
03GateSignal quality→
04FuseCross-modal state→
05CalibrateUncertainty + OOD→
06InterpretUser-facing signals
RuntimeCamera · microphone · edge inference
Research layerDataset validation · EEG teacher · stress tests
01
From signals to context—not from signals to certainty.
Audio and facial behavior are complementary but imperfect measurements. Public audiovisual affective-computing research shows why multimodal systems need temporal alignment, fusion logic and explicit handling of uncertainty rather than simple score averaging. [R1] [R2] [R3]
NDMS extends that engineering philosophy toward a longitudinal masking-support system. The current implementation work is focused on a robust audiovisual foundation first; masking-specific prospective validation remains a separate scientific milestone.
System map
A semi-technical architecture with explicit assurance layers
The public architecture intentionally shows system logic without publishing implementation details that could compromise unpublished IP.
InputFacial videoVoice audioSynchronized windows
→
RepresentationVisual encoderAcoustic encoderShort + long temporal context
→
CoreQuality estimationDynamic gatingFusion + discordance
→
AssuranceCalibrationUncertaintyOOD checks
→
OutputInterpretable indicatorsLongitudinal profileHuman context required
Training path
Evidence-building before deployment
01Actor-aware partitioning
02Multimodal representation learning
Visual and acoustic streams learn complementary temporal features before fusion.
03Quality + uncertainty objectives
Signal reliability, calibration and interval coverage are treated as measurable model properties.
04Ablation + stress testing
Components are removed or degraded deliberately to test whether gains survive controlled failure modes.
05Teacher-student roadmap
EEG remains a research-side physiological teacher/validator pathway, not a required consumer input.
Inference path
A deployable path built around quality
01Synchronized capture
Standard camera and microphone streams are windowed onto a shared temporal axis.
02Local feature extraction
Runtime design favors processing useful representations near the user rather than moving raw media by default.
03Quality-aware fusion
Noisy channels are down-weighted instead of being treated as equally trustworthy.
04Calibration + OOD
Confidence, uncertainty and distribution shift are surfaced before downstream interpretation.
05Longitudinal context
Outputs are designed to be interpreted relative to the same user over time, not as a population label.
Privacy architecture
Edge-first by design.
NDMS is being engineered so raw camera and microphone streams do not need to leave the local device for routine inference. The platform direction favors derived features, explicit consent, minimal retention, user-controlled sessions and visible uncertainty.
Runtime inputsCamera + microphone
Required wearableNone
EEG at runtimeNo
DeviceCapture + feature extraction
↓ModelQuality-aware multimodal inference
↓User layerInterpretable output + context
Raw-media cloud transfer is not a default requirement.Prototype
Productizing the technical stack
The prototype is being built around the same principle as the research stack: signal quality and uncertainty are visible first-class objects, not hidden implementation details.
Cross-modal stateDiscordance monitored
Confidence and signal quality remain visible alongside any model output.
Video quality 0.94Audio quality 0.89OOD status in-distribution
Illustrative interface view. Product UI is in active development; displayed values are representative, not a clinical reading.
02
Research architecture with a commercial path.
NDMS is being structured so the same core engine can support multiple consent-based product surfaces without changing its safety boundaries.
Self-directed mobile experience
Private sessions, longitudinal personal baselines and interpretable self-regulation feedback.
Validation + instrumentation
Structured study outputs, reproducible metrics and multimodal experiment tooling.
API / SDK pathway
Future integration layer for approved partners under strict use and consent policies.
Regulated path remains separate
Any future medical use would require dedicated evidence, regulatory strategy and human oversight.
Patent + implementation
PCT/IB2025/060348 establishes the disclosed IP foundation.
The public site separates what is disclosed as an international patent application from what is currently implemented, experimentally tested, or planned. Filing status is not presented as patent grant, product validation or regulatory approval.
Selected references
Research and IP context
These references provide context for audiovisual emotion datasets, multimodal fusion, uncertainty-aware modeling, calibration and OOD evaluation. They do not constitute independent validation of NDMS.
R1
CREMA-D: Crowd-Sourced Emotional Multimodal Actors DatasetCao H. et al. · IEEE Transactions on Affective Computing · 2014
R2AuxFormer: Robust Approach to Audiovisual Emotion RecognitionGoncalves L., Busso C. · ICASSP · 2022
R3COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion RecognitionTellamekala M.K. et al. · 2022
R4On Calibration of Modern Neural NetworksGuo C. et al. · ICML · 2017
R5A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural NetworksHendrycks D., Gimpel K. · ICLR · 2017
P1Neuro Dynamic Masking System — International Patent ApplicationPCT/IB2025/060348 · Applicant/inventor disclosure
Build with us
Research, product and validation partnerships.
NDMS is entering the stage where system engineering, dataset design and prospective validation need to move together.