Access Required
This report contains proprietary research. Enter the password to continue.
Incorrect password. Try again.
This report contains proprietary research. Enter the password to continue.
Enter the download password to export the full report as Markdown.
Complete catalog of 346 health markers: 120 core rPPG + facial analysis markers, 34 expansion module markers across 6 scan types, 72 frequency-based markers, plus 120 coverage gap closure markers across 17 new sections including tongue, nail, breath, body metrics, and mood scans. Technical implementation specs included.
QRMA Finger Scanner vs VHScanner Atlas| Field | Status | Applicable Scans | Why It's Needed |
|---|---|---|---|
| Date of Birth / Age | Required | All scans | 60-70% of markers need age — BP, HRV, SpO2, skin age, pupil norms, voice baseline, all risk scores use age-normalized ranges |
| Biological Sex | Required | All scans | BP, HR, HRV, hemoglobin thresholds, voice pitch baselines, cardiac output, skin thickness all have sex-specific norms |
| Height | Required | 👤 Face ☝ Fingertip | BP estimation from pulse transit time, PWV calculation, BSA for cardiac index — not needed for Pupil, Voice, Palm, or Motion scans |
| Weight | Required | 👤 Face ☝ Fingertip | BMI, body surface area, cardiac index, metabolic markers — not needed for Pupil, Voice, Palm, or Motion scans |
| Fitzpatrick Skin Type (I-VI) | Auto-Detect | 👤 Face 🔍 Skin 🌈 Spectral ✋ Palm | Auto-detected via CIE-Lab L* analysis. Critical for rPPG channel weighting, SpO2 calibration, and spectral analysis. Not needed for Pupil, Fingertip, Voice, or Motion scans. |
| Known Medical Conditions | Recommended | All scans | Hypertension, diabetes, anemia, Parkinson's, concussion history, depression — context for markers across all scan types |
| Current Medications | Recommended | All scans | Beta blockers: HR/HRV. Pupil-dilating drops: Pupil AI Scan. Opioids: constrict pupils (pain index confound). SSRIs: voice prosody. Stimulants: pupil dilation. |
| Fitness Level | Recommended | 👤 Face ☝ Fingertip | Athletes have resting HR 40-55 bpm and 2-3x higher HRV — without context these flag as bradycardia |
| Pregnancy Status | Recommended | 👤 Face ☝ Fingertip 🌈 Spectral ✋ Palm | HR +10-20 bpm, BP changes per trimester, physiological anemia, palmar erythema is normal in pregnancy |
| Recent Head Injury | Recommended | ㈰ Pupil | Critical for concussion screening (#127). Records time since injury for longitudinal tracking. Flags urgency level. |
| Known Neurological Conditions | Recommended | ㈰ Pupil 🎤 Voice 🎬 Motion | Parkinson's, essential tremor, MS, prior TBI — provides baseline context for tremor detection, voice biomarkers, and pupillometry |
| Pain Level (current, 0-10) | Optional | ㈰ Pupil | Self-reported pain scale anchors objective pupil-based pain index (#128). Comparison validates the measurement. |
| Smoking Status | Optional | 👤 Face 🔍 Skin 🌈 Spectral | COHb falsely elevates SpO2, accelerates skin aging, increases arterial stiffness |
| Time of Last Meal | Optional | 👤 Face ☝ Fingertip | Postprandial HR +10-15 bpm, BP drops 10-20 mmHg. Also affects fingertip glucose trend (#133) |
| Caffeine / Alcohol (last 2hr) | Optional | 👤 Face ☝ Fingertip ㈰ Pupil | Caffeine: HR +5-10, BP +5-15, constricts pupils. Alcohol: vasodilation, flushing, dilates pupils. |
| Current Emotional State | Optional | 👤 Face ㈰ Pupil 🎤 Voice | Stress anchoring for pupil dilation, voice prosody analysis, and HRV stress markers. Validates algorithmic detection. |
| Eye Drops (last 4hr) | Optional | ㈰ Pupil 👁 Eye | Dilating drops (tropicamide, atropine) invalidate pupillometry. Redness-reducing drops mask scleral analysis. Must note type. |
FREQUENCY ANALYSIS MODULE — TECHNICAL IMPLEMENTATION
═══════════════════════════════════════════════════════
HARDWARE REQUIREMENTS
═══════════════════════════════════════════════════════
1. Bone Conduction Transducer (Bidirectional)
- Input: reads micro-vibration responses from bone/tissue
- Output: sends corrective/reference frequencies
- Frequency response: 1 Hz - 10 MHz (minimum)
- Placement: mastoid process (behind ears)
- Options: custom piezoelectric transducer, or modified Shokz/AfterShokz headset
- Connection: Bluetooth 5.0+ or USB-C wired (lower latency preferred)
2. Optional: Electrodermal Sensors
- For meridian endpoint measurements (Ryodoraku/EAV style)
- Measures skin conductance at acupuncture points
- Connection: Bluetooth or wired to phone/tablet
═══════════════════════════════════════════════════════
FREQUENCY DATABASE (Blueprint)
═══════════════════════════════════════════════════════
Build a reference database of healthy frequency ranges:
interface FrequencyBlueprint {
id: string; // e.g. "organ_heart", "nutrient_iron"
category: "organ" | "nutrient" | "toxin" | "meridian" | "chakra" | "system" | "blood";
name: string; // human-readable name
frequency_hz: number; // primary resonant frequency
frequency_range: [number, number]; // healthy range (low, high)
harmonic_frequencies: number[]; // related harmonics
deviation_thresholds: { // for 1-9 scoring
minimal: number; // 1-2
moderate: number; // 3-5
significant: number; // 6-7
severe: number; // 8-9
};
}
Data sources:
- Published bioresonance frequency tables (Rife, Clark, Voll)
- Peer-reviewed bioelectrical impedance studies
- EEG/EMG/ECG reference ranges for brainwave/muscle/heart
- TCM electrodermal measurement databases (Ryodoraku)
- Chakra frequency mappings from traditional texts
═══════════════════════════════════════════════════════
SCANNING PIPELINE
═══════════════════════════════════════════════════════
Phase 1: Calibration (5 seconds)
1. Send known reference tone through headset
2. Record bone conduction response
3. Calculate transfer function (compensate for individual skull density)
4. Verify headset placement quality
Phase 2: Frequency Sweep (30-180 seconds)
1. Sweep through Blueprint frequencies sequentially
2. At each frequency:
a. Send reference sine wave at target frequency
b. Record response amplitude and phase shift
c. Compare to Blueprint healthy baseline
d. Calculate deviation score (1-9)
3. Sweep order: low→high for organs, specific bands for meridians/chakras
4. Total: 550+ frequencies in 3-6 minutes
Phase 3: Analysis (< 2 seconds post-scan)
1. For each scanned frequency:
- deviation = abs(measured_response - blueprint_baseline)
- score = map_to_1_9_scale(deviation, thresholds)
- direction = measured > baseline ? "over-active" : "under-active"
2. Aggregate into system-level scores
3. Generate before-scan snapshot
Phase 4: Optimization (30-60 seconds, optional)
1. For each marker with score > 4:
- Generate corrective frequency = blueprint_baseline - measured_deviation
- Send through headset for 3-5 seconds per marker
2. Re-scan to generate after-optimization snapshot
3. Compare before/after for session report
═══════════════════════════════════════════════════════
BINAURAL BEAT GENERATION (Brain Entrainment)
═══════════════════════════════════════════════════════
function generateBinauralBeat(targetHz, baseHz = 200) {
// Left channel: pure tone at baseHz
// Right channel: pure tone at baseHz + targetHz
// Brain perceives the difference as the beat frequency
// Target ranges:
// Delta: 0.5-4 Hz (deep sleep, healing)
// Theta: 4-8 Hz (meditation, creativity)
// Alpha: 8-13 Hz (relaxation, calm focus)
// Beta: 13-30 Hz (alertness, concentration)
// Gamma: 30-100 Hz (peak cognition, insight)
leftChannel = sineWave(baseHz);
rightChannel = sineWave(baseHz + targetHz);
return stereoAudio(leftChannel, rightChannel);
}
// REQUIRES stereo headphones — mono output will NOT produce entrainment
// Session duration: 5-30 minutes for measurable effect
// Measurement: track how quickly EEG alpha/theta/etc power increases
═══════════════════════════════════════════════════════
FILE STRUCTURE
═══════════════════════════════════════════════════════
src/expansion/
├── frequency-scan/
│ ├── transducer-interface.ts # Bluetooth/USB headset communication
│ ├── frequency-sweep.ts # Sequential frequency sweep engine
│ ├── blueprint-database.ts # Healthy frequency reference data
│ ├── deviation-scoring.ts # 1-9 scale mapping
│ ├── optimization-engine.ts # Corrective frequency generation
│ └── frequency-markers.ts # Markers #155-194, #214-221
├── meridian-scan/
│ ├── meridian-database.ts # 12 meridian frequency profiles
│ ├── electrodermal-interface.ts # Optional EAV sensor input
│ └── meridian-markers.ts # Markers #195-206
├── chakra-scan/
│ ├── chakra-database.ts # 7 chakra frequency profiles
│ └── chakra-markers.ts # Markers #207-213
└── brain-entrainment/
├── binaural-generator.ts # Stereo binaural beat synthesis
├── entrainment-measure.ts # Brainwave sync readiness scoring
└── brain-markers.ts # Markers #222-226
═══════════════════════════════════════════════════════
DISCLAIMERS (MUST DISPLAY)
═══════════════════════════════════════════════════════
- Frequency scanning measures electromagnetic frequency deviations, NOT direct physical biomarkers
- Not FDA cleared — emerging bioresonance technology
- NOT a substitute for blood tests, imaging, or medical diagnosis
- Meridian and chakra assessments are based on traditional medicine systems without full scientific consensus
- Brain entrainment effectiveness varies by individual
- All frequency-based markers should be clearly labeled as "Frequency Analysis" tier
- Corrective frequency optimization is emerging — do not make medical claims
FITZPATRICK AUTO-DETECTION (runs during calibration phase, frame 1-10)
Pipeline:
1. Detect face ROI (forehead + cheeks) from first stable frame
2. Extract mean pixel values from ROI in CIE-Lab color space
3. Use L* (lightness) channel to classify:
L* 75-95 → Fitzpatrick I-II (Very light / Light)
L* 60-75 → Fitzpatrick III (Medium)
L* 45-60 → Fitzpatrick IV (Olive / Moderate brown)
L* 30-45 → Fitzpatrick V (Dark brown)
L* 15-30 → Fitzpatrick VI (Very dark)
4. Apply correction based on ambient lighting:
- Measure highlight intensity on forehead specular reflection
- Normalize L* by ambient brightness estimate
5. Store detected type in user_profile.fitzpatrick (int 1-6)
6. Show user: "We detected your skin type as [X]. Is this correct?" with override option
Impact on rPPG pipeline:
- Fitzpatrick I-III: weight green channel 60%, red 25%, blue 15%
- Fitzpatrick IV: weight green 45%, red 35%, blue 20%
- Fitzpatrick V-VI: weight red 50%, green 30%, blue 20% (or use NIR if available)
- SpO2 calibration curve shifts by ~0.5-1.5% per Fitzpatrick level
- Hemoglobin/bilirubin DC baseline correction applied per skin type
Libraries: OpenCV cv2.cvtColor(frame, cv2.COLOR_BGR2Lab), numpy for ROI averaging
Timing: <50ms per frame, runs once during calibrationPRE-SCAN FORM SCHEMA
All fields stored in user_profile object. Persists across sessions.
Form shown on first scan, then editable from profile settings.
─────────────────────────────────────────────────
REQUIRED FIELDS (block scan if missing)
─────────────────────────────────────────────────
1. date_of_birth
Type: date (ISO 8601: YYYY-MM-DD)
UI: Date picker with year/month/day dropdowns
Validation: Must be in past, age 1-120
Derived: user_profile.age = floor((today - dob) / 365.25)
Used by: ALL markers for age-normalized reference ranges
2. biological_sex
Type: enum
Options: ["male", "female"]
UI: Two-button toggle (not dropdown)
Note: This is biological sex for physiological calibration, not gender identity.
If needed for UX sensitivity, label as "Sex assigned at birth"
Used by: BP, HR, HRV norms, hemoglobin thresholds, cardiac output,
skin thickness baselines, CVD risk scoring
3. height_cm
Type: float
Range: 50-250 cm
UI: Number input with unit toggle (cm / ft+in)
Conversion: if ft/in input → cm = (feet * 30.48) + (inches * 2.54)
Store as: centimeters (float, 1 decimal)
Used by: PWV = height / PTT, BSA calculation, BMI
4. weight_kg
Type: float
Range: 10-300 kg
UI: Number input with unit toggle (kg / lbs)
Conversion: if lbs → kg = lbs * 0.453592
Store as: kilograms (float, 1 decimal)
Derived: user_profile.bmi = weight_kg / (height_cm/100)^2
Derived: user_profile.bsa = 0.007184 * height_cm^0.725 * weight_kg^0.425
Used by: BMI, BSA, cardiac index, metabolic markers
5. fitzpatrick_type
Type: int (1-6)
UI: Auto-detected during calibration (see spec above).
Show result with visual skin tone swatches and "Is this correct?" prompt.
Allow manual override via 6-swatch selector.
Default: null (triggers auto-detection on first scan)
Used by: rPPG channel weighting, SpO2 calibration, hemoglobin baseline,
bilirubin detection threshold, all skin markers
─────────────────────────────────────────────────
RECOMMENDED FIELDS (prompt but allow skip)
─────────────────────────────────────────────────
6. medical_conditions
Type: array of enum strings (multi-select)
Options: [
"hypertension",
"diabetes_type1",
"diabetes_type2",
"heart_disease",
"atrial_fibrillation",
"heart_failure",
"anemia",
"thyroid_hyper",
"thyroid_hypo",
"copd",
"asthma",
"raynaud",
"rosacea",
"eczema",
"psoriasis",
"liver_disease",
"kidney_disease",
"none"
]
UI: Scrollable checklist with search filter. "None" deselects all others.
Used by: Context flags on BP, HR, HRV, SpO2, skin markers.
If "atrial_fibrillation" selected → suppress AFib false-positive alerts.
If "hypertension" selected → flag BP in context of known condition.
7. medications
Type: array of enum strings (multi-select)
Options: [
"beta_blocker",
"calcium_channel_blocker",
"ace_inhibitor_arb",
"diuretic",
"antiarrhythmic",
"blood_thinner",
"statin",
"ssri_snri",
"stimulant_adhd",
"corticosteroid",
"retinoid_topical",
"insulin",
"metformin",
"thyroid_medication",
"birth_control",
"none",
"other"
]
UI: Scrollable checklist with search. "None" deselects all others.
"Other" opens a free-text field.
Critical: If "beta_blocker" selected →
- Adjust HR interpretation: expected range shifts down 10-30 bpm
- Adjust HRV: suppress low-HRV alerts (beta blockers reduce HRV)
- Flag in report: "Subject is on beta blockers — HR and HRV readings reflect medicated state"
8. fitness_level
Type: enum
Options: [
"sedentary", // desk job, no regular exercise
"lightly_active", // 1-2 sessions/week
"moderately_active", // 3-4 sessions/week
"very_active", // 5+ sessions/week
"competitive_athlete" // structured training, competition
]
UI: 5-option radio buttons with brief description for each
Used by: HR classification (athlete bradycardia is normal),
HRV percentile ranking, recovery/readiness scoring,
VO2max context
9. pregnancy_status
Type: enum
Options: ["not_pregnant", "trimester_1", "trimester_2", "trimester_3", "postpartum", "not_applicable"]
UI: Show only if biological_sex == "female" AND age 13-55.
Default: "not_applicable" for males.
Used by: BP thresholds change per trimester, HR +10-20 bpm normal,
hemoglobin (physiological anemia), skin pigmentation changes.
Critical: If pregnant → suppress hypertension alerts below 140/90,
use pregnancy-specific reference ranges.
─────────────────────────────────────────────────
OPTIONAL FIELDS (shown on scan-day form, not profile)
─────────────────────────────────────────────────
These are collected fresh before each scan session, not persisted in profile.
10. smoking_status
Type: enum
Options: ["never", "former", "current_light", "current_heavy", "vaping"]
Labels: "Never smoked", "Former smoker", "Current (<10/day)", "Current (10+/day)", "Vaping/e-cig"
UI: 5-option radio buttons
Used by: SpO2 interpretation (COHb bias), skin age adjustment,
arterial stiffness context, CVD risk scoring
11. last_meal_hours
Type: enum
Options: ["fasting_8plus", "4_to_8", "2_to_4", "1_to_2", "less_than_1"]
Labels: "Fasting (8+ hours)", "4-8 hours ago", "2-4 hours ago",
"1-2 hours ago", "Less than 1 hour ago"
UI: 5-option radio buttons
Used by: If "less_than_1" → flag postprandial state on HR, BP, HRV readings.
If "fasting_8plus" → glucose estimates are fasting glucose (different thresholds).
12. caffeine_recent
Type: boolean
UI: Toggle switch: "Caffeine in the last 2 hours?"
If true → show sub-field:
caffeine_amount: enum ["1_cup", "2_cups", "3_plus", "energy_drink"]
Used by: HR and BP context flag. If true →
"Subject consumed caffeine within 2 hours — cardiovascular readings may be elevated."
13. alcohol_recent
Type: boolean
UI: Toggle switch: "Alcohol in the last 2 hours?"
If true → show sub-field:
alcohol_amount: enum ["1_drink", "2_drinks", "3_plus"]
Used by: BP, skin redness, HRV context flags. If true →
"Subject consumed alcohol — skin flushing may affect erythema and rPPG signal quality."
14. emotional_state
Type: enum
Options: ["calm", "slightly_anxious", "moderately_stressed", "very_stressed"]
UI: 4-button horizontal selector with emoji:
😌 Calm | 😐 Slightly anxious | 😰 Moderately stressed | 😫 Very stressed
Used by: Stress score calibration anchor, white-coat-hypertension flag on BP.
Compare self-report vs algorithmic stress detection for validation.
─────────────────────────────────────────────────
DATA MODEL
─────────────────────────────────────────────────
type ScanType = 'face' | 'skin' | 'eye' | 'mouth' | 'pupil' | 'fingertip' | 'voice' | 'motion' | 'spectral' | 'palm';
interface UserProfile {
// Required (persisted)
date_of_birth: string; // ISO 8601 YYYY-MM-DD
biological_sex: 'male' | 'female';
height_cm: number; // required for: face, fingertip scans
weight_kg: number; // required for: face, fingertip scans
fitzpatrick_type: 1|2|3|4|5|6|null; // required for: face, skin, spectral, palm. null = auto-detect
// Derived (computed, not stored)
age: number;
bmi: number;
bsa: number;
// Recommended (persisted)
medical_conditions: string[];
medications: string[];
fitness_level: string; // relevant for: face, fingertip
pregnancy_status: string; // relevant for: face, fingertip, spectral, palm
// Expansion-specific (persisted)
neurological_conditions: string[]; // relevant for: pupil, voice, motion
recent_head_injury?: { // relevant for: pupil
occurred: boolean;
date?: string; // ISO 8601
severity?: 'mild' | 'moderate' | 'severe';
};
}
interface ScanSession {
// Which scans are being performed this session
selected_scans: ScanType[];
// Optional (per-session, not persisted in profile)
timestamp: string; // ISO 8601 datetime
smoking_status: string;
last_meal_hours: string;
caffeine_recent: boolean;
caffeine_amount?: string;
alcohol_recent: boolean;
alcohol_amount?: string;
emotional_state: string;
// Expansion-specific (per-session)
current_pain_level?: number; // 0-10, for pupil scan pain index
eye_drops_recent?: boolean; // last 4hr, for pupil + eye scans
eye_drops_type?: 'dilating' | 'redness_reducing' | 'lubricating' | 'other';
// Auto-detected (by system)
ambient_lux: number; // from camera exposure metadata
fitzpatrick_detected: 1|2|3|4|5|6; // from calibration frame
lighting_quality: 'good'|'fair'|'poor'; // from variance analysis
motion_quality: 'stable'|'minor_motion'|'excessive';
ambient_noise_db?: number; // for voice scan — reject if > 40dB
}
// ─── SCAN TYPE ROUTING ───
// Which pre-scan fields to show based on selected scan types:
//
// ALWAYS SHOW: age, sex, medical_conditions, medications, emotional_state
// face, fingertip: + height, weight, fitness_level, meal_time, caffeine, alcohol, smoking
// skin, spectral: + fitzpatrick, smoking
// eye: + eye_drops
// pupil: + neurological_conditions, head_injury, eye_drops, caffeine, pain_level
// voice: + neurological_conditions, emotional_state
// motion: + neurological_conditions
// palm: + fitzpatrick, pregnancy
//
// COMBINED SCAN SESSIONS:
// face + fingertip: Show once, apply to both — fingertip confirms face readings
// face + motion: Same video if tripod — record once at 60fps, run both pipelines
// face + spectral: Same positioning, just requires strict white lighting
// pupil → eye: Do pupil first (dim room), then turn on lights for eye scan
// face + voice: Sequential — face first, then voice immediately after
// palm + fingertip: Same hand, back-to-back — warm hands once for both
─────────────────────────────────────────────────
VALIDATION RULES
─────────────────────────────────────────────────
Before allowing scan to start — UNIVERSAL:
1. All required fields must be non-null (age, sex always; height/weight for face+fingertip)
2. age must be >= 1 and <= 120
3. height must be >= 50cm and <= 250cm (if applicable)
4. weight must be >= 10kg and <= 300kg (if applicable)
5. If biological_sex == 'female' AND age 13-55 → pregnancy_status should be prompted
6. If fitzpatrick_type == null AND scan requires it → auto-detect, confirm with user
SCAN-SPECIFIC VALIDATION:
Face/Skin/Mouth:
- Ambient light check: reject if estimated lux < 100
- Motion check: reject if face tracking confidence < 0.8
- Face completeness check: reject if forehead or both cheeks not in frame
Pupil AI Scan:
- Ambient light check: reject if estimated lux > 50 (room must be DIM)
- Both eyes must be detected with iris landmarks visible
- If eye_drops_recent == true AND type == 'dilating' → BLOCK scan with message:
"Pupil-dilating eye drops invalidate pupillometry. Wait 4-6 hours."
Fingertip AI Scan:
- Camera coverage check: >80% of frame must be occluded by finger (red channel dominant)
- Flash must be on: verify torch API is active
- Signal quality check: reject if AC amplitude < threshold (finger not placed correctly)
Voice AI Scan:
- Ambient noise check: measure 3 seconds of silence first, reject if >40dB
- Microphone access: verify permissions granted
Motion AI Scan:
- Frame rate check: reject if < 30fps, warn if < 60fps
- Camera stability check: reject if inter-frame jitter > threshold in first 3 seconds
Spectral AI Scan:
- White balance check: reject if color temperature < 4500K or > 7000K
- Same face completeness checks as Face AI Scan
Palm AI Scan:
- Palm detection check: MediaPipe Hands must detect open palm with >0.8 confidence
- Ambient light check: reject if estimated lux < 200
Pre-scan warnings (don't block, but flag):
UNIVERSAL:
- caffeine_recent == true → "Caffeine may elevate HR, constrict pupils, and affect BP"
- alcohol_recent == true → "Alcohol may affect skin color, dilate pupils, and alter BP"
- last_meal_hours == "less_than_1" → "Recent meal may affect cardiovascular readings"
- emotional_state == "very_stressed" → "Elevated stress may affect resting vitals and pupil dynamics"
PUPIL-SPECIFIC:
- caffeine_recent == true → additional: "Caffeine constricts pupils — may reduce constriction amplitude"
- medications includes 'stimulant_adhd' → "Stimulants dilate pupils — note in interpretation"
- medications includes 'ssri_snri' → "SSRIs may affect pupil dynamics"
VOICE-SPECIFIC:
- If cold/flu reported → "Respiratory illness affects voice biomarkers — note in interpretation"
FINGERTIP-SPECIFIC:
- If cold hands detected (low initial signal) → "Warm your hands for 1-2 minutes before retrying"
═══════════════════════════════════════════════════════════════
CORE PIPELINE (Face AI Scan — 120 markers)
═══════════════════════════════════════════════════════════════
Camera (60fps RGB, locked exposure/WB)
|
v
Layer 0: Raw Frames ─────────────────────────> Ring Buffer
|
v
Layer 1: Face Detection (MediaPipe 478pts) ──> 9 ROIs x RGB spatial means
| 478 landmarks, face quality flag
|── Pool H (Facial Action Units)
|── Pool I (Eye Metrics)
v
Layer 2: Temporal Signals ───────────────────> Per-ROI channel time series
| DC/AC decomposition
|── Pool F (SpO2 Ratio-of-Ratios)
|── Pool G (Skin DC Color Analysis)
v
Layer 3: BVP Waveform (POS/CHROM + bandpass)─> bvp_primary, bvp_cheeks
| motion_mask, snr_db
|── Pool E (Respiratory Signal)
|── Pool D (Waveform Morphology)
|── Pool J (PTT / PWV)
v
Layer 4: Peak Detection & IBI ───────────────> nn_intervals, peak/trough times
|
|── Pool A (Time-Domain IBI Stats)
|── Pool B (Freq-Domain IBI Analysis)
|── Pool C (Nonlinear IBI Analysis)
v
Layer 6: 120 Core Markers (pure arithmetic on pool outputs)
v
Layer 7: Composite Scores (weighted marker combinations)
═══════════════════════════════════════════════════════════════
EXPANSION MODULE PIPELINES (2.0V — 34 additional markers)
Each module is INDEPENDENT of the core pipeline.
Only activated when user selects that scan type.
═══════════════════════════════════════════════════════════════
┌─ Pupil AI Scan (#121-128) ─────────────────────────────────────
│ Selfie camera + screen flash stimulus (dim room)
│ → Iris segmentation (MediaPipe / Hough transform)
│ → Pupil diameter tracking per frame
│ → PLR speed, constriction amplitude, dilation latency
│ → Consensual response, cognitive load, concussion score
│ FDA precedent: Brightlamp Reflex (510(k) K182546)
│
├─ Fingertip AI Scan (#129-134) ─────────────────────────────────
│ Rear camera + flash transillumination (finger on lens)
│ → Contact PPG extraction (red channel dominant)
│ → Clinical HR, AFib screening, SpO2, hemoglobin est.
│ → Blood glucose trend, peripheral perfusion index
│ FDA precedent: FibriCheck, Preventicus (AFib)
│
├─ Voice AI Scan (#135-140) ─────────────────────────────────────
│ Phone microphone (quiet room <40dB)
│ → openSMILE feature extraction (eGeMAPSv02)
│ → Whisper ASR for linguistic analysis
│ → Tremor, depression, respiratory, cognitive, stress, Parkinson's
│ Research: Vocalis Health, Sonde Health
│
├─ Motion AI Scan (#141-145) ────────────────────────────────────
│ Camera on tripod (60fps+, MUST be stable)
│ → Eulerian Video Magnification (MIT CSAIL)
│ → Tremor characterization, respiratory waveform
│ → Micro-expressions, carotid pulse, neonatal vitals
│ Can share video with Face AI Scan if tripod + 60fps
│
├─ Spectral AI Scan (#146-149) ──────────────────────────────────
│ Front camera + strict white light (5000-6500K)
│ → Enhanced RGB spectroscopy (DC channel analysis)
│ → Anemia screening, jaundice/bilirubin, dehydration, bruise aging
│ CE-marked precedent: BiliCam, Picterus (jaundice)
│
└─ Palm AI Scan (#150-154) ──────────────────────────────────────
Rear camera viewing open palm (bright, even light)
→ Palm rPPG (same POS/CHROM as face, palm ROI)
→ Perfusion mapping, palmar erythema, nail bed color
→ Capillary refill timing
Less skin tone bias than face (Fitzpatrick V-VI preferred)
═══════════════════════════════════════════════════════════════
v4.0 COVERAGE GAP CLOSURE PIPELINES (#227-346, 120 markers)
Frequency extensions use existing v3.0 headset pipeline.
New scan modalities use phone camera/mic/manual input only.
═══════════════════════════════════════════════════════════════
┌─ Frequency Extensions (#227-308) ──────────────────────────
│ Digestive (#227-235), Hepatobiliary (#236-243),
│ Pancreatic & Renal (#244-249), Pulmonary (#250-252),
│ Bone & Skeletal (#253-265), Blood Chemistry Ext (#266-271),
│ Lipid & Fatty Acid (#272-279), Trace Element Ext (#280-286),
│ Vitamin Extensions (#287-289), Extra Meridians (#290-293),
│ Cerebrovascular (#294-299), Thyroid Detail (#300-301),
│ Eye Health (#302-308)
│ → Uses same bone conduction bioresonance pipeline as v3.0
│ → Extended frequency sweep ranges + organ-specific analysis
│
├─ Tongue AI Scan (#309-316) ───────────────────────────────────
│ Rear camera aimed at extended tongue (well-lit room)
│ → Color segmentation (pale/red/purple classification)
│ → Coating analysis (thin/thick, white/yellow detection)
│ → Shape assessment (swollen/thin/cracked/deviated)
│ → Moisture level, sublingual vein visibility
│ Based on Traditional Chinese Medicine tongue diagnosis
│
├─ Nail Bed Analysis (#317-322) ─────────────────────────────
│ Rear camera macro on fingernails (daylight preferred)
│ → Nail bed color classification (pink/pale/cyanotic)
│ → Surface texture (ridges, pitting, Beau's lines)
│ → Lunula assessment (size, shape, visibility)
│ → Capillary refill time estimation via video
│
├─ Breath Analysis (#323-326) ───────────────────────────────
│ Phone microphone near nose/mouth (quiet room <30dB)
│ → Audio waveform analysis of breathing patterns
│ → Breathing rate variability, tidal volume estimate
│ → Inspiratory capacity proxy, pattern regularity
│ No new hardware — uses existing phone mic
│
├─ Body Metrics (#327-336) ──────────────────────────────────
│ Manual input: height, weight, age, sex
│ Optional: standing photo for posture estimation
│ → BMI, body fat % (Navy method), lean body mass
│ → BMR (Mifflin-St Jeor), waist-hip ratio, BSA
│ → Obesity classification, ideal weight range
│ All calculations are pure arithmetic — no AI inference
│
└─ Mood & Consciousness (#337-346) ─────────────────────────
PHQ-2/GAD-2 questionnaire + facial expression camera
→ Emotional valence, stress resilience, anxiety level
→ Depression screening, motivation, mindfulness readiness
→ Sleep quality proxy, social connection, life satisfaction
→ Consciousness composite score
Validated scales + AU-based expression analysis
Main Thread (UI + Scan Type Router)
|
|═══ CORE PIPELINE ═══════════════════════════════════════
|
|── Worker 1: Camera + Face Detection
| Captures frames, runs MediaPipe Face Mesh,
| extracts ROI means, streams to Worker 2
|
|── Worker 2: Signal Processing
| Accumulates temporal signals, extracts BVP (POS/CHROM),
| detects peaks, computes IBI series
|
|── Worker 3: Analysis Engine
| Runs all 10 pools (A-J) in parallel waves,
| computes 120 core markers, generates composite scores
|
|═══ EXPANSION MODULES (2.0V) ════════════════════════════
| Only spawned when user selects corresponding scan type
|
|── Worker 4: Pupillometry Engine (if Pupil AI Scan selected)
| Iris segmentation, screen flash control, PLR tracking,
| pupil diameter time series → markers #121-128
|
|── Worker 5: Contact PPG Engine (if Fingertip AI Scan selected)
| Rear camera PPG extraction, flash management,
| clinical HR/SpO2/AFib/hemoglobin → markers #129-134
|
|── Worker 6: Voice Analysis Engine (if Voice AI Scan selected)
| Audio capture, openSMILE features, Whisper ASR,
| prosody/tremor/NLP classifiers → markers #135-140
|
|── Worker 7: Motion Amplification (if Motion AI Scan selected)
| Eulerian pyramid decomposition, temporal filtering,
| tremor/respiratory/carotid analysis → markers #141-145
|
|── Worker 8: Spectral + Palm Engine (if Spectral/Palm selected)
Enhanced RGB spectroscopy, palm rPPG, nail analysis,
capillary refill timing → markers #146-154
src/
├── workers/
│ ├── camera.worker.ts # Frame capture + face mesh
│ ├── signal.worker.ts # BVP extraction + peak detection
│ ├── analysis.worker.ts # Pools + markers + composites
│ ├── pupil.worker.ts # [2.0V] Iris segmentation + PLR
│ ├── fingertip.worker.ts # [2.0V] Contact PPG engine
│ ├── voice.worker.ts # [2.0V] Audio features + ASR
│ ├── motion.worker.ts # [2.0V] Eulerian magnification
│ └── spectral-palm.worker.ts # [2.0V] Spectral + palm analysis
├── pipeline/
│ ├── layer0-frames.ts # Camera config, frame buffer
│ ├── layer1-face.ts # MediaPipe init, ROI extraction
│ ├── layer2-temporal.ts # Signal assembly, DC/AC split
│ ├── layer3-bvp.ts # POS, CHROM, bandpass, motion rejection
│ └── layer4-peaks.ts # Peak detection, IBI, ectopic removal
├── pools/
│ ├── pool-a-time-domain.ts # SDNN, RMSSD, pNN50, tri-index...
│ ├── pool-b-freq-domain.ts # Welch PSD, LF/HF, VLF...
│ ├── pool-c-nonlinear.ts # SampEn, DFA, Poincare, Lyapunov...
│ ├── pool-d-morphology.ts # Rise time, AIx, SDPPG, IPA...
│ ├── pool-e-respiratory.ts # RIIV, RIFV, RIAM, resp rate...
│ ├── pool-f-spo2.ts # Ratio-of-ratios, perfusion index
│ ├── pool-g-skin-color.ts # Melanin, erythema, CIE-Lab...
│ ├── pool-h-action-units.ts # AU detection from landmarks
│ ├── pool-i-eye-metrics.ts # EAR, blink rate, PERCLOS...
│ └── pool-j-ptt-pwv.ts # Cross-correlation, PWV
├── markers/
│ └── compute-markers.ts # 120 core marker calculations
├── expansion/ # ═══ 2.0V EXPANSION MODULES ═══
│ ├── scan-router.ts # Routes to correct module by ScanType
│ ├── pupillometry/
│ │ ├── iris-segmentation.ts # Pupil detection + diameter tracking
│ │ ├── flash-stimulus.ts # Screen brightness control + timing
│ │ └── pupil-markers.ts # Markers #121-128
│ ├── fingertip-ppg/
│ │ ├── contact-ppg.ts # Rear camera + flash PPG extraction
│ │ └── fingertip-markers.ts # Markers #129-134
│ ├── voice-analysis/
│ │ ├── audio-features.ts # openSMILE feature extraction
│ │ ├── speech-nlp.ts # Whisper transcription + NLP
│ │ └── voice-markers.ts # Markers #135-140
│ ├── motion-amplification/
│ │ ├── eulerian-magnification.ts # EVM pyramid + temporal filter
│ │ └── motion-markers.ts # Markers #141-145
│ ├── spectral-analysis/
│ │ ├── rgb-spectroscopy.ts # Enhanced channel analysis
│ │ └── spectral-markers.ts # Markers #146-149
│ ├── palm-scanning/
│ │ ├── palm-rppg.ts # Palm ROI detection + rPPG
│ │ ├── nail-analysis.ts # Nail bed color + capillary refill
│ │ └── palm-markers.ts # Markers #150-154
│ ├── frequency/ # ═══ v3.0 FREQUENCY + v4.0 EXTENSIONS ═══
│ │ ├── bioresonance-engine.ts # Bone conduction frequency sweep
│ │ ├── freq-vitals.ts # Blood chemistry #155-164
│ │ ├── freq-nutrition.ts # Nutritional #165-176
│ │ ├── freq-organs.ts # Organ deviation #177-186
│ │ ├── freq-toxicity.ts # Toxicity #187-194
│ │ ├── freq-systems.ts # Body systems #214-221
│ │ ├── freq-brain.ts # Brain entrainment #222-226
│ │ ├── freq-digest.ts # Digestive frequencies #227-235
│ │ ├── freq-hepato.ts # Hepatobiliary frequencies #236-243
│ │ ├── freq-panc-renal.ts # Pancreatic & renal #244-249
│ │ ├── freq-pulm.ts # Pulmonary frequencies #250-252
│ │ ├── freq-bone.ts # Bone & skeletal #253-265
│ │ ├── freq-blood-chem.ts # Blood chemistry extensions #266-271
│ │ ├── freq-lipid.ts # Lipid & fatty acid #272-279
│ │ ├── freq-trace-ext.ts # Trace element extensions #280-286
│ │ ├── freq-vitamin-ext.ts # Vitamin extensions #287-289
│ │ ├── freq-extra-merid.ts # Extraordinary meridians #290-293
│ │ ├── freq-cerebro.ts # Cerebrovascular #294-299
│ │ └── freq-thyroid-detail.ts # Thyroid detail #300-301
│ ├── eye-health/ # ═══ v4.0 EYE HEALTH ═══
│ │ └── eye-health.ts # Eye health analysis #302-308
│ ├── tongue-scan/ # ═══ v4.0 TONGUE SCAN ═══
│ │ └── tongue-scan.ts # Tongue scan #309-316
│ ├── nail-scan/ # ═══ v4.0 NAIL SCAN ═══
│ │ └── nail-scan.ts # Nail bed analysis #317-322
│ ├── breath-scan/ # ═══ v4.0 BREATH SCAN ═══
│ │ └── breath-scan.ts # Breath analysis #323-326
│ ├── body-metrics/ # ═══ v4.0 BODY METRICS ═══
│ │ └── body-metrics.ts # Body metrics #327-336
│ └── mood-assess/ # ═══ v4.0 MOOD ═══
│ └── mood-assess.ts # Mood & consciousness #337-346
├── composites/
│ └── composite-scores.ts # 9 composite health scores
├── types/
│ ├── scan-types.ts # ScanType union, scan routing config
│ ├── scan-result.ts # ScanResult, MarkerResult interfaces
│ └── pool-outputs.ts # Typed pool output interfaces
└── utils/
├── dsp.ts # FFT, Butterworth, filtfilt, Welch
├── stats.ts # Mean, std, entropy, DFA helpers
└── calibration.ts # Fitzpatrick detection, light check
type ScanType = "face" | "skin" | "eye" | "mouth"
| "pupil" | "fingertip" | "voice" | "motion" | "spectral" | "palm"
| "frequency" | "meridian" | "chakra" | "brain-entrainment"
| "tongue" | "nail" | "breath" | "body-metrics" | "mood";
interface ScanResult {
id: string; // UUID
userId: string;
timestamp: number; // Unix ms
duration: number; // scan duration in seconds
scanTypes: ScanType[]; // which scans were performed
quality: "good" | "fair" | "poor";
markers: MarkerResult[]; // up to 346 markers (depends on scan types)
composites: CompositeScores; // 9 composite health scores
poolOutputs: AllPoolOutputs; // raw pool data for debugging
expansionOutputs?: { // 2.0V expansion module outputs
pupillometry?: PupilResults;
fingertipPPG?: FingertipResults;
voiceAnalysis?: VoiceResults;
motionAmplification?: MotionResults;
spectralAnalysis?: SpectralResults;
palmScanning?: PalmResults;
};
metadata: ScanMetadata; // device, lighting, motion stats
}
interface MarkerResult {
id: number; // 1-346 (1-120 core, 121-154 expansion, 155-226 frequency, 227-346 v4.0)
name: string;
value: number | null; // null if pool/module failed
unit: string;
confidence: number; // 0-1
tier: "verified" | "estimate" | "emerging" | "clinical";
scanType: ScanType; // which scan type produced this marker
normalRange: { low: number; high: number } | null;
errorReason?: string; // why null if applicable
}
📄 Download the .MD file for the complete 346-marker computation spec, full pipeline details, all pool algorithms, expansion module specs, and v4.0 coverage gap closure details.
═══ CORE PIPELINE (Face/Skin/Eye/Mouth — 120 markers) ═══
Layer 0: Raw Frames 60fps RGB, locked exposure/WB, 640x480 min
Layer 1: Face & ROI MediaPipe 478 landmarks, 9 ROIs x 3 channels = 27 values/frame
Layer 2: Temporal Signals 14 named signals at 60Hz (SIG_G_FH, SIG_R_NB, etc.)
Layer 3: BVP Waveform POS/CHROM extraction, 0.7-4.0Hz bandpass, motion rejection
Layer 4: Peak Detection Systolic peaks, diastolic troughs, dicrotic notch, NN intervals
Layer 5: Derived Pools 10 signal pools (A-J), computed once, cached
Layer 6: Markers 120 core biomarkers from pool arithmetic
Layer 7: Composites 9 weighted composite health scores
═══ EXPANSION MODULE PIPELINES (2.0V — 34 markers) ═══
Each module runs INDEPENDENTLY — does not affect or depend on core pipeline.
EXP-P: Pupil Pipeline Selfie cam + screen flash → iris seg → PLR/dilation → #121-128
EXP-F: Fingertip Pipeline Rear cam + flash → contact PPG → HR/SpO2/AFib → #129-134
EXP-V: Voice Pipeline Microphone → openSMILE + Whisper → classifiers → #135-140
EXP-M: Motion Pipeline Camera (tripod) → Eulerian magnification → tremor/resp → #141-145
EXP-S: Spectral Pipeline Front cam + white light → enhanced RGB spectroscopy → #146-149
EXP-L: Palm Pipeline Rear cam + palm detect → palm rPPG + nail analysis → #150-154
| Pool | Name | Input Layer | Key Outputs | Markers Fed |
|---|---|---|---|---|
| A | Time-Domain IBI | Layer 4 | mean_nn, sdnn, rmssd, pnn50, hr_mean/min/max, tri_index, tinn | 36 |
| B | Freq-Domain IBI | Layer 4 | vlf/lf/hf_power, lf_hf_ratio, lf/hf_norm, peak freqs | 23 |
| C | Nonlinear IBI | Layer 4 | SampEn, ApEn, DFA alpha1/2, Poincare SD1/SD2, Lyapunov | 17 |
| D | BVP Morphology | Layers 3+4 | rise_time, AIx, stiffness_index, SDPPG ratios, IPA | 20 |
| E | Respiratory | Layers 3+4 | resp_rate, RIIV/RIFV/RIAM fusion, ie_ratio, depth_proxy | 8 |
| F | SpO2 (Ratio-of-Ratios) | Layer 2 | spo2_estimate, perfusion_index, ratio_of_ratios | 4 |
| G | Skin DC Color | Layer 2 | melanin/erythema/hemoglobin/bilirubin index, ITA, Fitzpatrick | 13 |
| H | Facial Action Units | Layer 1 | 14 AUs, pain_estimate, valence, arousal, tension | 21 |
| I | Eye Metrics | Layer 1 | EAR, blink_rate, PERCLOS, pupil_size, sclera color | 8 |
| J | PTT / PWV | Layer 3 | ptt_mean, pwv_estimate, vascular_age, arterial_stiffness | 4 |
═══ CORE PIPELINE DEPENDENCY TREE ═══
Layer 0: Raw Frames
|
└─> Layer 1: Face & ROI Detection
|
├─> Pool H (AUs) ─────> Markers #101-#120
├─> Pool I (Eyes) ─────> Markers #95-#100, #88-#89
|
└─> Layer 2: Temporal Signals
|
├─> Pool F (SpO2) ─────> Markers #3, #69-#71
├─> Pool G (Skin) ─────> Markers #55, #85-#94
|
└─> Layer 3: BVP Waveform
|
├─> Pool E (Resp) ──> Markers #7, #72-#76
├─> Pool D (Morph) ─> Markers #4-#6, #34-#37, #77-#84
├─> Pool J (PTT) ──> Markers #33, #38-#40
|
└─> Layer 4: Peaks & IBI
|
├─> Pool A (Time) ─> Markers #1-#2, #8-#16
├─> Pool B (Freq) ─> Markers #17-#26
└─> Pool C (NL) ─> Markers #41-#54
═══ EXPANSION MODULE DEPENDENCIES (2.0V) ═══
Independent of core. Each activated only when scan type selected.
EXP-P: Selfie Camera + Screen Flash API
└─> Iris Segmentation ─> Pupil Diameter Time Series
└─> PLR, Constriction, Dilation, Consensual ─> #121-128
EXP-F: Rear Camera + Flash (Torch API)
└─> Contact PPG Extraction (red channel)
└─> Peak Detection ─> HR, AFib, SpO2, Hemoglobin ─> #129-134
EXP-V: Microphone (Web Audio API)
└─> openSMILE Features + Whisper ASR
└─> Classifiers (tremor, depression, cognitive) ─> #135-140
EXP-M: Camera on Tripod (60fps+)
└─> Eulerian Video Magnification (Laplacian pyramid)
└─> Frequency Analysis + Displacement Tracking ─> #141-145
EXP-S: Front Camera + White Light Validation
└─> Enhanced RGB Spectroscopy (DC channels, CIE-Lab)
└─> Chromophore Estimation ─> #146-149
EXP-L: Rear Camera + MediaPipe Hands
└─> Palm ROI rPPG + Nail Detection
└─> Perfusion Map, Erythema, Nail Color, CRT ─> #150-154
═══ CORE PIPELINE SCHEDULE ═══
Phase 1: Layer 0 + Layer 1 (real-time during scan, 30-60s)
Phase 2: Layer 2 (post-scan, <200ms)
Phase 3a: Pools F, G (parallel, from Layer 2) <50ms
Phase 3b: Pools H, I (parallel, from Layer 1) <50ms
Phase 4: Layer 3 (from Layer 2) <500ms
Phase 5: Layer 4 (from Layer 3) <100ms
Phase 6: Pools A,B,C,D,E,J (parallel, from Layers 3+4) <500ms (bottleneck: Pool C)
Phase 7: 120 core markers (from pools) <50ms
Phase 8: Composite scores (from markers) <10ms
─────────────────────────────────────────────────────────────────
Core post-scan processing: <1.5 seconds
═══ EXPANSION MODULE SCHEDULE (2.0V) ═══
Each runs independently. Can run in parallel with core if scan types overlap.
EXP-P: Pupil AI Scan Real-time capture (2-3 min) → PLR analysis <500ms → 8 markers <20ms
EXP-F: Fingertip AI Scan Real-time capture (30-60s) → PPG extraction <300ms → 6 markers <20ms
EXP-V: Voice AI Scan Audio capture (1-2 min) → Feature extraction <2s → 6 markers <50ms
EXP-M: Motion AI Scan Video capture (30-60s) → EVM processing <3s → 5 markers <50ms
EXP-S: Spectral AI Scan Frame capture (15-30s) → Spectral analysis <200ms → 4 markers <10ms
EXP-L: Palm AI Scan Video capture (30-60s/hand) → Palm rPPG + analysis <1s → 5 markers <20ms
─────────────────────────────────────────────────────────────────
Expansion bottleneck: Motion (EVM ~3s), Voice (feature extraction ~2s)
Parallel combos: Face+Motion share video. Spectral reuses face frames.
| Pool Failed | Markers Lost | Severity |
|---|---|---|
| A (Time-Domain) | 36 | CATASTROPHIC — abort scan |
| B (Freq-Domain) | 23 | Major — report partial results |
| C (Nonlinear) | 17 | Major — degrades risk scores |
| D (Morphology) | 20 | Major — no BP estimates |
| E (Respiratory) | 8 | Moderate — flag missing |
| F (SpO2) | 4 | Moderate — flag missing |
| G (Skin) | 13 | Moderate — flag missing |
| H (AUs) | 21 | Moderate — no emotion/pain |
| I (Eyes) | 8 | Moderate — no fatigue |
| J (PTT/PWV) | 4 | Minor — flag missing |
| Module Failed | Markers Lost | Cause | Impact on Core |
|---|---|---|---|
| Pupillometry | 8 (#121-128) | Iris not detected, room too bright, dilating eye drops | NONE — core pipeline unaffected |
| Fingertip PPG | 6 (#129-134) | Finger not covering camera, flash unavailable, cold hands | NONE — core pipeline unaffected |
| Voice Analysis | 6 (#135-140) | Background noise >40dB, microphone denied, insufficient speech | NONE — core pipeline unaffected |
| Motion Amplification | 5 (#141-145) | Camera not stable (handheld), frame rate <30fps, subject moved | NONE — core pipeline unaffected |
| Spectral Analysis | 4 (#146-149) | Non-white lighting (<4500K or >7000K), makeup present | NONE — core pipeline unaffected |
| Palm AI Scanning | 5 (#150-154) | Palm not detected, insufficient lighting, nail polish present | NONE — core pipeline unaffected |
Expansion modules are fully isolated. A failure in any expansion module has zero impact on the core 120 markers. Each module gracefully degrades independently — failed markers return null with error reason.
| Requirement | Minimum Spec | Recommended | Why |
|---|---|---|---|
| OS | iOS 15+ / Android 11+ | iOS 17+ / Android 14+ | Camera API exposure lock, Web Audio API, Bluetooth 5.0 |
| Camera | Front: 8MP, 30fps | Front: 12MP+, 60fps | rPPG signal quality depends on resolution and frame rate |
| Flash/Torch | Rear LED flash | Multi-LED flash | Fingertip contact PPG transillumination |
| Microphone | Single mic | Dual mic with noise cancellation | Voice scan quality, ambient noise measurement |
| Bluetooth | BLE 4.2 | Bluetooth 5.0+ | Bone conduction headset connection (3.0 frequency scans) |
| Speaker | Mono | Stereo | Binaural beat delivery for brain entrainment (stereo required) |
| Processor | A12 Bionic / Snapdragon 730 | A15+ / Snapdragon 8 Gen 1+ | Real-time MediaPipe face mesh + signal processing |
| RAM | 3GB | 6GB+ | Video frame buffer + signal processing pools |
| Storage | 50MB app + 200MB data | 500MB+ | Frequency database, scan history, audio files |
| Internet | Required for first sync | Always connected | Offline capable after initial download. Cloud sync for history. |
| Permission | Scans That Need It | When Requested |
|---|---|---|
| Camera | Face, Skin, Eye, Mouth, Pupil, Motion, Spectral (v1.0 + v2.0), Tongue, Nail, Eye Health, Body Metrics, Mood (v4.0) | On first scan attempt |
| Microphone | Voice (v2.0), Breath Analysis (v4.0) | On first voice or breath scan |
| Bluetooth | Frequency, Meridian, Chakra, Brain Entrainment (v3.0), Frequency extensions (v4.0) | On first frequency scan — headset pairing flow |
| Notifications | All — scan reminders, results ready | After first completed scan |
| Storage/Files | Report export (PDF/MD), audio file storage | On first export or audio generation |
APP ARCHITECTURE — SCAN VERSION ROUTING
═══════════════════════════════════════════
VERSION DEPLOYMENT STAGES
═══════════════════════════════════════════
v1.0 — Core rPPG (Phone Only)
Scans: Face, Skin, Eye, Mouth
Markers: #1-120
Hardware: Smartphone camera only
APIs: getUserMedia, MediaPipe Face Mesh, Web Audio (for analysis)
Permissions: Camera
Processing: 100% client-side
Target: MVP launch
v2.0 — Expansion Modules
Scans: Pupil, Fingertip, Voice, Motion, Spectral, Palm
Markers: #121-154
Hardware: Phone + optional tripod + ring light
APIs: Camera (rear for fingertip/palm), Microphone, Screen brightness
Permissions: Camera, Microphone
Processing: 100% client-side
Target: 3 months post-v1.0
v3.0 — Frequency Analysis
Scans: Frequency, Meridian, Chakra, Brain Entrainment
Markers: #155-226
Hardware: Phone + Bluetooth bone conduction headset
APIs: Web Bluetooth API, Web Audio API (generation + analysis)
Permissions: Camera, Microphone, Bluetooth
Processing: Client-side audio gen + analysis. Server-side Blueprint DB.
Target: 6 months post-v1.0
v4.0 — Coverage Gap Closure
Scans: Digestive, Hepatobiliary, Pancreatic, Renal, Pulmonary, Bone, Blood Chemistry, Lipid,
Trace Element Ext, Vitamin Ext, Extra Meridians, Cerebrovascular, Thyroid Detail,
Eye Health, Tongue, Nail, Breath, Body Metrics, Mood
Markers: #227-346 (120 markers)
Hardware: Phone camera + microphone + manual input (height, weight, age)
APIs: Camera (tongue/nail/eye/body-metrics/mood), Microphone (breath), Manual Input (body metrics)
Permissions: Camera, Microphone
Processing: 100% client-side. No new hardware beyond existing phone sensors.
Target: 9 months post-v1.0
═══════════════════════════════════════════
BLUETOOTH HEADSET INTERFACE
═══════════════════════════════════════════
interface HeadsetConnection {
deviceId: string;
deviceName: string;
batteryLevel: number; // 0-100
isConnected: boolean;
isStereo: boolean; // required for brain entrainment
frequencyResponse: [number, number]; // Hz range [low, high]
transferFunction: Float32Array; // calibration data per user
lastCalibrated: string; // ISO 8601 date
}
interface FrequencySweepConfig {
startHz: number; // typically 1
endHz: number; // typically 10_000_000 (10 MHz)
sweepDuration: number; // seconds
resolution: number; // Hz per step
amplitude: number; // 0-1 (volume/intensity)
}
// Bluetooth connection flow
async function connectHeadset(): Promise<HeadsetConnection> {
// 1. navigator.bluetooth.requestDevice({ filters: [{ services: ['audio'] }] })
// 2. device.gatt.connect()
// 3. Get audio characteristic for bidirectional data
// 4. Send calibration tone, record response
// 5. Return HeadsetConnection object
}
═══════════════════════════════════════════
SCAN TYPE ROUTER
═══════════════════════════════════════════
// Determines which UI flow, permissions, and hardware to activate
function getScanRequirements(scanType: ScanType) {
const reqs = {
// v1.0
face: { version: '1.0', hardware: ['camera'], permissions: ['camera'], setup: 'face-guide' },
skin: { version: '1.0', hardware: ['camera'], permissions: ['camera'], setup: 'skin-guide' },
eye: { version: '1.0', hardware: ['camera'], permissions: ['camera'], setup: 'eye-guide' },
mouth: { version: '1.0', hardware: ['camera'], permissions: ['camera'], setup: 'mouth-guide' },
// v2.0
pupil: { version: '2.0', hardware: ['camera','screen'], permissions: ['camera'], setup: 'pupil-guide' },
fingertip:{ version: '2.0', hardware: ['camera-rear','flash'], permissions: ['camera'], setup: 'finger-guide' },
voice: { version: '2.0', hardware: ['microphone'], permissions: ['microphone'], setup: 'voice-guide' },
motion: { version: '2.0', hardware: ['camera','tripod'], permissions: ['camera'], setup: 'motion-guide' },
spectral: { version: '2.0', hardware: ['camera','white-light'], permissions: ['camera'], setup: 'spectral-guide' },
palm: { version: '2.0', hardware: ['camera-rear'], permissions: ['camera'], setup: 'palm-guide' },
// v3.0
frequency:{ version: '3.0', hardware: ['bluetooth-headset'], permissions: ['bluetooth'], setup: 'headset-guide' },
meridian: { version: '3.0', hardware: ['bluetooth-headset','electrodermal-optional'], permissions: ['bluetooth'], setup: 'meridian-guide' },
chakra: { version: '3.0', hardware: ['bluetooth-headset'], permissions: ['bluetooth'], setup: 'chakra-guide' },
// v4.0
tongue: { version: '4.0', hardware: ['camera'], permissions: ['camera'], setup: 'tongue-guide' },
nail: { version: '4.0', hardware: ['camera-rear'], permissions: ['camera'], setup: 'nail-guide' },
breath: { version: '4.0', hardware: ['microphone'], permissions: ['microphone'], setup: 'breath-guide' },
'body-metrics': { version: '4.0', hardware: ['camera','manual-input'], permissions: ['camera'], setup: 'metrics-guide' },
mood: { version: '4.0', hardware: ['camera','questionnaire'], permissions: ['camera'], setup: 'mood-guide' },
};
return reqs[scanType];
}
═══════════════════════════════════════════
AUDIO API USAGE
═══════════════════════════════════════════
// v2.0 Voice AI Scan — microphone input analysis
const audioCtx = new AudioContext();
const mic = await navigator.mediaDevices.getUserMedia({ audio: true });
const source = audioCtx.createMediaStreamSource(mic);
const analyser = audioCtx.createAnalyser();
source.connect(analyser);
// FFT for voice frequency extraction
// v3.0 Frequency AI Scan — tone generation through headset
function generateTone(frequencyHz, durationMs) {
const osc = audioCtx.createOscillator();
osc.type = 'sine';
osc.frequency.value = frequencyHz;
osc.connect(audioCtx.destination); // routes to Bluetooth headset
osc.start();
setTimeout(() => osc.stop(), durationMs);
}
// v3.0 Brain Entrainment — binaural beats (stereo required)
function generateBinauralBeat(targetBeatHz, baseHz = 200) {
const merger = audioCtx.createChannelMerger(2);
const oscL = audioCtx.createOscillator(); // left channel
const oscR = audioCtx.createOscillator(); // right channel
oscL.frequency.value = baseHz;
oscR.frequency.value = baseHz + targetBeatHz;
oscL.connect(merger, 0, 0); // left
oscR.connect(merger, 0, 1); // right
merger.connect(audioCtx.destination);
oscL.start(); oscR.start();
}
| Component | Recommended | Why |
|---|---|---|
| Face Detection | MediaPipe Face Mesh | 478 landmarks, 60fps in-browser (WASM/WebGL), Apache 2.0, built-in 3D pose |
| rPPG Extraction | POS (baseline) + EfficientPhys (prod) | POS: 100 lines, solid baseline. EfficientPhys: ~1.5 BPM MAE, motion tolerant |
| Signal Processing | Custom DSP + fft.js | Float64 precision, full control over Butterworth/Welch, zero dependency bloat |
| Image Processing | OpenCV.js + Canvas API | Canvas for ROI pixel extraction (zero-dep), OpenCV for CIE-Lab/texture analysis |
| DL Inference | ONNX Runtime Web | WebGPU + WebGL + WASM fallback, 8-15ms inference, int8 quantization, PyTorch direct export |
| Camera Access | navigator.mediaDevices.getUserMedia | Native API, exposure/WB lock on Chrome 89+, no library needed |
| HRV Analysis | Custom JS (port hrv-analysis) | No production JS HRV lib exists. ~300 lines to port all metrics. Validate vs Kubios. |
| AU Detection | MediaPipe Blendshapes (baseline) | 52 blendshapes free with Face Mesh, 60fps. Custom CNN for clinical-grade AU intensity. |
| Skin Analysis | OpenCV.js (GLCM/Gabor) + CNN (ONNX) | Classical CV for texture metrics, CNN for classification (acne, lesions) |
| Data Storage | IndexedDB (local) + Supabase (cloud) | Offline-first, sync when online. Auth + RLS + real-time built in. |
| UI Framework | React + Next.js | Standard stack, SSR for marketing pages, client for scan UI |
| Charting | D3.js + Recharts | D3 for Poincare/heatmaps, Recharts for dashboards |
npm install @mediapipe/tasks-vision # Face mesh, 478 landmarks
npm install fft.js # FFT for Welch PSD, freq analysis
npm install @techstark/opencv-js # Color space, texture, skin analysis
npm install onnxruntime-web # DL model inference (EfficientPhys)
npm install idb # IndexedDB wrapper for local storage
npm install @supabase/supabase-js # Cloud sync, auth, RLS
npm install d3 # Poincare plots, heatmaps
npm install recharts # Dashboard charts
npm install next react react-dom # UI framework
const stream = await navigator.mediaDevices.getUserMedia({
video: {
frameRate: { ideal: 60, min: 25 },
width: { ideal: 640 },
height: { ideal: 480 },
facingMode: "user",
},
audio: false,
});
// Lock exposure and white balance (critical for rPPG signal quality)
const track = stream.getVideoTracks()[0];
const caps = track.getCapabilities();
if (caps.exposureMode)
await track.applyConstraints({ advanced: [{ exposureMode: "manual" }] });
if (caps.whiteBalanceMode)
await track.applyConstraints({ advanced: [{ whiteBalanceMode: "manual" }] });
import { FaceLandmarker, FilesetResolver } from "@mediapipe/tasks-vision";
const vision = await FilesetResolver.forVisionTasks(
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
const faceLandmarker = await FaceLandmarker.createFromOptions(vision, {
baseOptions: {
modelAssetPath: "face_landmarker.task",
delegate: "GPU",
},
runningMode: "VIDEO",
numFaces: 1,
outputFaceBlendshapes: true,
outputFacialTransformationMatrixes: true,
});
// POS (Plane Orthogonal to Skin) — Wang et al., 2017
// Input: temporal RGB signals [N, 3], Output: BVP signal [N]
for each sliding window (1.6s = 32 frames at 20fps):
1. Temporal normalize: divide R,G,B by their window means
2. Project onto POS plane:
S1 = Gn - Bn
S2 = -2*Rn + Gn + Bn
3. Alpha = std(S1) / std(S2)
4. BVP_window = S1 + alpha * S2
5. Overlap-add into output buffer
Then: Butterworth bandpass 0.7-4.0 Hz (42-240 BPM)
Zero-phase filtering (filtfilt)
// 4th-order Butterworth bandpass for rPPG
// Passband: 0.7 Hz (42 BPM) to 3.5 Hz (210 BPM)
// Use pre-computed coefficients for your exact sample rate
// For 30 fps, order 4:
const b = [0.0226, 0, -0.0905, 0, 0.1357, 0, -0.0905, 0, 0.0226];
const a = [1.0, -4.4531, 9.2087, -11.3908, 9.1655, -4.8527, 1.6524, -0.3303, 0.0303];
// Apply forward pass, reverse, apply again = zero-phase (filtfilt)
// Use Float64Array for precision (Web Audio is only float32)
📄 Download the .MD file for the complete SDK decision matrix with full code snippets, comparison tables, and mobile SDK setup.
Advanced biomarkers requiring additional scan modes beyond standard facial rPPG. Each module uses a different sensor, camera position, or input method.
Electromagnetic frequency-based biomarkers using bone conduction headset and bioresonance analysis. These markers measure frequency deviations from healthy baselines, not direct physical measurements.
Frequency-based and camera-based biomarkers closing coverage gaps with traditional QRMA finger scanner devices. Includes digestive, hepatobiliary, skeletal, lipid, trace element, and new scan modalities.
Novel scanning modes using tongue imaging, nail bed analysis, breath patterns, body measurements, and mood assessment. These modules introduce entirely new input methods beyond camera, frequency, and voice.