Case Study - Building a Class II Medical Device That Examines Patients Through a Webcam
A computer-vision and speech-processing platform, registered and listed with the FDA as a Class II medical device, that guides patients through neurological and speech exercises from their own computer or phone — built and run under a quality management system stood up for it.
- Client
- Modality.AI
- Year
- Service
- Regulated Medical Device (SaMD)
- My role
- Director of Product & Engineering
I led product, engineering and the regulatory work for a multimodal assessment platform registered with the FDA — standing up the quality management system it is built under, and adding the finger-tapping assessment that made Parkinson’s disease progression measurable from an ordinary webcam.
The challenge
Neurological and psychiatric conditions are tracked largely through examinations a specialist performs in person: watching how a patient speaks, moves and responds. Doing that remotely, at scale and consistently enough to measure change over time is what Modality set out to do — through the webcam on a patient's own computer or phone.
That ambition carries a regulatory consequence. Software that analyzes a patient's video and audio to produce clinical measurements sits squarely inside FDA device regulation; it can't lean on the exemptions that cover clinical decision support. So the product had to be designed, built, changed and documented the way a medical device is — while still moving at the pace of a software company.
My role
As Director of Product and Engineering, I led the product and engineering teams and the regulatory work, including standing up and implementing the quality management system the device was built under. I also built the finger-tapping capability myself.
Leadership decisions
Decision 1
Build the quality system into how the team works, not alongside it.
Instead of: Shipping like a typical software startup and assembling the regulatory paperwork afterwards.
A medical device is built under a quality management system: a documented process for how requirements are set, how the design is verified, how risks are managed and how every change is controlled. I stood that system up and implemented it as the way product and engineering actually worked, so each release carried its own evidence rather than needing it reconstructed later, and a regulated product could keep shipping.
Decision 2
Measure the exam a clinician would give — not just whatever the camera records.
Instead of: Passively recording patients and searching the data for signals afterwards.
The platform's guide leads patients through structured exercises modeled on what a speech-language pathologist or neurologist asks for in person. That keeps each measurement tied to a task clinicians already recognize and trust, which matters both for how its results are read and for how the device is described to regulators.
Decision 3
Bring a standard Parkinson’s motor test to an ordinary webcam.
Instead of: Requiring wearables or specialized hardware to capture movement.
Finger tapping is part of the standard clinical examination for Parkinson's disease, and the speed and rhythm of the taps change as the disease progresses. I built a finger-tapping assessment using Google's MediaPipe hand tracking, so the movement could be measured from the same webcam the patient already uses for the rest of the session — no extra hardware to ship, charge or explain.
The solution
A patient opens a session on their computer or phone, and Tina, the platform's virtual guide, walks them through a series of speech, facial and movement exercises. The platform records the session and extracts audio and visual measurements from it, giving clinicians and research teams a consistent, repeatable assessment that can be run in clinic or at home, as often as the care or the study requires.
A finger-tapping test a neurologist gives in the clinic, measured from the webcam a patient already owns.
Impact
- Parkinson’s progression became measurable remotely. The finger-tapping assessment unlocked the platform's ability to assess Parkinson's disease progression without an in-person visit.
- A registered Class II device, built under its own quality system. The platform is registered and listed with the FDA as a Class II medical device, built and changed under the quality management system I stood up.
- One platform across neurology and psychiatry. The same assessment approach is used in research and care across Parkinson's disease, ALS and FTD, schizophrenia, mild cognitive impairment, depression, autism and ADHD, Huntington's disease, multiple sclerosis and other conditions.
Under the hood — the technical detail
Sessions are captured through the browser or phone camera and microphone, and the platform derives audio and visual measurements — from speech, facial movement and motor tasks — from the recordings. The finger-tapping assessment uses Google's MediaPipe hand-landmark tracking to follow the fingers frame by frame and derive the characteristics of the tapping movement over the course of the task.
- FDA Class II medical device
- Quality management system
- Computer vision (MediaPipe)
- Speech and audio analysis
- Remote neurological assessment