Presage Breathing Metrics for Spot Model Card


Model Details


Intended Use


Factors

Breathing metric model first requires Mediapipe’s face detection algorithm to identify the face and pose of the subject. Thus, if these features are not identifiable by Mediapipe’s algorithm, then breath metrics will not be calculable.
These factors can affect the ability to detect the subject's face and pose. Reference the Mediapipe model cards can be found here: Full Range Face detection model card and lite pose detection:


These factors can affect model performance:


Other factors:



Metrics

  1. RMSD of point estimate of breath rate: root mean square deviation between all aggregate measured values evaluated over a 30 second period of breath rate and ground truth measurements. This is used because it is an aggregated measure of error that can be easily evaluated against alternative devices. For reference, the predicate device Oxehealth claims 1.17 bpm RMSD.
  2. MAE of point estimate and 95% CI of breath rate: median absolute error or deviation of aggregate measurements as compared to ground truth. Unlike RMSD, MAE allows for interpretability of error and distribution of error and is more robust to outliers than mean evaluations.
  3. Mean proportion of returned values of breath rate: for every 30 seconds of video, a single weighted measurement can be measured. Of all possible sets of 30 second clips within a video, this is a measure of the proportion of them that returned a valid measurement of breath rate. The breath rate model must be evaluated for accuracy and reliability. For reference, the predicate device Oxehealth claims 73% (95% CI 68% - 79%).

Evaluation Data

The evaluation data consists of a set of 223 videos. Corresponding quantities of breath rate were measured from a Biopac research grade strain gauge breathing sensor. A clip of 30s from each video was run through the Presage breath rate model for evaluation and a single measurement returned, leading to a total number of 642 samples. Each video was acquired on a different user covering a range of demographic variability, including age, gender and Fitzpatrick scale.


Quantitative Data

  1. RMSD of point estimate of breath rate: 2.34 bpm
  2. MAE of point estimate and 95% CI of breath rate: 1.54 bpm with 95% CI of [1.36, 1.72]
  3. Mean proportion of returned values of breath rate and 95% CI: 0.65

Distribution of error figures:


Skin Tone (Fitzpatrick) % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
1 0.18 (115) 2.60 1.70 [1.23, 2.17] 0.68
2 0.14 (91) 2.30 1.59 [1.12, 2.06] 0.67
3 0.10 (61) 1.98 1.44 [0.95, 1.94] 0.51
4 0.15 (97) 1.96 1.28 [0.89, 1.67] 0.58
5 0.12 (74) 2.41 1.62 [1.14, 2.10] 0.72
6 0.15 (94) 1.92 1.29 [0.90, 1.69] 0.73


Sex % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
M 0.36 (232) 2.40 1.56 [1.26, 1.86] 0.62
F 0.47 (300) 2.12 1.45 [1.21, 1.69] 0.68


Camera Type % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
Android 0.49 (315) 2.48 1.64 [1.37, 1.91] 0.74
Econ 0.51 (327) 2.14 1.42 [1.18, 1.65] 0.57











Ethical Considerations

As a remote sensing device, the risks posed to the subjects in the trial are minimal, including the association of each subject with corresponding biometric data. Mitigation of these risks include de-identifying all subject data, including videos, prior to saving it. Additionally, all data is securely stored in a HIPPA compliant database with access to a select number of trained researchers.

The model is not intended for human life-critical decisions, diagnostics or prognostication.

Limitations and Tradeoffs