Presage Pulse Rate for Spot Model Card


Model Details


Intended Use


Factors

The pulse metrics model first requires Mediapipe’s face detection algorithm to identify the face and 468 key feature points on the face (mesh model). Thus, if these features are not identifiable by Mediapipe’s algorithm, then pulse metrics will not be calculable. Reference the Mediapipe model cards can be found here: Full Range Face detection model card and lite pose detection:


These factors can affect model performance:


Other factors:



Metrics

  1. RMSD of point estimate of pulse rate: root mean square deviation between all aggregate measured values evaluated over a 30 second period of pulse rate and ground truth measurements. This is used because it is an aggregated measure of error that can be easily evaluated against alternative devices. For reference, the predicate device Oxehealth claims 1.81 bpm.
  2. MAE of point estimate and 95% CI of pulse rate: median absolute error or deviation of aggregate measurements as compared to ground truth. Unlike RMSD, MAE allows for interpretability of error and distribution of error and is more robust to outliers than mean evaluations.
  3. Mean proportion of returned values of pulse rate: for every 30 seconds of video, a single weighted measurement can be measured. Of all possible sets of 30 second clips within a video, this is a measure of the proportion of them that returned a valid measurement of pulse rate. The pulse rate model must be evaluated for accuracy and reliability. For reference, the predicate device Oxehealth claims 58% (95% CI 51% - 65%).

Evaluation Data

The evaluation data consists of a set of 233 videos. Corresponding quantities of pulse rate were measured from a Biopac research grade pulse oximeter. A clip of 30s from each video was run through the Presage aggregate pulse rate model for evaluation, leading to a total number of 812 samples. Videos were acquired on users covering a range of demographic variability, including age, gender and Fitzpatrick scale.


Quantitative Data

  1. RMSD of point estimate of pulse rate: 1.71 bpm
  2. MAE of point estimate and 95% CI of pulse rate: 1.11 bpm with 95% CI of [0.99, 1.23]
  3. Mean proportion of returned values of pulse rate and 95% CI: 0.71

Distribution of error figures:


Skin Tone (Fitzpatrick) % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
1 0.17 (136) 2.35 1.20 [0.80, 1.59] 0.91
2 0.13 (108) 1.29 1.03 [0.79, 1.27] 0.99
3 0.08 (69) 1.35 1.04 [0.72, 1.36] 0.99
4 0.15 (125) 1.56 1.17 [0.90, 1.44] 0.70
5 0.13 (102) 1.71 1.23 [0.89, 1.57] 0.52
6 0.15 (122) 2.14 1.22 [0.84, 1.60] 0.30


Sex % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
M 0.35 (283) 2.10 1.23 [0.99, 1.47] 0.75
F 0.47 (379) 1.49 1.06 [0.91, 1.21] 0.69


Camera Type % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
Android 0.46 (377) 1.56 1.09 [0.94, 1.25] 0.84
Econ 0.54 (435) 1.89 1.13 [0.95, 1.30] 0.60











Ethical Considerations

As a remote sensing device, the risks posed to the subjects in the trial are minimal, including the association of each subject with corresponding biometric data. Mitigation of these risks include de-identifying all subject data, including videos, prior to saving it. Additionally, all data is securely stored in a HIPPA compliant database with access to a select number of trained researchers.

The model is not intended for human life-critical decisions, diagnostics or prognostication.

Limitations and Tradeoffs