Presage PRQ Metrics for Spot Mode Model Card


Model Details


Intended Use


Factors

The PRQ model first requires Mediapipe’s face detection algorithm to identify the face and 468 key feature points on the face (mesh model). Thus, if these features are not identifiable by Mediapipe’s algorithm, then PRQ will not be calculable. Reference the Mediapipe model cards can be found here: Full Range Face detection model card.


These factors can affect model performance:


Other factors:



Metrics

  1. RMSD of point estimates of PRQ: root mean square deviation between all PRQ values measured within a 30 second period and ground truth measurements evaluated over the same window.
  2. MAE of point estimate and 95% CI of PRQ: median absolute error or deviation of all measures of PRQ as compared to ground truth.
  3. Mean proportion of returned values of PRQ: for every 30 seconds of video, a single weighted measurement can be measured. Of all possible sets of 30 second clips within a video, this is a measure of the proportion of them that returned a valid measurement of PRQ.

Evaluation Data

The evaluation data consists of a set of 225 videos. Corresponding quantities of PRQ were measured from a Biopac research system which quantified heart rate from 3 lead ECG and breath rate from a strain gauge sensor centered on the chest. A clip of 30s from each video was run through the Presage PRQ model for evaluation, leading to a total number of 749 samples. Videos were acquired on users covering a range of demographic variability, including age, gender and Fitzpatrick scale.


Quantitative Data

  1. RMSD of point estimate of PRQ: 1.11
  2. MAE of point estimate and 95% CI of PRQ: 0.61 with 95% CI of [0.53, 0.69]
  3. Mean proportion of returned values of PRQ and 95% CI: 0.26

Distribution of error figures:


Skin Tone (Fitzpatrick) % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
1 0.18 (135) 1.09 0.78 [0.59, 0.96] 0.37
2 0.14 (104) 1.13 0.65 [0.44, 0.86] 0.43
3 0.09 (68) 0.56 0.34 [0.21, 0.48] 0.19
4 0.16 (119) 0.53 0.38 [0.28, 0.47] 0.29
5 0.11 (84) 0.77 0.52 [0.38, 0.66] 0.29
6 0.14 (109) 0.50 0.30 [0.22, 0.38] 0.06


Sex % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
M 0.36 (270) 1.04 0.64 [0.52, 0.76] 0.30
F 0.46 (349) 0.80 0.53 [0.45, 0.61] 0.27


Camera Type % of Dataset (num samples) RMSD MAE [95% CI] Mean Return Rate
Android 0.49 (366) 1.00 0.62 [0.52, 0.72] 0.25
Econ 0.51 (387) 1.19 0.61 [0.49, 0.72] 0.26











Ethical Considerations

As a remote sensing device, the risks posed to the subjects in the trial are minimal, including the association of each subject with corresponding biometric data. Mitigation of these risks include de-identifying all subject data, including videos, prior to saving it. Additionally, all data is securely stored in a HIPPA compliant database with access to a select number of trained researchers.

The model is not intended for human life-critical decisions, diagnostics or prognostication.

Limitations and Tradeoffs