A MATLAB Toolbox for Photoplethysmography Imaging
by Christian S. Pilz,
Aachen, 2019
Version beta0.1
News
04.04.2022
I uploaded the full implementation of the diffusion process model [3,4].
An example implementation of a stochastic oscillator is added as well:
29.11.2021
Our work was recently featured in the following Nature article about digital medicine (npj Digit. Med.):
Dasari, A., Prakash, S.K.A., Jeni, L.A. et al… Evaluation of biases in remote photoplethysmography methods. npj Digit. Med. 4, 91 (2021).
https://doi.org/10.1038/s41746-021-00462-z
Supported by
Funded by
Introduction
During the last years measuring blood volume changes and heart rate measurements from facial images
gained attention at top computer vision conferences frequently. Most of these contributions focus
on how to cope with motion like head pose variations and facial expressions since any kind of motion on a specific
skin region of interest will destroy the raw signal in a way that no reliable information can be extracted anymore.
Beside from being able to estimate vitality parameters like heart rate and respiration, the functional survey of wounds
as well as quantification of allergic skin reaction are further topics of discovered employments of skin blood
perfusion analysis. Recently, prediction of emotional states, stress, fatigue and sickness became interesting new achievements in this area, pushing the focus of this technology further towards human-machine interaction.
In contrast to the genuine medical use-case of the technology, in computer vision and human-machine interaction we can’t expect any cooperative behavior of the user without introducing lack of convenience and a reduction of
the general user acceptance. Further, beyond any well tempered clinical and laboratory like scenarios, the majority
application will face strong challenging environmental changes and differences much more quite common. Thus,
there’s an emerging demand to produce better features and models significant more robust to nuisance factors, still
preserving the desired target information. To reach such a formulation a fundamental profound understanding of the
underlying optical and mathematical properties is one of the current foci of this research discipline.
Figure 1. Rigid and non-rigid facial motions act as nuisance factors
on the tiny blood volume changes inherently destroying the target
information of heart rate under the conventional formulation of the
problem. Utilizing features invariant with respect to the action of the group of nuisance
transformations making it possible to estimate heart rate information under everyday facial motions. As illustrated for the above head motions, the green channel information doesn’t yield to reasonable heart rate information in the frequency domain. However, in case we're able to extract invariant features we obtain a clear signal in both time
and frequency domain.
Example Data
The example data can be download by the following link:
https://www.dropbox.com/s/vv6ethy5az16wt4/example_data.mat?dl=0
(NOTE: Leave me a message in case the file isn’t available anymore via dropbox!)
Alternative download location: https://gw.cancontrols.com/LGI_DATABASE/toolbox/example_data.mat
Place the example_data.mat file into ./media/data/ folder.
The example_data.mat contains the reference finger pulse oximeter waveform (PPG)
and the color image data of a face finder detection result in the rgb cell.
Tests
In the tests directory there’re separate example Matlab scripts. One for each algorithm respectively.
Initially the startup.m script must be executed to add all needed directories.
As second step executing test_skin.m will perform skin segmentation on the example_data.mat in case it was
downloaded and placed into the ./media/data/ folder. The obtained skin pixels will be stored into the example_data.mat file
and can be reused to test all other algorithm scripts.
Algorithms
- Channel Mean (G) [8,9]
- Spatial Subspace Rotation (SSR) [6]
- Algorithmic Principles of Remote PPG (POS)[5]
- Local Group Invariance (LGI) [1,2]
- Diffusion Process (DP) [3,4]
- Riemannian-PPGI (SPH) [1]
Evaluation
- Correlation
- Bland-Altman
- RMSE/ MSE
- SNR [7]
Databases
References
- Christian S. Pilz, Ibtissem Ben Makhlouf, Vladimir Blazek, Steffen Leonhardt,
On the Vector Space in Photoplethysmography Imaging,
The IEEE International Conference on Computer Vision (ICCV) Workshops,
Seoul/ South Korea, 2019
- Christian S. Pilz, S. Zaunseder, J. Krajewski, V. Blazek,
Local Group Invariance for Heart Rate Estimation from Face Videos in the Wild,
The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1254-1262, Salt Lake City, 2018
- Christian S. Pilz, Jarek Krajewski, Vladimir Blazek.
On the Diffusion Process for Heart Rate Estimation from Face Videos under Realistic Conditions.
Pattern Recognition: 39th German Conference, GCPR 2017, Basel, Switzerland.
Proceedings (Lecture Notes in Computer Science), pp. 361-373, Springer, 2017
- Christian S. Pilz, Sebastian Zaunseder, Ulrich Canzler, Jarek Krajewski.
Heart rate from face videos under realistic conditions for advanced driver monitoring.
Current Directions in Biomedical Engineering, De Gruyter, Berlin, pp. 483–487, 2017.
- Wang, W., den Brinker, A. C., Stuijk, S., & de Haan, G. (2017). Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering, 64(7), 1479-1491
- W. Wang, S. Stuijk and G. de Haan, “A Novel Algorithm for Remote Photoplethysmography: Spatial Subspace Rotation,” in IEEE Transactions on Biomedical Engineering, vol. 63, no. 9, pp. 1974-1984, Sept. 2016.
- De Haan, G., & Jeanne, V. (2013). Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering, 60(10), 2878-2886
- Verkruysse, W., Svaasand, L. O., & Nelson, J. S. (2008). Remote plethysmographic imaging using ambient light. Optics express, 16(26), 21434-21445
- M. Hülsbusch. A functional imaging technique for opto-electronic assessment of skin perfusion. PhD thesis, RWTH Aachen University, 2008.
- S. Bobbia, R. Macwan, Y. Benezeth, A. Mansouri, J. Dubois, “Unsupervised skin tissue segmentation for remote photoplethysmography”, Pattern Recognition Letters, 2017.
License
If you use this toolbox, please cite this paper:
Christian S. Pilz, Ibtissem Ben Makhlouf, Vladimir Blazek, Steffen Leonhardt,
On the Vector Space in Photoplethysmography Imaging,
The IEEE International Conference on Computer Vision (ICCV) Workshops,
Seoul/ South Korea, 2019
Results
Evaluation was conducted on the following databases:
- UBFC-RPPG
- LGI Multi Session
Overview
Comparison of heart rate prediction accuracy utilizing different feature operators on diverse face video
databases. Each corresponding table entry represents the average Pearson’s correlation coefficient together with the
average root-mean-square error (RMSE) value in BPM.
| Database |
Green [7,8] |
SSR [5] |
POS [3] |
LGI [2] |
SPH [1] |
| UBFC |
0.16/22.1 |
0.54/4.95 |
0.68/4.42 |
0.75/5.94 |
0.73/3.21 |
| LGI Resting |
0.41/2.61 |
0.49/1.99 |
0.41/2.10 |
0.69/1.41 |
0.71/1.49 |
| LGI Rotation |
0.15/13.2 |
0.06/10.9 |
0.12/5.32 |
0.67/1.92 |
0.56/2.54 |
| LGI Talk |
0.15/46.6 |
0.12/27.8 |
0.01/37.7 |
0.51/14.7 |
0.23/27.8 |
| LGI Gym |
0.01/33.5 |
0.03/21.2 |
0.15/12.2 |
0.42/2.65 |
0.26/3.65 |
In the following the box plot statistics for the UBFC and the LGI database are visualized. In addition to previous table the
influence of the Diffusion process [3,4] incorporated into the LGI and SPH approach is constituted. Both approaches benefit
from its stochastic interpretation as quasi-periodic nature of blood volume changes in contrast to the traditional standard Fourier based spectral peak search.
UBFC-RPPG:
| Pearson |
RMSE |
 |
 |
LGI Multi Session:
- Session 1: Head Resting
| Pearson |
RMSE |
 |
 |
- Session 2: Head Rotation
| Pearson |
RMSE |
 |
 |
- Session 3: Bicycle Ergometer
| Pearson |
RMSE |
 |
 |
- Session 4: Outdoor City Talk
| Pearson |
RMSE |
 |
 |
评论