Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home
We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person’s energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.
KeywordsGround Truth Activity Recognition Calorific Expenditure Ground Truth Label Slide Window Method
This work was performed under the SPHERE IRC project funded by the UK Engineering and Physical Sciences Research Council (EPSRC), Grant EP/K031910/1.
- 3.Cosmed K4b2. http://www.cosmed.com/
- 8.Woznowski, P., et al.: A multi-modal sensor infrastructure for healthcare in a residential environment (2015)Google Scholar
- 12.Tao, L., Burghardt, T., Hannuna, S., Camplani, M., Paiement, A., Damen, D., Mirmehdi, M., Craddock, I.: A comparative home activity monitoring study using visual and inertial sensors. In: IEEE International Conference on E-Health Networking, Application and Services (2015)Google Scholar
- 13.Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding, pp. 675–678 (2014)Google Scholar
- 14.Oreifej, O., Liu, Z.: Hon4d: Histogram of oriented 4D normals for activity recognition from depth sequences, pp. 716–723 (2013)Google Scholar
- 16.Laptev, I., Marszałek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies, pp. 1–8 (2008)Google Scholar
- 18.Ryoo, M., Rothrock, B., Matthies, L.: Pooled motion features for first-person videos, pp. 896–904 (2015)Google Scholar
- 21.Edgcomb, A., Vahid, F.: Estimating daily energy expenditure from video for assistive monitoring, pp. 184–191 (2013)Google Scholar
- 22.Tsou, P.F., Wu, C.C.: Estimation of calories consumption for aerobics using kinect based skeleton tracking, pp. 1221–1226 (2015)Google Scholar
- 23.OpenNI organization: OpenNI User Guide (2010)Google Scholar
- 25.Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection, vol. 1, pp. 886–893 (2005)Google Scholar