Stairway to elders

Bridging space, time and emotions in their social environment for wellbeing

Project funded by Fondazione Cariplo with a grant on aging and social research: people, places and relations.

Phuse Lab, Università degli Studi di Milano
Malga, Università di Genova
Università Vita-Salute San Raffaele
Fondazione Cariplo

Context

In 1947 the World Health Organisation defined health as “a state of complete physical, mental and social wellbeing”. Although physical wellbeing has been a priority of the vast majority of national health services, the equal importance of the emotional sphere of an individual has been recently emphasised. Emotional wellbeing is related to the sense of fulfilment; it includes satisfaction, optimism, having a purpose in life as well as being able to make the most of your abilities to cope with the normal challenges of life. An increasing body of research suggests that initiatives promoting physical wellbeing disregarding mental and social wellbeing may lead to failure.

Emotional wellbeing

Emotional wellbeing together with health, is a key factor for aging well. The psychological stability is often undermined over the years, when older people have to face more and more life challenges than weaken their independence and self-confidence.

Social interaction

Social interaction is a key aspect influencing the wellbeing of elderly’s life. Its lack may produce cognitive deconstruction, which includes emotion suppression, altered sense of time, poor planned actions and mobility, decline of meaningful thought.

The main goal of the project is to design methods for automatically evaluate the emotional wellbeing of elderly, including novel computational models and testing procedures to assess the emotional state in an unconstrained, unbiased and ecological way.

Multimodality

to cover appropriate space and time, without any restriction to the person way of life

Adaptive continous analysis

to identify subtle changes in the normal way of life

Mild interventions

to the space and time dimensions, to improve comfort and stimulate positive feelings

Personalization

of the intervention with respect to a specific person or a particular mental state

Scientific background

The team provides complementary competences deployed in this project to achieve its main goal: designing and validating methods for continuous monitoring from multiple source of data of the emotional wellbeing and social interaction level of aging people, with strategic interventions from the environment to help them to get back both to positive emotions, in case of low-mood detection, and to a good level of social interaction.

  • Aging population monitoring and Ambient Assistive Living

    Assistive environments and the potential benefit for aging population have been discussed in [Noceti2017; Martini2018] where we present a feasibility study for a shortstay post-hospitalization protected residence, built in the vicinity of a hospital, and designed as a smart environment with all the comforts of a private home.

  • Motion information is the key element of the computation of a mobility index, we introduced in [Noceti2017], to evaluate the level of motility of elder people on the long period and in ecological conditions. The core functionalities of the proposed system are based on computer vision, and contribute to the assessment of the so-called frailty index, used by physicians to monitor the overall wellbeing of patients. Of relevance for this project are also previous work on motion understanding based on methods that capture human motion regularities. In [Noceti2017b], we introduced a computational model based on the Two-Thirds Power Law, and show how it can be used for visual action recognition; a multi-scale temporal extension of such model is discussed in [Vignolo2017] where we show its ability in characterizing the complexity and variability of a large class of movements. In [Malafronte2017] we investigated the use of space-time primitives to understand human movements and discuss the tolerance to view-point changes of the obtained representation.

  • In [Cuculo2014] we present a facial landmark localization, demonstrating its effectiveness in presence of different expressions. In [D’Amelio2017] we design a model to pass from facial landmarks to the corresponding AU activations, exploiting the CANDIDE-3 face model. In [Boccignone2018] the processing of physiological data is discussed. After denoising, the wavelet transform is adopted to compensate the difference in activation time and persistence of each signal. Moreover, a in [Boccignone2017] a Multimodal dataset for HUmour SEnsing, AMHUSE, is presented along with a novel web-based annotation tool named DANTE (Dimensional ANnotation Tool for Emotions). The dataset collects the reactions of 36 subjects in presence of 3 amusing and 1 neutral video stimuli. Gathered data include RGB video and depth sequences along with physiological responses (electrodermal activity, blood volume pulse, temperature). The videos are annotated in terms of valence and arousal continuous dimensions.

  • Full-body movements description is at the basis of the system for emotion recognition we proposed in [Piana2016]. Information on kinematics and joints motion is used to extract expressive features inspired by psychology and humanistic theories, as e.g. contraction index, fluidity, and impulsiveness. In [Boccignone2018] we introduce a Bayesian framework based on deep latent representation of a continuous affective space which involves both facial cues and autonomic, physiological signals.

  • We have investigated speed perception of ordinary video clips in adults [de’Sperati2017; Rossi2018] and children [Zuliani2018], finding systematic and robust speed biases, which change significantly during childhood. We have put forth the hypothesis that one’s own motor rhythms can affect the sense of speed for external scenes, which in turn suggests that elders may prefer watching slower (or in some cases faster) TV programs.

Latest activities

Learning dictionaries of kinematic primitives for action classification
Conference paper

25th International Conference on Pattern Recognition

accepted paper
Single view learning in action recognition
Conference paper

25th International Conference on Pattern Recognition

accepted paper
HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty
Conference paper

IEEE/CVF Winter Conference on Applications of Computer Vision

accepted paper
Exploring Fusion Strategies in Deep Multimodal Affect Prediction
Conference paper

International Conference on Image Analysis and Processing

accepted paper
A quantitative evaluation framework of video de-identification methods
Conference paper

25th International Conference on Pattern Recognition

accepted paper
Social traits from stochastic paths in the core affect space
Conference paper

EAI International Conference on Pervasive Computing Technologies for Healthcare

accepted paper
Head Pose Estimation with Uncertainty
Journal article

Pattern Recognition

in review
The moca dataset, kinematic and multi-view visual streams of fine-grained cooking actions
Journal article

Scientific Data

accepted paper
Cross-view action recognition with small-scale datasets
Journal article

Image and Vision Computing

accepted paper
If you are old, videos look slow. The paradoxical effect of age-related motor decline on the kinematic interpretation of visual scenes
Journal article

Frontiers Human Neuroscience

accepted paper
Quasi-3D: Reducing vergence effort to improve visual comfort of stereoscopic displays
Journal article

Virtual Reality

in review
Enhancing visual exploration through augmented gaze: high acceptance of immersive virtual biking by oldest olds
Journal article

International Journal of Environmental Research and Public Health

in review
An open framework for remote-PPG methods and their assessment
Journal article

IEEE Access

accepted paper
Improving the accuracy of automatic facial expression recognition in speaking subjects with deep learning
Journal article

MDPI Applied Sciences

accepted paper
CARE2020 workshop
Stairway to Elders closing workshop
Workshop

Final Meeting among partners, stakeholders and collaborators

CARE2020 workshop
CARE 2020
Workshop

International Workshop on Pattern Recognition for Positive Technology and Elderly Wellbeing

Stairway to Elders: Bridging Space, Time and Emotions in Their Social Environment for Wellbeing
Conference paper

9th International Conference on Pattern Recognition Applications and Methods

accepted paper
Positive technology for elderly well-being: A review
Journal article

Pattern Recognition Letters

doi:10.1016/j.patrec.2019.03.016
Cortical Resonance to Visible and Invisible Visual Rhythms
Journal article

Brain Sciences

doi:10.3390/brainsci10010037

Research team

Raffaella Lanzarotti

Raffaella Lanzarotti

Principal Investigator

Nicoletta Noceti

UniGe Research Unit Coordinator

Claudio de'Sperati

UniSR Research Unit Coordinator

Giuseppe Boccignone

UniMi Research Unit member

Francesca Odone

UniGe Research Unit member

Giuliano Grossi

Dissemination coordinator