VoicingElder is a reminiscence storytelling platform designed for older adults, using interactive virtual puppets. Storytelling and reminiscence nurture intergenerational sharing and communication, allowing seniors to express and affirm their identities as they review and share their memories. In VoicingElder, the puppeteer (the older adult user) controls an on-screen virtual puppet (avatar) through speech and gesture, enacting digitally embodied therapeutic puppetry.The virtual puppet mimics their gestures (head, hands, eyebrows…), via Microsoft Kinect camera and microphone in real time. Users act out representations of themselves or significant others at various ages (child, teenager, adult, elder), shaping their life history in an engaging way.

The project strives to enhance seniors’ quality of life by engaging with their memories and emotions through virtual puppets. By putting themselves in the position of a puppeteer watching their own puppet performance on screen, they can develop their life story new perspectives, revealing deep memory in “thick” documentation. Those unique life histories will help the older adults’ better communication with their family members (inter-generationally), caregivers and friends.

VocingElder will detect the emotional contents of the older adult’s live speech contents (Natural Language Processing-AI algorithm developed by Dr. Faralli: sad, happy, angry, positive, negative….) and use it to inform the virtual puppet’s facial expressions, behaviors, and background sounds, to facilitate their immersion into affective memory.

This project turns the senior’s life review sessions into a playful virtual puppet performance that can be viewed by family members and even subsequent generations in a live or recorded format. VocingElder supports the beauty, wisdom and experiential knowledge of older adults’ oral stories, and promotes cross disciplinary discussions between many areas: art, gerontology, health, technology, psychology, theater, etc. .

 

3ppl