So everyone can fully participate in everyday life!
What do wheelchairs, pedestrian lights with sound and hearing aids have in common? They all support millions of people worldwide who live with physical disabilities. Assistance systems like these are an indispensable part of their lives. Over the years, science has been able to make further and further advances that are constantly improving the lives of people with physical disabilities and, in future, even people with mental disabilities as well. The advances mainly relate to information technology aids, which often utilise forms of artificial intelligence. Assistance systems like these have one goal: to enable these people to participate in all aspects of life – in their own homes, in their leisure time and at work.
Like a video game
The Research Centre for Assistance and Participation (CAsPar) at the Rhine-Waal University of Applied Sciences is committed to contributing to this mission. The team led by Prof Dr Ing Christian Ressel, Professor for Ambient Intelligent Systems, is currently developing a digital showroom, i.e. a virtual environment in which assistance systems can be experienced. “Until now, you had to set up an entire living room, for example, in order to present an assistance system with all its sensors and devices,” says professor Ressel. The digital showroom makes setting up rooms with furniture and sensors on the computer based on personal preferences possible – a bit like in the video game “The Sims”. This virtual room can then be connected to an assistance system. Virtual reality glasses can be used to simulate what using these rooms and the assistance on offer would be like if they were real. For example, people with dementia can experience support from the interactive, sensor-based doll OurPuppet.
Building corporate partnerships
Another aim of the project is to establish and maintain contacts with partners in the region. As professor Ressel explains, “We want to approach companies that deal with assistance systems or already offer support services and establish new partnerships. Our aim is for them to benefit from our research and incorporate our solutions into their products.”
Late-breaking work
Tangents: Designing a System that Fosters Storytelling and Reading Through the Natural Use of Tangible Playthings
Tangents is an ongoing project exploring how children can engage with narratives through traditional, nonaugmented physical objects. The system enables children to create or extend stories through object manipulation or interact with existing narratives by reading and responding to AI-generated questions. Using computer vision and speech recognition, Tangents identifies children’s interactions with toys and employs generative AI to transcribe, expand, or generate questions about the stories. This approach offers a multisensory learning experience that supports various stages of reading development, including vocabulary acquisition, reading fluency and comprehension, while intrinsically motivating children to read. To ensure its effectiveness in fostering reading skills and engagement, the system follows a user-centered design approach, involving teachers and primary school children. Preliminary findings suggest that children have a positive acceptance of Tangents, showing its potential to foster literacy skills.
CAST Simulator: Simulating Digitally Augmented Reading in Virtual Reality
In recent years, the exploration of the design space for Digital Augmentation of Reading (DAR) has gained significant attention in diverse research pursuits, aiming to enhance reading comprehension, enjoyment, and learning. While a theoretical model has been proposed to assist designers in analysing DAR solutions, the lack of tools capable of systematically exploring specific configurations within the design space remains challenging. To address this gap, this paper introduces CAST Simulator, a virtual reality system designed to assist designers in crafting and experiencing DAR solutions. By immersing designers in interactive scenarios, the CAST Simulator facilitates a rapid design loop, enabling quick feedback from users. CAST Simulator aims to offer the capability to explore the design space, demonstrate existing DAR systems, and maintain synchronisation with the real developed system, allowing for continuous, rapid evaluation in a safe, virtual, yet realistic environment. We showcase the application of CAST Simulation in a classroom scenario, wherein a DAR system serves as a tool to foster reading comprehension and motivation. Concurrently, to maximise synergies and ensure a comprehensive understanding of requirements, the DAR system development is currently conducted through design thinking in a primary school. Finally, we conclude by discussing potential axes of evolution for the simulator.
VR classroom environment showcasing a digitally augmented reading system running on a tablet.
More information
Publication: Ribeiro, P., Büchner, M. A., & Ressel, C. (2024, October). CAST Simulator: Simulating Digitally Augmented Reading in Virtual Reality: Innovating the design process of creating Digitally Augmented Reading solutions in Virtual Reality. In Adjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction.
Poster presented at 2024 Nordic Conference on Human-Computer Interaction
Digital Augmentation in Special Needs Reading
This paper explores using Digital Augmentation of Reading – with multimedia like sound, illustrations, and light effects—to enhance reading engagement for special needs education. The initiative builds on an established Design-Based Research approach, previously successful in developing STREEN, a digitally augmented reading tool aimed at promoting reading comprehension and motivation among third-grade students. By engaging with stakeholders in special needs education, the project applies Design Space exploration and Design Thinking methodologies to tailor STREEN to this new context. A Design Thinking Club was also established to generate ideas for STREEN’s adaptation, while a Virtual Reality Simulator is currently in development to support rapid prototyping.
More information
Poster presented at ACM Interaction Design and Children (IDC) Conference in June 2024
