Mixed-Reality Platform for Simulation and Synthesis of Multi-Modal Hallucinations with Applications to Schizophrenia Treatment

Abstract

This project aimed at enhancing the Avatar Therapy method for schizophrenia hallucination treatment. While traditional antipsychotic medications remain the primary approach, one in three patients show resistance, making alternative treatments essential. The avatar therapy method externalizes hallucinations by creating a visual and auditory representation of the voices patients hear. This allows them to engage in direct dialogue with their hallucinations, often guided by a therapist. Patients who actively participate in creating their avatar may develop greater confidence in challenging their hallucinations, rather than solely relying on therapist-led interventions. In this project, a mixed-reality platform is developped enabling patients to create auditory and visual hallucinations on developed websites and finally interact with generated 3D avatar in a therapist-controlled Unity VR environment. A user study was conducted at a local hospital.