Music for Alice Dali Augmented Reality Experience: Multisensory Design Soundscapes for Locative Mobile Phone Gaming (via Synaesthesia)

Abstract

Composing music for digitally enhanced realities (AR/VR) requires synchronisation of the audio and visual design efforts. The recent book Multisensory Experiences: where the senses meet technology (Velasco and Obrist, 2020, p. 70) pointed out the necessity of following the multisensory nature of perception. In this project, our core hypothesis is that the quality of locative Augmented Reality (AR) Art/Music experiences can benefit considerably from improving the associated design processes, specifically by adopting multisensory design practice modelling synaesthesia in composing music for visual image. This paper offers insights into methodology of multisensory design soundscapes for Augmented Reality Alice Dali experience, based on 12 images painted by Salvador Dali for an 1873 special edition of Lewis Carroll’s book “Alice’s Adventures in Wonderland.” Music composed by Svetlana Rudenko interprets the Art shapes of lines, characters and emotional perception of colour into musical textures and genres. The narration of text by Mads Haahr adds a verbal component into the AR soundscape. Research on artificially induced synaesthesia has shown that “cognitive training including synesthetic associations may in the future be a promising new tool for vulnerable clinical groups to enhance general mental ability” (Bor et al, 2018). For this reason, multisensory design applications based on a synaesthesia model might have a very promising potential for training creativity, thinking outside the box, education and mental health. The Augmented Reality Alice Dali app will be available for download for free for Android and iOS. Prototype: https://rebrand.ly/alice-dali-ar-prototype