Generative AI for the Sound Arts and Music Performance
Investigate the potential of Generative AI through an interdisciplinary project that explores the application of AI for Sound Art creation.
Fiche signalétique
- Département responsable Technique et informatique
- Autres départements Haute école des arts de Berne
-
Institut(s)
Institut Interprétation
Institute for Data Applications and Security (IDAS) -
Unité(s) de recherche
Intersections de la musique contemporaine
IDAS / Applied Machine Intelligence - Organisation d'encouragement BFH
- Durée (prévue) 01.10.2022 - 31.01.2023
- Responsable du projet Prof. Dr. Souhir Ben Souissi
- Direction du projet Prof. Dr. Souhir Ben Souissi
-
Équipe du projet
Prof. Dr. Teresa Carrasco
Franziska Baumann - Mots-clés Generative AI, Novel research, Sound Art, Augmented Intelligence
Situation
Generative AI is taking the academic and industrial world by storm. The ability of Deep Learning architectures such as Transformers to sustain textual conversations with humans, produce realistic and surrealistic images from textual descriptions or generate plausible chemical compositions for drug discovery are merely the first few prominent examples. Here at BFH, our research and educational efforts around AI and Deep Learning has thus far focused on Computer Vision and NLP (Natural Language Processing) for classification, segmentation, regression, prediction and decision making. With this project we aim to expand our portfolio to content generation with an intriguing and interdisciplinary case-study: The use of Generative AI for the Sound Arts.
Approche
The project will run as a collaboration between the Engineering and Art Departments of BFH. From a computational perspective, we will explore: • Generation of music lyrics through transfer learning and LLMs (Large Language Models) in both co-operative (human/machine) and semi-independent mode (artificial lyrics produced from an initial seed). • Generation of MIDI music scores for different music genres, using RNNs (Recursive Neural Networks) and Transformers. Exploring both offline generation (with longer inference times) and on-line (with real-time segments generated during a performance). • Offline and real-time generation of contextual visualizations (images and video sequences) using transfer learning and diffusion models, that can accompany live music performances.
