Designing and Performing with Live Coding Languages for Signal Processing and Machine Intelligence on the Web

Francisco Bernardo (University of Sussex), Chris Kiefer (University of Sussex), Thor Magnusson (University of Sussex)

Abstract

This workshop will give participants the opportunity to learn how to design and perform with their own live coding mini-languages, based on web audio technology. Participants will get familiar with our live coding language design environment and learn language design techniques for signal processing, machine listening, and selected machine learning models. Users will engage hands-on with our user-friendly and web-based toolkit Sema, which provides an accessible entry point to applying these techniques to the exciting domain of live coding musical performance. The workshop will include group discussion on how participants experienced the system and the techniques, and brainstorming session with suggestions for improvement and desired new features. The workshop is part of the MIMIC research project, which explores how machine learning and machine listening can be designed in user-friendly way and provide for rapid prototyping and fast development cycles of musical applications.

Authors’ short bios

Thor Magnusson is a worker in rhythm, frequencies and intensities. His research interests include musical improvisation, new technologies for musical expression, live coding, musical notation and digital scores, artificial intelligence and computational creativity, programming education, and the philosophy of technology. These topics have come together in the ixiQuarks, ixi lang, and the Threnoscope live coding systems he has developed. As well as performing and writing about music, he lectures in music at the University of Sussex, Brighton.

Chris Kiefer is a computer-musician and musical instrument designer, specialising in musician-computer interaction, physical computing, and machine learning. He performs with custom-made instruments including malleable foam interfaces, touch screen software, interactive sculptures and a modified self-resonating cello. Chris’ work also concentrates on machine learning and signal processing for audio and interaction, with a particular emphasis on nonlinear and dynamical systems.

Francisco Bernardo is an interactive media artist, a software engineer and a scholar. His research is focused on human-computer interaction approaches to toolkits that broaden and accelerate user innovation with interactive machine learning. Francisco has been working in applied research in projects at the intersection of art and innovation, front-end software engineering, interaction design and greenfield product management.

More info

• For additional information about the workshop, visit Extended Abstract "Designing and Performing with Live Coding Languages for Signal Processing and Machine Intelligence on the Web" in PDF format.

Live coding with Makkeróni

Balázs Kovács (University of Pécs)

Abstract

Makkeróni is a web-based live coding system which simulates a linux shell. The user operates with commands known from the BASH (ls, help, cat etc.), and also there are some audio-specific commands (play, freq, tone etc.). In this workshop we're going to learn how to use it, while possibly focusing on efficiency (pipes, joker characters, command history tricks) and network cooperation. If the participants are open for it, we could discuss the targets of further development as well. It's recommended for both new and experienced Linux shell users.

Author's short bio

Balázs Kovács, PhD habil., philosopher, holds a PhD in aesthetics of interactive sonification. Head of the Electronic Music and Media Arts programme at University of Pécs, Faculty of Arts. Founder of the Hangfarm (Soundfarm) open-air media art exhibition place in Ellend, Hungary.

More info

• For additional information about the workshop, visit Extended Abstract "Live coding with Makkeróni" in PDF format.

Workshop on Diversity

Miranda Moen (Balansekunst/The Art of Balance), Victoria Øverby Steinland (Balansekunst/The Art of Balance), Andrea Hegdahl Tiltnes (NTNU, Girl Geek Dinners Trondheim) and Kari Dahn (Girl Geek Dinners Trondheim)

Abstract

In this workshop, two Norwegian organizations working with equality and diversity in technology and the arts sector, will present their experiences and work, and offer some tools to stimulate and achieve diversity in the workplace and communities such as Web Audio. The workshop will thus have two modules: one by Girl Geek Dinners Trondheim and one by The Art of Balance. The workshops will be conducted in English. Previous knowledge of diversity and other equality concepts will not be necessary. 

Speakers bios

Victoria Øverby Steinland (The Art of Balance) is developing Balansekunst's project against sexual harassment within culture and the arts. She holds an MSc in sociology, and has previously worked with projects on gender and sexual diversity, as well as teaching adolescents about sex, boundaries and consent.

Kari Dahn (Girl Geek Dinner Trondheim) works as a user experience consultant in Bouvet, and has been active in Girl Geek Dinners since the network started up in Trondheim. Dahn holds a Master's degree in Information design.

Workshop organizers

Miranda Moen is educated in aesthetics (University of Bergen) arts management (OsloMet), and graduated June 2019 with a master's in Equality and Diversity at the NTNU institute of interdisciplinary cultural studies. Affiliated with KOSO, the Art of Balance and AKKS.

Andrea Hegdahl Tiltnes is educated journalist (Nord University), and worked as a news reporter for some years after finishing her studies. Since 2014 she has worked as Communications Advisor at NTNU.

More info

• For additional information about the workshop, visit Extended Abstract "Workshop on Diversity" in PDF format.

Workshop on AudioWorklet

Paul Adenot (Mozilla)

Abstract

The AudioWorklet interface of the Web Audio API is used to supply custom audio processing scripts. User-supplied code is run in the AudioWorkletGlobalScope global execution context in a separate Web Audio rendering thread along with other nodes, allowing for zero-latency audio processing. In this workshop, we will look into the potentials of AudioWorklet, and to a number of angles to approach it from.

Author’s short bio

Paul Adenot is a platform engineer at Mozilla, working on the Firefox web browser. He is involved in Firefox Web Audio implementation, as well as the system audio code on all platforms, and occasionally helps on WebRTC. He also co-edits the Web Audio API specification at the W3C, and dabbles in music production.

Information for the participants

Each participant should bring: a development laptop with Linux, Windows or OSX, headphones, an installation of stable and development version of web browsers (Chrome stable and Canary, Firefox stable and Nightly).

More info

• For additional information about the workshop, visit Extended Abstract "Workshop on AudioWorklet" in PDF format.

Audiovisual Programming in Live Performance

Charlie Roberts (Worcester Polytechnic Institute)

Abstract

In this workshop, we will explore a variety of live coding technologies (including Gibber, Hydra, Livecodelab, among others) and discuss different approaches to multimodal programming / audiovisual synchronization with web technologies.

Author's short bio

Charlie Roberts is an Assistant Professor of Computer Science at Worcester Polytechnic Institute, with an affiliation in the Interactive Media & Game Development program. His research examines human-centered computing in digital arts practice. He designed and developed a creative coding environment for the browser, Gibber, that he uses both for educational research and audiovisual performances. Gibber is used to teach computational media to middle school, high school and university students in locations around the world, and Roberts has performed with it throughout the US, UK and Asia in the experimental performance genre known as live coding.

More info

• For additional information about the workshop, visit Extended Abstract "Audiovisual Programming in Live Performance" in PDF format.