Here is my poster on musical neurosemantic decoding using a modified media annotation algorithm from Weston et al 2011
Beginner Sound Play Experience Studies for Higgles
The following three short pieces were designed to illustrate and highlight the many facets of musical experience. The first emphasizes sound play, the second involves constructing a sound environment and analyzing the variety of experience in that space, the third asks a solo listener to stretch the limits of their auditory attention. Not only do we engage with music in many different ways, but our understanding of music can be expanded by realizing the role that we play in constructing our own experience. These studies emphasize music as a process of embodied cognition and encourage a phenomenological analysis of musical experience. Each study takes place in a different place in Hallgarten Hall and is intended to be a tool for anyone working in Hallgarten to help them explore their capacity for musical experience.
**************************************************
1. For solo player in the conference room of Hallgarten Hall.
Instructions:
-
Play with two empty water jugs.
**************************************************
2. Guided improvisation for any reverberant restroom with ceiling fan and solo experiencer.
Instructions:
Experience, perception, and physicality in experimental music: An argument for the role of neuroscience in music phenomenology
Phenomenology is broadly defined as the study of the structures of experience. However, a more specific definition may describe phenomenology as one of several fields working together toward an understanding of human experience. For example, cognitive neuroscience seeks to uncover the neural mechanisms underlying human perception, while cultural studies analyzes the effect of social activities on experience. In music, there are several disparate fields that describe various aspects of musical experience, but some of these disciplines rarely interact. I suggest that a phenomenological understanding of music is a common goal among auditory cognitive neuroscience and a variety of other fields and that a collaborative research effort will lead to a better understanding of musical experience.
NIPS conference in Lake Tahoe in December
I’ll be presenting on “Musical neurosemantic decoding using probabilistic online weighted approximate-rank loss optimization in a joint semantic space” at the Neural Information Processing Systems (NIPS) workshop on Machine Learning and Interpretation in Neuroimaging (MLINI) in early December.
The MUNSE (Musical Understanding by NeuroSemantic Embedding) algorithm is an adaptation of MUSLSE (Music Understanding by Semantic Large Scale Embedding) which was originally described by Weston et al (2011).
Check back for a link to the talk/poster/paper soon.
Audio processing scripts ported to Python
Columbia’s Matt McVicar (along with others) ported Dan Ellis’ audio processing scripts from MATLAB to Python at the October 2012 Monthly Music Hackathon NYC. The code is here: https://github.com/bmcfee/librosa.
He also made a demonstration of what’s possible with this new library by making a tool to automatically “gear shift” any pop song. To hear samples and learn more about what Matt made, check out his blog post about it: http://www.mattmcvicar.com/music-hackathon-october-2012/.
ISMIR 2012
I’ll be leaving for Porto in just over a week to attend the conference of the International Society for Music Information Retrieval (ISMIR).
See you there!
- 30 Plays
This is a 2 second audio clip that I made from brain data (fMRI). The goal was to make audio that sounded like ‘Heavy Metal’ (à la Ozzy) given several examples of brains listening to music of this style. I used LDA to extract the archetypal brain response to this style, then used multivariate linear regression to predict brain activity for a large database of audio. The 5 audio clips whose predicted brain activity was most similar to the ‘Metal’ archetype were mixed (aka mashed up). This is the result.
Montreal Musitk
I’m getting excited for Musitk in Montreal this weekend!!! :D
I made this at Rethink Music Hacker’s weekend in Boston last weekend. This is an example using a 200ms sample of laughter.
A simple genetic algorithm learns the short-time fourier transform of a target static sound texture. The approximation gradually acquires information about the target sound via repeated semi-random modifications to the spectrogram. Phase and magnitude are learned separately. The learning process is sonified by inverting the estimated spectrogram at each iteration of the algorithm. The visualization is calculated by taking the inverse 2-dimensional fourier transform of the spectrogram at each iteration. I pass the 2D ifft only the real values of the spectrogram, resulting in symmetric images. This sonification and visualization allows for the gradual evolution of the sound from silence to target approximation can be seen and heard. The goal is not necessarily to accurately model the target sound, but rather to hear and see the learning process. Some target textures are easier to approximate than others but personally I find the ones that are difficult to approximate more interesting.
