Mmmmm : A Multi-modal Mobile Music Mixer

This paper presents Mmmmm; a Multimodal Mobile MusicMixer that provides DJs a new interface for mixing musicon the Nokia N900 phones. Mmmmm presents a novel wayfor DJ to become more interactive with their audience andvise versa. The software developed for the N900 mobilephone utilizes the phones built-in accelerometer sensor andBluetooth audio streaming capabilities to mix and apply effects to music using hand gestures and have the mixed audiostream to Bluetooth speakers, which allows the DJ to moveabout the environment and get familiarized with their audience, turning the experience of DJing into an interactiveand audience engaging process.Mmmmm is designed so that the DJ can utilize handgestures and haptic feedback to help them perform the various tasks involved in DJing (mixing, applying effects, andetc). This allows the DJ to focus on the crowd, thus providing the DJ a better intuition of what kind of music ormusical mixing style the audience is more likely to enjoyand engage with. Additionally, Mmmmm has an "Ambient Tempo Detection mode in which the phones camera isutilized to detect the amount of movement in the environment and suggest to the DJ the tempo of music that shouldbe played. This mode utilizes frame differencing and pixelchange overtime to get a sense of how fast the environmentis changing, loosely correlating to how fast the audience isdancing or the lights are flashing in the scene. By determining the ambient tempo of the environment the DJ canget a better sense for the type of music that would fit bestfor their venue.Mmmmm helps novice DJs achieve a better music repertoire by allowing them to interact with their audience andreceive direct feedback on their performance. The DJ canchoose to utilize these modes of interaction and performance or utilize traditional DJ controls using MmmmmsN900 touch screen based graphics user interface.