We built a platform that converts 3D objects into a piece of music. We delivered a tangible music for users. Users can edit the model to change the music. It's a mutual channel between 3D world and music world.
I was the sound designer/engineer and front-end programmer in this team. Tasks including:
Auto-accompany system: Designed and programmed the "pattern" of 5 accompaniment instruments, so that user can modify the chord at run-time and generated the midi file.
Implement visual effects and UI using WebGL and CSS, included the piano keys side bar, instrument selection UI, particle effects, music player control panel.