MILA - Musical Instrument Learning Apps

A collection of web-based apps for musical instrument practice.

online demo

video (YouTube) | video (download)

DOI to TVCG | arXiv

citation:

@article{heyen2026make,
  title    = {Make the Unhearable Visible: Exploring Visualization for Musical Instrument Practice}, 
  author   = {Heyen, Frank and Gleicher, Michael and Sedlmair, Michael},
  journal  = {IEEE Transactions on Visualization and Computer Graphics (TVCG)}, 
  year     = {2026},
  pages    = {1-17},
  doi      = {10.1109/TVCG.2026.3658216}
  keywords = {Music;Instruments;Data visualization;Visualization;Education;Prototypes;Aerospace electronics;Timing;Musical instruments;Games;Temporal data;application-motivated visualization;education;personal visualization;music;instrument practice},
}

What You Need to Use the Demo

Most apps require a MIDI instrument, though a few work with audio or allow you to use a PC keyboard or touch screen.

While several apps should work with any (MIDI) instrument, we only tested them with keyboard, e-drum, and a MIDI guitar pickup.

Setup

This is only necessary for local testing or development. If you just want to use the apps, go here.

• Install NodeJS and npm
• Clone/download this repository
• cd mila
• npm i
• npm start
• Open the URL shown in the console

Used Libraries and Packages

• Svelte components
• Observable Plot visualization
• D3 scales, data processing
• tonal music theory such as scales, chords, progressions
• musicvis-lib more technical musical functions
• pitchy pitch detection from audio
• fflate compression of share links
• webmidi Web MIDI API wrapper
• fast-kde density estimation
• a-frame for the 3D fretboard design
• @fontsource noto-music, noto-sans, noto-sans-symbols, noto-sans-symbols-2 for cross-plattform note symbols and icons
• https://github.com/danigb/soundfont-player and midi-js-soundfonts for playback