Pitch-Perfect -- Personalized Sonification for Accessible Data Visualization

Advisor -- Prof. Yea-Seul Kim (Visualization Lab, Computer Sciences, UW-Madison)

Research Question:

Audio charts make data accessible for the visually impaired populations. However, it can mislead listeners if the mapping function between data and audio display is not aligned with one’s internal mapping. Therefore, we studied the individual differences of the internal mappings among users.

Method:

  1. In the user study, participants were first presented with two audio clips representing the minimum and maximum values. Then, they listened to three audio clips and were instructed to estimate the encoded data value.
  2. Applying a power law function to each participant’s data to investigate individual differences in pitch-to-data mapping
  3. Contructed the personalized models with varied numbers of data points and normalized/non-normalized data to investigate the properties of the models

Main Findings:

  1. We confirmed the uniqueness of each person’s internal mapping, implying that participants assigned divergent numbers to a given pitch.

  2. Models fitted with normalized data had fewer errors. However, data standardization is not necessary when minimum and maximum mapping are assigned. The result suggests that it is hard for participants to learn the mapping used in a system.