Pictorize: Transforming image region luminosity to sound brightness for monitoring location

Abstract

A novel algorithm for transforming static and moving images into sound is presented. The algorithm turns image areas into a sound whose brightness depends on image area luminocity. Subjective evaluation investigated the extent to which listeners could identify if a moving target entered an image area based on the timbre of the sound produced from a sonified camera stream, while engaging with a parallel transcribing task. Results show that listeners identified target entries well above threshold, however, false detections increased for image regions of similar luminosity. Although providing visual feedback improved monitoring performance significantly, it increased dual-task cost when both displays were not accessible by peripheral vision.