Hip Cat
Armadillo Ray
Frank was a Monster who
       Wanted to Dance

There are three Listen Readers placed in a quiet corner near the rear of the exhibition. Each one consists of a big, fluffy chair and a small table that can be pulled up to the chair. Mounted on each table is an illustrated children's book. As the visitor moves their hands over these books wondrous sounds emerge, both relating to the stories and providing a sonic atmosphere for the books.

What's it about? Each of the three stories is different and I don't want to give away the plots here. Suffice to say that each soundscape offers the reader a deeper, immersive experience of each book.

 

What is the experiment? RED wanted to experiment by combining reading with rich, interactive sound to see what new ways of understanding could be achieved. Books often have rich, beautiful graphic illustrations; what if the audio illustrations could be just as deep and complex? This is a different experience for the reader than the commercial childrens' books that offer a simple button-based sound experience (where, for example, if you touch the picture of a duck, the duck quacks). In the case of Listen Reader the visitor "conducts" the sounds of the book, creatively adjusting the mix of the soundscapes for each page. RED also wanted to probe the future of printed books enhanced with digital sonic effects. Here the print is static, but the sounds are dynamic. Lastly, RED is very interested in exploring how to take existing written literature, like the three books presented here, and digitally add to them, without destroying the charm of the original.

How does it work? Listen Reader uses electric field sensors located in the book's binding to sense the proximity of the visitor's hand. As the visitor moves a hand over the book, the electric fields bend. The sensors can pick up this bending and determine where the hand is. This information is sent to a computer that uses it to decide what sound and music samples to play, at what volumes and at what pitches. The speakers are built directly into the chairs. Small tags (called RFID tags) are embedded in each page of each book. Using these sensors the computer can tell what page is opened in the book.