There are three Listen Readers placed in a quiet
corner near the rear of the exhibition. Each one consists of a big,
fluffy chair and a small table that can be pulled up to the chair. Mounted
on each table is an illustrated children's book. As the visitor moves
their hands over these books wondrous sounds emerge, both relating to
the stories and providing a sonic atmosphere for the books.
What's it about? Each of the three stories is different and
I don't want to give away the plots here. Suffice to say that each soundscape offers the reader a deeper, immersive experience of each book.
What is the experiment? RED wanted to experiment by combining
reading with rich, interactive sound to see what new ways of understanding could be achieved.
Books often have rich, beautiful graphic illustrations; what if the audio illustrations could be just as deep and complex? This is a different experience for the reader than the commercial childrens' books that offer a simple button-based sound experience (where, for example, if you touch the picture of a duck, the duck quacks). In the case of
Listen Reader the visitor "conducts" the sounds of the book, creatively adjusting the mix of the soundscapes for each page. RED also
wanted to probe the future of printed books enhanced with digital sonic
effects. Here the print is static, but the sounds are dynamic. Lastly,
RED is very interested in exploring how to take existing written literature,
like the three books presented here, and digitally add to them, without
destroying the charm of the original.
How does it work? Listen Reader uses electric field sensors
located in the book's binding to sense the proximity of the visitor's
hand. As the visitor moves a hand over the book, the electric fields
bend. The sensors can pick up this bending and determine where the hand
is. This information is sent to a computer that uses it to decide what
sound and music samples to play, at what volumes and at what pitches.
The speakers are built directly into the chairs. Small tags (called
RFID tags) are embedded in each page of each book. Using these sensors
the computer can tell what page is opened in the book.
