Interactive Text Visualization

Keith Edwards

Project Goal:

Reading is not passive. Reading is highlighting, underlining, and annotating. It is showing relationships, taking notes in the margin, and elucidating the structure of a document. It's moving papers and lining up pages to compare just the right paragraphs with each other. It is jumping non-linearly between and within documents for comparison and search. Of course, some reading is passive - like perusing a novel on the beach. But the type we are interested in, known as active reading, constitutes a substantial portion of the time knowledge workers spend reading. Adler, et al. found that cross-referencing, in order to integrate information, alone constituted nearly 27% of reading activity for the wide variety of knowledge workers they studied. Reading to answer questions also constituted a significant fraction, at nearly 25%. And as seen in O'Hara's broad reading taxonomy, these types of reading activities are very active indeed, including non-linearity, note taking, comparison, and the like.

Given this high level of interactive engagement that reading frequently requires, O'Hara and Sellen found that computers - at least in 1997 - were not up to the task. They simply could not provide the same level of flexibility and breadth of interaction affordances as paper. But paper is not a panacea - even for the things at which is usually bests computers. We propose instead that paper is rather inflexible; it assumes a particular way of presenting a document and while it allows one to embellish upon it, it provides little room to alter or restructure this presentation.

So, rather than building computing systems that attempt to mimic the affordances of paper (as is the approach with current Tablet PCs), we aim to take computer-mediated reading in a very different direction, to address the fundamental interaction challenges of active reading and document usage, leveraging new interaction and display technologies. As we describe below, we seek to build a reading experience on a more fluid metaphor, where the structure of a text is akin to a liquid, naturally affording reorganization, comparison, and more expressive forms of annotation. To enable efficient manipulation of so flexible a representation, we require far more expressive input methods than traditional mice and keyboards. For this, we plan to use a modern multi-touch platform, which enables high dimensional input by allowing the use of complex hand gestures and many-finger input.

The high-level golas of our work are to develop new interface metaphors for interacting with documents and text on multi-touch systems, create and evaluate an interaction vocabulary of techniques to support browsing, annotation, comparison, hyperlinking, organization, and other aspects of active reading, and provide a compelling and novel application of multi-touch technology that can support the needs of knowledge workers in advanced office environments.