next up previous
Next: Developer insights Up: EXPERIENCE AND INITIAL EVALUATION Previous: Qualitative evaluation

Evaluation of AI and FCE classes

The prototype used in the HCI class was relatively stable before that course began. This was not the case for the other two courses (AI and FCE). Consequently, both the AI and FCE prototypes changed quite a bit during the quarter and we were unable to collect much quantitative evaluation information. In the AI class, the only capture that occurred during the live recording phase was the video and audio recording. The manual post-production activity of augmenting the HTML notes with audio and video links was so time consuming that after the fourth lecture we were no longer able to devote the resources to continue the service. We see now the advantage of having tools that use natural actions of the teacher to automate the audio and video augmentation of Web pages, as described in [2, 9]. Such actions could include long pauses in their speech or mouse movements to draw attention to some area on the page.

In the FCE seminar, we both videotaped and used the MessagePad outline-annotator. We provided a template file to prepare the outline of the class discussion; this was a useful service for the presenters. The student note-takers found it easy to understand how the outliner application worked, but did not find it all that useful to attach notes to the outline. The main problem was that discussion in the class did not follow the outline very closely. When the note-taker wanted to jot down a thought, it was hard to determine which entry in the outline to choose for annotation. Sometimes, the choice of entry was entirely arbitrary, and the resulting enhanced Web page looked somewhat confusing. As the quarter progressed, we altered the application by removing the outline and allowing the user to bring up blank, time-stamped note pages for writing whatever came to mind. This simpler interface, similar in spirit to Stifelman's audio notebook [14] was much less confusing to the user, but was only used in the class a few times.


next up previous
Next: Developer insights Up: EXPERIENCE AND INITIAL EVALUATION Previous: Qualitative evaluation
Future Computing Environments
College of Computing at Georgia Tech University