Constructed by Dongmei Wang.
|
Winter Quarter 1996 |
DreamPad Project Team: |
The current system is comprised of two parts: the standard classroom, and the classroom augmented with the basic classroom 2000 technology. The specific characteristics of the standard classroom that are related to the problem are:
The specific portions of the classroom 2000 technology related to our problem are:
This project lends itself to three principle criteria for evaluation:
Dialog initiative deals with the ability to free the user from artificial constraints imposed by the system. The current system for questioning and feedback is manual, therefore we wish to look at the ability to free the user from peer pressure and fear of instructor disagreement, the ability to provide immediate critique of instructional materials, the ability to provide real-time feedback on class climate and generally free the user from the old "raise-hand = get answer" paradigm.
Multi-Threading deals with the ability to support more than one task at a time. The current system, since it is manual, only supports multi-threading in the sense that the instructor may be answering a question while writing on a slide. The question however had to be acknowledged, asked, and answered which broke the "flow" of the class interaction. We wish to look at how many times instructors have said to hold a question for a moment and ask it later, only to have the student forget what context it was that the original question was asked. We want to look at the ability of the current system to "que" up questions so the students can ask immediately, but the instructor can recall them later for answering at a logical place in the class. We also want to evaluate the ability to provide an individual instructor feedback on his current slide, and on the class climate as a whole in an unobtrusive fashion.
Observability deals with the ability of the users to evaluate the state of the system from its representation. Again, since the existing system is a manual one, we wish to determine how instructors currently decide which slides they use in a particular class are good ones, and which are poor and need to be redone. Also we want to determine how instructors decide how a class is proceeding, i.e. are the students following the discussion or are they lost? We also wish to determine how instructors know that students have questions, and how they store questions for later answering: i.e. do they stop the class and address each question, or do they "store" them somehow and answer them later? We also want to evaluate how instructors determine the overall "mood" of a class, i.e. are they bored or excited?
Due to time constraints, the initial evaluation of the current system will be conducted through interviews of students, selected faculty and graduate teaching assistants. A copy of the basic interview questions for students are included at appendix A and for instructors at appendix B.
We will also perform observations of classes in progress using both traditional and the CLASSPAD application. These observations will key on the following areas:
We will administer the student portion of the survey to 5 graduate and 10 undergraduate students at GA Tech from multiple disciplines. The instructor portion will be administered to 6 professors from different schools/colleges at GA Tech. The results will be initially qualitatively evaluated.
Prior to beginning phase 2, we will increase our sample size and prepare quantitative analysis to more accurately analyze the data.1. Dialog Initiative:
2. Multi-Threading:
3. Observability:
Our current data is incomplete. We have not yet conducted the required number of interviews to get accurate data, however some initial anaysis based upon completed discussions follows:
Student interviews have been conducted with 2 high school students, 2 undergraduate students and 1 graduate student. The data suggests that the importance of anonymity in questioning decreases with grade, with the high school freshman feeling it was most important and the graduate student feeling it was of minimal use. This probably follows from our intuitive understanding of social development and the expectations of the environment. As one might expect, graduate students and the classes they take are more questioning and open-ended where discourse and the challenging of instructor ideas is encouraged. On the other hand, grades K-12 are more cognizant of peer-pressure and the existing social context and much more emphatic about the benefits of anonymous questioning.
As one might also expect, instructor feedback has a similar variance, with the high school students never giving feedback, to the graduate students giving more frequent feedback. None of the respondents give "mood" type feedback to instructor (state of boredom or excitement) during a class. All have admitted to feelings of boredom at one time or another. The primary method of feedback was the end-of-course critque, which all the upper-level students felt was useful in improving future iterations of the course, but was of limited value in helping them while in the course itself.
Only 1 university instructor interview has been completed. He felt that students, especially undergraduates gave little feedback during the course. He did not feel that questions ever disrupted the flow of class, therefore queing questions in some manner did not seem important. His preference was to answer related questions while the related slide was visible. The assessment of which slides were good was largely qualitative and was based upon how many questions the student had. He felt there was a correlation that many questions on a particular slide was indicative of a complex concept and therfore might signal the need to redo the slide in question. He felt that receiving feedback on each slide would be valuable. Anonimity from his perspective was unimportant, although he believed that it might be more important to certain student personality-types.
Determining the mood of the class was also largely a qualitative "skill." By observing student behavior, he attempted to guage the level of boredom/interest and adjust teaching styles accordingly. The instructor felt that receiving feedback on class mood and the general level of understanding would be beneficial. Again anonymity to the instructor was unimportant.
Based upon some indications that the need for anonymity is greater in K-12 and decreases into college and university studies, we will attempt to add at least some additional high-school students and instructors to our interview base before completing phase 2 of the project.
Our observations so far are limited to one class period in a senior-level undergraduate course in the College of Computing and 2 class periods in CS6751 where the CLASSPAD application was used. As might be expected, the student questioning process followed its traditional form. The questioning process did not seem to differ between the CLASSPAD classroom and the more traditional classroom. Overt instructor feedback was not observed during either class, although the instructor in CS6751 did make self-initiated efforts to more formally determine the class mood.
Based upon our limited data set, it is difficult to form valid recommendations, however the following are very preliminary recommendations that will be modified as more reliable data is received: