APPENDIX

The following is a sample interview with one of the users who played the role of the student in the simulated classroom.  He was asked the following questions:

1. Did you use the interface to ask questions? If so, did you feel it 
is a "better" way than the old way?

Ans. I used the interface to ask question to the instructor.  It helped me 
ask questions anonymously at any point I did not follow the instructor,without 
interrupting the class.  
     I personally did not feel it a better way than the traditional way for the 
following reasons:
Firstly, the instructor tended typically to answer the  questions at the end of 
his/her lecture.  So I lost interest when the questions were finally answered at
the end of the class.  Secondly, there was no facility provided by the system
by which I could specify the slide each question referred to.  So, when the 
instructor finally started to answer the questions, I forgot the actual slides 
for the questions and a lot of time was wasted in finding the relevant slides and
have the questions finally answered.  Thirdly, the questions were not numbered
and there was no way to go to the previous question once the instructor moved to 
the next question.  So, similar questions either got answered twice or the 
instructor had to manually keep a record of which questions were actually asked.
I sent a question to the teacher but then I found that I understood it.  But 
there was no way I could remove the question.  I could not see the questions that others asked.  There was no way I could ask multiple questions at the same time.

Also, another problem that I faced was when I typed in the question, by default I pressed <return> from keyboard to have the same purpose served as pressing 
"send" button on the interface.  But it rather took me to the next line.  I think this is not a good design. 
  
2. Did you use the stoplight to signal your understanding of a 
particular slide? If so, did you feel this is of benefit to your overall 
understanding of the class? 

Ans. I did use the stoplight to signal my understanding.  In one case, when I 
voted for red ( i.e. asked the instructor to stop, I was lost), the instructor
started to speed up since no information was given on his console what different
colors for light stood for ( although it had resemblance with traffic light).  In
another case, I voted for green and the counter value for both yellow and green was
the same.  So, the instructor chose to go through the slide slowly and I got 
bored.  In another case I was not decided what to vote for, but the counter value
for green was maximum and the instructor just skimmed through the slide.

So, in some cases I felt that due to the above mentioned  interface design 
problem, still the traditional way serves better role to serve the purpose of
understanding in the class.     

    
3. Did you communicate your mood to the instructor? If so, did you feel
this helped in improving the class? 

Ans. I did not really use the moodmeter.  I thought I would rather interact with the teacher 
by sending questions or using stoplight.  This ensured a positive response from the instructor. Also, 
the moodmeter appeared to be redundant at least during the in-class phase.  It also gave the average 
reading for all the students.

Overall Evaluation and Recommendations
--------------------------------------
The design crtireria contained too much of technical detail.  Also there is the constraint on the 
system that it needs to use MS Windows 3.1 or above and need to use WINSOCK API.  In fact we had 
a hard time finding a PC with WINSOCK running.

Another problem that we faced was to know the IP address of the machine we were using.  This was found
confusing since the application showed a default IP address which we took to be the address needed for
connection.  But we had to find the hard way that it was the default address of the machine the development
team had been using.  So, if the system needs IP address specification, it should show the user a blank
screen initially.  Even better, the algorithm can be so designed as to take the machine address without the
user having to be worried.  There is no way for the user to know that (s)he has entered a wrong IP address.

When the user enters the IP address and presses the connect button, no feedback is given if connection
has been established.  Also, after the user is connected, (s)he should be continuously be shown a screen, 
may be at the bottom that the user is still connected.

The student can give anonymous feedback regarding overall mood.  But the moodmeter gives only the average
mood of the class.  This does not buy the students much so far the in-class phase is concerned.  The purpose
of the meter is also redundant and almost serves the same purpose as the stoploght particularly during the
in-class phase.  So,  the functions served by the meter should be made more versatile.  Moodmeter
rather than showing value instantly can have a log for every five minutes and for separate topics. 
There is no way to convey,  
 for example, five topics are interesting and five are boring with the present design. It is not clear if 
a single user can send multiple votes for the classmood.   If not, there should be a message when one
tries to send a second message.

For the instructor, (s)he will expect for the icon itself to show the mood rather than him/her to click on
it except in the case when (s)he wants to change its size. If the teacher clicks on the moodmeter and 
forgets to close it, the moodmeter should get updated as the students give further feedback.
Colors are not consistent between what appears on
teacher's console and the icon that appears clicking the moodmeter part of the console.  Whenever the
teacher needs to retrieve previous information, (s)he has to go the qq files.  This is true for quseions
and stoplight as well.  Going to qq file is not obvious from the system design at all.  So, some interface
must be provided that can tell the teacher where to go to retrieve previous information.  Even better will 
be to provide all the necessary information in some tables which the teacher can click and see rather than
him/her going to the qq file.  Also the name "qq" does not bring out at all the purpose it serves.
 
The stop light  lets the student convey their understanding on each slide and this  improves his 
comprehension of material on a particular slide and helps him understand major topics. But if some of 
the students may be undecided about  particular topic and may not vote for a slide while some students vote
for a particular color and the teacher takes that action. To avoid this,
algorithm should consider the history of the student participation.  So, if the student is undecided,
it will, by default give the last value the student voted for.   Also, the algorithm should take the 
number of students in class into consideration.  The red in stoplight can start with a counter value for
the number of students in class.  This will improve the handling the undecided part of the class.
Also, it is not clear from the interface design if the student is allowed to send multiple votes.
This gives a direct feedback to the instructor unlike the moodmeter.   But again for retrieving previous 
information, the instructor needs to go to the qq files.  The teacher gets both the number of votes and the
color of light.  This is redundant and confusing for the teacher.  So, it is better to give him/her
just the number of votes and let him take the action.  Also specify what the colors stand for in the 
teacher's console. 

The user uses the questioning icon to anonymously send questions to the user on each slide, without 
interrupting the class. It improves his comprehension of material on a particular slide and helps him
understand major topics.  But there are some limitations associated with the design of the interface.
Instead of using a single global counter and reinitializing it, the system should maintain log. So, when
one goes to the next slide and returns to the previous slide, the number
of questions now get changed from a finite number to zero.   So, to get any information
of the previous slides, one has to go to the qq files, which is inconvenient.  
 Also there is no provision
for saying if the user can send multiple questions at the same time. "Enter" pressing should serve the same purpose as clicking "send". In prei=sent design, it takes the user to the next line.  The questions are not
numbered.  So, the instructor has to manually remember how many questions were asked if he wants to know 
immediately the arrival of a new question.
The teacher cannot return to the previous question after he pops up the next question. 
So, keeping a log is a good idea which keeps a record of the questions and relate the questions by topic 
and slide.  There is no way for the students to see what questions other students have asked. So there is no
 way to avoid repitition of questions.  No facility
for removal of questions by the student is given.  So, the student should also be provided with the
record of all the questions which he can see and modify his own questions that he ashked. Also it will be
better if the student is given the option of setting the editor for typing questions.

There is no help provided, not enough feedback of the action taken by the user.  Also, no error message
was given that could let the user know that (s)he has taken a wrong step.

Provision for resizing the icons should be provided.

The student feedback is correlated for each slide, giving the questions were asked, the average mood of
that class and the counter value for the different lights.  These need to be retrieved from the qq files.
But this action is not explanatory from running the application.  So, the user should be given the feedback
that all the informations for each slide are being saved in qq files.  Also, for each file there is 
a separate file but results for all slides together could have been better.

Feedback is provided by the students in real time.  The teacher can dynamically act on the feedback from
the questiona nd stoplight.  But the same cannot be said about the moodmeter since it depends on the 
teacher when (s)he will click on the moodmeter menu.

The instructor can view the questions and opinions grouped by a sinle slide per file after the class is 
over.  So, this will enable the instructor to improve his/her lecture based on this feedback.

On the good side, however, the system interface served the aesthetic and minimalist design.  Although the
moodmeter interface could be improved, for all practical purposes the icons for the question and stop light
are self-explanatory. The prompts were understandable.
The dialogue labels were adequate.  In most case the command's purpose was obvious.
 

Critique on evaluation plan:

	The evaluation plan is quite exaustive and it covers all functional 
	aspects of the application.   


	The evaluation plan consists of three phases,
	1. Cognitive walkthrough questions
	2. Heuristic evaluation
	3. questionnaire 

	In the cognitive walkthrough, the breakup of goals does not appear
	adequate.  
	The tasks of giving feedback on a slide and giving measure
	of the classmood appear similar. This results in repeating similar
	questions. 
	For answering the questions on the cognitive walkthrough, one has
	to read the question again and again.  Instead the questionnaire 
	could have been prepared in an action-question sequence which would
	have made the task of answering a little easier.
		
	A lot of questions have been repeated in the heuristic evaluation.
	This part can be made more concise  by removing some of the repeated
	questions.  However, this part covers most aspects of the application
	.