The Design Discussion Area:
A Collaborative Learning Tool in Support of Learning from Problem-Solving and Design Activities

Janet L. Kolodner and Kristine Nagel

Georgia Institute of Technology, College of Computing

Abstract: The Design Discussion Area (DDA) supports student reports to each other about their design efforts as they are learning science from design activities. Its scaffolding was designed to support both the writing of well-formed stories about their design efforts and discussion about each design effort. At their best, these discussions are a source of ideas for other students; a venue for talking science, using evidence in support of claims, getting advice, and providing explanations that others can understand; and a vehicle for some of the reflection necessary to turn oneās experiences into well-formed and well-indexed cases in oneās memory. DDA was trialed with 200 students; it was easy for students to use but wasnāt as straightforward to integrate into classroom activities as we had supposed and didnāt provide as useful scaffolding as we had imagined.
Keywords: learning through design, case-based reasoning, inquiry

1.0 Introduction

Learning well from complex problem-solving, design, or project activity almost necessarily requires collaborative (Kafai and Ching, 1998; Jackson, et. al., 1996). When problems students are addressing are of serious complexity, they need each other. Sharing perspectives, experiences, insights, and understandings helps learners to come up with ideas, to debug their ideas, and to notice the complexities of concepts and skills. From the other side, evidence shows that if collaborative learning is done well, learners can solve much more complex problems and come to far more sophisticated understandings than they could do on their own (Barrows, 1985).

But it is a long way from what research suggests to figuring out the practices that will promote success in the average classroom. For each of the successful learning-from-problem-solving or learning-from-design, or project-based learning classroom, there are others that are a disaster. What does it take to orchestrate and facilitate collaboration to insure that students have success in their project endeavors and gain a sophisticated understanding of the targeted concepts and skills? And how can the computer help?

We address these issues in the context of Learning by Design, an approach to learning from hands-on design activities (Kolodner, 1997; Kolodner, et. al., 1998). Learning by Design builds on a variety of cognitive theories addressing learning (e.g., case-based reasoning, constructivism) and classroom practices supporting collaborative learning and learning from hands-on activities (e.g., PBL, communities of learners). In Learning by Design (LBD), students learn science (grades 6 to 8) by addressing design challenges that require application of science for their best solution. The need to address the design challenge motivates students to ask questions, provides a purpose for learning science, and gives students a vehicle for applying what they are learning. As students iteratively plan, construct, test, and refine solutions to design challenges, their understanding of science concepts and skills becomes more and more sophisticated. Students learning about motion and forces, for example, design and build miniature vehicles and their propulsion systems, achieving the challenge of getting over a hill and as far beyond as possible. Students learning about erosion model ways of controlling the erosion on a barrier island in Georgia.

Collaborative learning is a central value in LBD. Students collaborate in small groups to achieve design challenges. They also collaborate with each other in whole-class discussions, sharing their ideas and what theyāve found out so that others can provide them with advice and suggestions and so that they can benefit from each othersā ideas. The LBD approach encourages students to build on the work other students have done, so long as they give credit and donāt just copy, but rather integrate and adapt the ideas of others with their own. The most important vehicle for making such collaboration happen in an LBD classroom is the "gallery walk" (Kolodner et al., 1998). Groups present their in-progress design solutions to the rest of the class for suggestions, comments, and advice.

The Design Discussion Area (DDA) was designed to support and enhance gallery walks, allowing students to report to and get feedback and ideas from groups outside of their own class. It supports both the small-group collaboration needed to describe oneās project in a way that others can understand well and larger-scale between-group and between-class discussion about design ideas. Its scaffolding structures these activities, provides hints, and provides examples. Both students and teachers have been excited about and engaged in using the tool. But it had to be integrated into classroom activities somewhat differently than we had planned, and once integrated, we were surprised by some of the ways it was and wasnāt used.

2.0 Our approach to promoting and supporting collaborative learning

Our approach to promoting and supporting collaboration and collaborative learning is based on a variety of approaches. Problem-Based Learningās approach (Barrows, 1985) is seen in the orchestration of activities, the roles the teacher takes on, the interactions between individual and group activities, and the centrality of reflection. Students work together in small groups to address challenges: designing and running experiments; planning, constructing, trying out, and analyzing solutions; and preparing to present to the class. Whole-class time spent on generating questions for inquiry, sharing insights, reflecting on whatās been learned, and presenting ideas and experiences to each other. Homework provides time for individual students to prepare themselves for small-group activities or reflect on those small-group activities, make sense of them, and articulate what theyāve learned.

Case-based reasoningās (Kolodner, 1993) suggestions can be seen in the orchestration of the classroom and in the content of student reports and class discussions. The challenge of constructing, testing, and making physical devices work provides students with experiences to anchor their discussions. They discuss science in the context of the needs that come up as they are trying to achieve a real challenge. The challenge pushes them to talk about what they need to learn, what they are confused about, and how they made things work. Turning their experiences into cases requires making connections between goals, implementations, and results, and the need to report to the class requires them to reflect on their experiences in such a way that they can draw those connections and report them to the others. Telling their story to the class helps them make sense of their experience. Hearing the stories of others allows them to vicariously experience what others have done, giving them additional experience with the results of applying the skills and concepts they are learning.

The design studio approach complements PBL and CBR and can be seen in the setup of the classroom. In addition to formal pin-ups and gallery walks, design ideas and plans of students are displayed around the classroom, and students have free access to each other.

In our 1996-97 experiment with paper-and-pencil and computer-based tools (Puntambekar et. al., 1997), we found that different kinds of discussions were natural within and between groups, and that both were needed for deep learning. Within groups, students tend to focus on how things work and how to make them work better, discussing the structure and behavior of things they are building but not always focusing on the whyās. They use science within groups but donāt tend to talk science. On the other hand, discussion across groups requires making oneās goals, analyses, and justifications clear. Here is where more science talk resides, in the explanations of function and mechanisms, and use of evidence to justify. Each type of talk, we discovered, needed different kinds of scaffolding. We said then that we would move forward by providing better between-group facilitation to support sharing, critique, and analysis.

3.0 Our tool: The Design Discussion Area

The "gallery walk" implements the suggestions made by all three approaches and by our previous experiment, and it is this ritual that we support in the Design Discussion Area (DDA). During a gallery walk, each student group shows off the device they are building, explaining their design decisions, showing the class how their device works, reporting on the data theyāve collected and their analysis of it, and talking about their proposed next steps. Other students ask questions about things they donāt understand or about why different choices were made, point out problems, and make suggestions about how to proceed. Students are motivated to play both roles. Showing off what theyāve done puts them into the spotlight and gives them an opportunity to get help from the rest of the class. At its best, the gallery walk is not only a source of ideas but also provides a venue for talking science, using evidence in support of claims, and providing explanations that others can understand.

The gallery walk has two collaborative parts to it. First, each small group needs to decide what to tell the rest of the class. Second, the presentations each group makes become anchors for large-group discussion. DDA supports both kinds of collaboration (Figure 1): (a) Collaboration within groups focuses on describing design ideas and results so that others can understand them. (b) Collaboration across groups is anchored in the design ideas and results of each small collaborative group.

Figure 1 shows the Design Discussion Area during discussion.

3.1 Helping small groups report on their designs

Student teams use this part of the tool to report on their construction-and-test experiences. The system prompts them for important concepts they should include in their report ÷ what they were trying to accomplish, how they decided to do it and why, what they were expecting to happen, how it actually worked, and why it worked that way. The structure within which they do their reporting implicitly organizes the students' articulation of their experiences, while the hints accompanying each prompt explicitly guide the students in what information to include. The structure and hints are adaptable for different challenges.

Figure 2 DDA prompts for writing up oneās design experiences ideas

3.2 Supporting discussion across groups and classes

After a team reports its design results, their report is available to other teams by clicking on its hyperlinked title. Looking at another team's idea will open two side-by-side windows: the presentation on the left and a comment window on the right. This anchored collaboration (Guzdial et al., 1997) ties each student presentation to its own threaded discussion space. Other students may add a new comment or question for the team (a new thread) or insert a comment into an existing discussion. Scaffolding is quite minimal, so as not to get in the way, but serves two essential purposes: (1) It differentiates between continuing an old discussion and beginning a new one. (2) Suggestions are made to students about the kinds of comments they might want to make ÷ "praises," "wonders," and "suggests" for new threads; "replies," "wonders," and "suggests" for continuing threads. Students can also add their own new types.

Figure 3 ÷ Prompts for participating in discussiondiscussion

3.3 Scaffolding in the Design Discussion Area

In both modes, scaffolding is in three forms: structuring, hints, and examples. Structuring implicitly helps students organize their thoughts. Hints provide explicit pointers on what to write. Examples serve as exemplars of what is expected; students use examples both to figure out what to write and to evaluate whether what theyāve written is good enough. The user interface integrates the three forms of scaffolding. Hints are to the right of each area where the students enter text, and a completed example is available by clicking the "example" icon at the top of the screen (Figure 2). In addition, we provide a guided tour with screen shots.

4.0 The classroom experiences

The Design Discussion Area was used during the 1998-99 academic year in two middle schools located in different suburban school districts of Atlanta, Georgia. The physical science teacher was in his second year with the LBD project and had five classes with 102 students. The earth science teacher was in her fourth year with the project and had four classes with 105 students.

Each classroom had an intranet of computers with one computer per team. Both teachers had used Web-SMILE (Guzdial et al., 1997), the predecessor to DDA, with prior versions of the same curriculum unit. Each class was well versed at gallery walks before beginning use of the tool.

The earth science students used DDA as one of many resources during their three weeks in the "Plate Tectonic Jump" problem. Each team chose one of seven specified latitude-longitude pairs and researched plate movement in that area in order to predict five hundred million years into the future. The teacher encouraged collaboration across her many classes, to insure that each group had at least one other group to talk to about their research and predictions. As well, the seven locations were not entirely disjoint from each other, so students' discussions with those working on different locations could also help students with the one they selected. Although the teacher directed the groups to share ideas and research notes, she left it up to the team members to determine exactly what to present to each other and how much to interact in the discussion area. The computers were set on each team's table every day and were always physically in the midst of each group. The teacher took the server home each night to read what students had entered during the day and to contribute her own comments.

In the physical science class, DDA was used over a two-month span to support our Vehicles in Motion unit. Students built, experimented with, and optimized a variety of propulsion systems during this time, and DDA was used during the development of each. Each team designed and built a balloon-powered vehicle, a Rubberband or Falling Weight-powered vehicle, and a final vehicle powered by a combination of propulsion systems. The teacher created four display spaces within DDA ÷ one for each of the types of vehicles ÷ and students entered descriptions of each of the propulsion systems they designed. Computers were set on countertops along the classroom walls; students used them in "kiosk" fashion, standing at the computer and going to it as needed or directed. The classroom had a certain quality of the design studio to it with walls covered with pin-ups and shelves of models in various states.

Students stayed within the same group through the Vehicles unit. However, DDA was used quite differently throughout the unit. Early on, when students were working on their balloon-powered vehicles, the teacher had students use DDA to report on experiments they had done testing the effects of different lengths and diameters of straws and different numbers of balloons on the behavior of their vehicles. They first reported to the class in gallery walks and then recorded their results in DDA ÷ to be used as a repository. Students continued by designing the best balloon-powered vehicle they could taking into account the full set of reported experimental results. For the Rubberband and Falling Weight cars, where students were doing more informal exploration of each carās propulsion characteristics, he had students write in DDA in preparation for their gallery walk. For the final challenge, he had students write in DDA as they were planning how they would merge the three propulsion systems into a hybrid that could go over a hill and beyond.

5.0 Analysis of classroom experiences

We were interested in finding out several things:

Feasibility: Are students able to do the kinds of writing and have the kinds of interaction we expected from them? Are teachers able to facilitate its use well?

Patterns of Use, Integration into Classroom Activities, and Usefulness: Are they using it in ways we had predicted? If not, how are they using it, and why? What purposes is it serving well? What purposes should it serve better? How natural does its use seem, and can we make it more natural?

Effects: In what ways do students seem to be benefiting from its use -- in terms of content learning and/or in terms of communication and collaboration skills?

To find these things out, we watched classroom use of the tool; interviewed teachers with regard to use, expectations, and observations; analyzed student patterns of use of the tool (using log files); and coded studentsā entries for appropriate use of science concepts, ability to communicate, how systematic their presentations were, and how well they were doing at justifying their claims and predictions.

5.1 Feasibility

Both classes began use of the tool with a high level of scaffolding by the teacher aimed at helping them learn to navigate the tool and to develop an understanding of what to include in their presentations. The teachers purposely provided less and less help over the course of DDAās use. Students made some use of the on-line tutorial with images of the various displays they would encounter. They had no problem with the web browser interface but some took a few sessions to become clear on the multiple windows and how they were related.

Students in both sets of classes were able to collaborate to write their reports, though each did it differently. In the physical science classes, which were far more structured, students wrote their reports based on a variety of resources: written homework of individually conceived ideas, group whiteboards written on butcher paper, and pin-ups of prior vehicles on the classroom walls. While one of member of a group always acted as recorder (typist), he or she was constantly consulting and negotiating with the others about what to write and how to write it. This collaboration was not afforded by the physical location of the computers. Though the computers were difficult to gather around, students were engaged enough to do it anyway. In the earth science classes, on the other hand, where such collaboration around the computer would have been easy, students tended to discuss in groups first, then ask somebody to write up their discussion. We can only guess that these differences are due to the teachersā instructions and styles. Both seemed to work fine.

5.2 Patterns of Use, Integration into Classroom Activities, and Usefulness

When we originally designed DDA, we intended that students would write up their design experiences using the tool for students in other classes to see. This way, we thought, we could increase studentsā access to the ideas of others. We had not foreseen, however, how this writing would interact with the in-class gallery walks. When teachers had students write in DDA after discussions in class; students were bored÷ they had just discussed their designs and were ready for the next iteration. On the other hand, their presentations were quite clear, and students in other classes read them, asked questions, and made comments. When they wrote before their in-class gallery walk; their on-line presentations were not as clear and therefore not as useful to others; students in other classes therefore werenāt very inclined to make suggestions.

How DDA was integrated into the classroom activities had a large effect on the number of comments made and the quality of those comments. In the physical science classes, students used the computers on particular days that the teacher chose. In the earth science classes, the computer was always available, and there was little interaction with planned gallery walks; DDA was their way of doing gallery walks. In the earth science classes, there were many more comments made per presentation (1.6 in phys. sci. vs. 2.7 in earth sci.) and the majority of the comments made in the earth science classes were on-task (76%), while only a minority (31%) were in the physical science classes. It is, of course, hard to say whether this difference was because DDA took the place of gallery walks in the earth science classes or because students were more restricted in their use of DDA in the physical science classes.

We also hadnāt foreseen that teachers would use DDA for purposes other than those we had designed it for. In earth science, DDA was used close enough to the way it was intended to serve its role well. In physical science, however, the teacher used DDA for three different purposes, only one of which we had anticipated, each requiring slightly different kinds of reporting. Students in physical science used DDA to report to others about the results of experiments they had run and their design plans in addition to using it to report on the results of their design efforts. The system was designed only to scaffold reports of design efforts, and there seemed to be some confusion among the students about how to tell their stories when their needs didnāt match the systemās scaffolding. The structure and hints we had provided were not always appropriate and didnāt always emphasize what was most important to report.

Both teachers were interested in DDA as a data repository; students in all the classes saw DDA as a natural repository as well, and they at least tried to use it that way. Earth science students report it useful as a repository, and weāve already seen the numbers that show that they used it for communication as well. Physical science students were not as happy with it as a repository, because often the reports they found in there were not well enough written to be truly useful. On the other hand, they tried. When we look at numbers of accesses during design and planning, we see that the number is quite high. While working on balloon cars, the 100 students (approx. 30 groups) accessed balloon-car reports over 1000 times. And when we look at access to the full range of reports on the days that students were designing their hybrid propulsion systems, we see 17 lookups of balloon-car reports, 96 of rubberband and falling-weight reports, and 295 accesses to their own and othersā hybrid designs.

5.3 Effects

Are students benefiting from DDA? DDA has effects to the extent that it helps students collaborate on line, that it promotes better in-class discussion, that it promotes sharing of ideas across classes, or that it enhances studentsā abilities to communicate scientifically with each other. Students in both sets of classes, when they saw benefit in communicating across classes, used DDA for such communication. Did that help them perform better or learn better? We donāt know. The teachers tell us that indeed, in-class discussions improved in quality and in science content over time, but we have no way of knowing what of that is due to DDA. We saw in both contexts that students did share ideas across classes. Some of that was due to reading about the ideas of others using DDA; other of that sharing was due to lunchtime discussions with students in other classes. If DDA promoted studentsā engagement in those discussions, we could say that it effected their sharing, but we donāt know for sure. Enhanced ability to communicate with each other we could measure using student entries into DDA, if it had been used homogeneously over time. Unfortunately, its use doesnāt let us make any conclusions about enhanced communication or scientific capabilities.

What about smaller effects? We saw some that confirm our predictions about student learning and development. We saw more use of science vocabulary the second time using the tool (for the rubberband and falling-weight cars) than we did the first time (for balloon cars). This is expected, as vocabulary should be cumulative over time. We did not see as much science vocabulary used the third time through ÷ for the final hybrid design. This, we guess, is because students were only at a planning stage at the time of writing. They had not yet tried their ideas out and gotten feedback from the physical world. We would expect far more science discussion after they had tried out their ideas and were in the process of revising them.

We analyzed discussions for communication skills, being systematic about investigations, and justifying explanations and conclusions. Student ability to communicate measured by looking at how clearly they stated their hypothesis or design solution remained relatively constant across their reports, indicating that as the teacherās scaffolding was lessened, students were still able to perform. Their ability to be systematic about investigations measured by looking at how students attempted to make repeated trials as fair as possible and what alternatives were compared in each iteration was highest for the balloon car, where they spent significant time planning investigations, collected significant data, and spent significant time presenting their results to the class and getting feedback before writing. Their ability to justify measured by looking at the type of evidence cited: experimental data, their own or others' experiences, scientific concepts, engineering and feasibility explanations also was highest for the balloon cars, but did not go down as significantly as it did for systematic investigation. This might suggest that scaffolding for these skills was removed too soon. On the other hand, their later activities were quite a bit more complicated than the balloon car. Even with much scaffolding removed, they were still able to perform, suggesting that our system of scaffolding (shared by teacher, DDA, and classroom rituals) is on the right track.

6.0 Discussion

Our design builds on what we and others have learned in past investigations, and a large measure of its success is due to taking into account the lessons of those projects, both those that report on creating a collaborative atmosphere (e.g., communities of learners, constructionism) and those that report on design of collaboration software and its integration into the classroom: CSILE (Scardamalia, et. al., 1994), CaMILE, McBAGEL, BGuILE (Tabak & Reiser, 1997) WebSMILE, Belevedere (Suthers, et. al., 1997), KIE (Linn, 1995).

On the other hand, everything was still not completely smooth. We've learned several additional lessons. (1) Students like publishing their work for others to see and like looking at the publications of others, but they donāt seem to like using it for discussion when they can do that face to face. Theyāre willing to read and comment on what others have proposed, but only if they think it will have a benefit to them and only if whatās written is well-enough described to be worth commenting on. (2) Integration of collaboration tools of this kind into the classroom requires quite a bit of planning to make sure that time is provided both for reading and commenting and to make sure that use of the tool and classroom discussion complement each other rather than being repetitive. Sometimes this is easy for teachers to manage; other times it's complex enough that we need to give teaches detailed guidelines about orchestration.

7.0 Next steps

In response to the broad uses the teachers have made of the DDA, we are making two kinds of changes. First, we are making some changes in DDA itself. Weāre customizing the scaffolding for three different kinds of uses of DDA: reporting experimental results, reporting about plans, and reporting on an iteration of constructing and testing. To encourage more scientific talk, a domain specific concept list will accompany each discussion area. We also need to help students assess their writing to make sure it has the content in it that they intended to write. We will make it simple to include photos, sketches, and videos of their artifacts. And we are working on an indexing scheme that will make it easier for students to get to the reports they are looking for. Second, weāre making suggestions about how to integrate DDA better into classroom activities. While writing before presentation helps students with their presentations, it doesnāt promote very good sharing of ideas. We will encourage separating out the writing to prepare for in-class presentations from the writing to share with students in other classes, asking teachers to have students revise their reports after their in-class discussions. We will have to find a way to make that work that capitalizes on studentsā want to use DDA as a repository and communication device and making sure they get some value added when they do their revisions. Perhaps expanding communication to outside of their school will provide that extra value. Perhaps the opportunity to go back and see how far theyāve come will be engaging as well.

Acknowledgments

This research has been supported in part by the National Science Foundation (ESI-9553583), the McDonnell Foundation, the BellSouth Foundation, and the EduTech Institute (with funding from the Woodruff Foundation). The views expressed are those of the authors.

References

Barrows, H.S. (1985). How to design a problem-based curriculum for the preclinical years. NY: Springer.

Guzdial, M., Hmelo, C., Hubscher,R., Nagel, K., Newstetter, W., Puntambekar, S., Shabo, A., Turns, J., Kolodner, J. L. (1997). Integrating and Guiding Collaboration: Lessons Learned In Computer-Supported Collaboration Learning Research at Georgia Tech. In R.Hall, N.Miyake, & N. Enyedy (Eds.), Proceedings of Computer-Supported Collaborative Learning '97 (pp. 91-100). Toronto, Ontario, Canada.

Jackson, Shari, L., Stratford, S. Krajcik, J., and Soloway, E. (1996). A Learner-Centered Tool for Students Building Models. Communications of the ACM, Vol. 39, No. 4, pp. 48-49.

Kafai, Yasmin B., Ching, Cynthia C., (1998). Talking Science Through Design: Children's Science Discourse Within Software Design Activities. In A. Bruckman, M. Guzdial, J. Kolodner, & A. Ram (eds.), Proceedings of International Conference of the Learning Sciences 1998 (pp. 160-166). Atlanta, Georgia.

Kolodner, J. (1993). Case Based Reasoning. San Mateo, CA: Morgan Kaufmann Publishers.

Kolodner, J. L., (1997). Educational Implications of Analogy: A View from Case-Based Reasoning. American Psychologistt, Vol. 52, No. 1, pp. 57-66.

Kolodner, J. L., Crismond, D., Gray, J., Holbrook, J., Puntambekar, S. (1998). Learning by Design from Theory to Practice. In A. Bruckman, M. Guzdial, J. Kolodner, & A. Ram (eds.), Proceedings of International Conference of the Learning Sciences 1998 (pp. 16-22). Atlanta, Georgia.

Linn, M. C. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), (pp. 103-126).

Puntambekar, S., Nagel, K., Hubscher, R., Guzdial, M., Kolodner, J. L. (1997). Intragroup and Intergroup: An Exploration of Learning with Complementary Collaboration Tools. In R.Hall, N.Miyake, & N. Enyedy (Eds.), Proceedings of Computer-Supported Collaborative Learning '97 (pp. 207-214). Toronto, Ontario, Canada.

Scardamalia, M., Bereiter, and Lamon, M., "The CSILE Project: Trying to Bring the Classroom into World 3", in Classroom Lessons: Integrating Cognitive Theory and Classroom Practice, K. McGilly, Ed. Cambridge, Mass.: MIT Press, 1994, pp. 201-228.

Suthers, Daniel D., Toth, Eva E., Winer, Arlene (1997). An Integrated Approach to Implementing Collaborative Inquiry in the Classroom. In R.Hall, N.Miyake, & N. Enyedy (Eds.), Proceedings of Computer-Supported Collaborative Learning '97 (pp. 272-279). Toronto, Ontario, Canada.

Tabak, Iris and Reiser, B.J. (1997). Complementary Roles of Software-based Scaffolding and Teacher-Student Interactions in Inquiry Learning. In R.Hall, N.Miyake, & N. Enyedy (Eds.), Proceedings of Computer-Supported Collaborative Learning '97 (pp. 289-298). Toronto, Ontario, Canada.

Authors' addresses

Janet L. Kolodner (jlk@cc.gatech.edu)
GCATT Building, EduTech; 250 14th Street, NW, Suite #138; Atlanta, GA 30318. Tel. (404) 894-7435. Fax (404) 894-5041.
Kristine Nagel (kris@cc.gatech.edu)
GCATT Building, EduTech; 250 14th Street, NW, Suite #138; Atlanta, GA 30318. Tel. (404) 894-7435. Fax (404) 894-5041.