1. The Prototype Home

The Aware Home prototype is currently under construction. This home will have two identical and independent living spaces, consisting of two bedrooms, two bathrooms, one office, kitchen, dining room, living room and laundry room. In addition, there will be a shared basement with a home entertainment area and control room for centralized computing services. We expect construction of the house to be complete by the end of the 1999 calendar year.

The reasons for building two independent living spaces are to allow for controlled experiments with technology and to allow inhabitants to live on one floor while prototyping or providing demonstrations on the other floor. We anticipate that the initial occupants will be students involved in the research project living in only one floor of the house. A longer-term goal is to have both floors occupied by a family or elderly occupants, as these are the targeted groups of our research. These occupants will give more realistic feedback on the performance of systems within the house.

We anticipate that the house will not be ready for us to occupy for nearly nine months from the date of this writing. We are moving ahead with the research project in this interim period. In order to test some of the systems that we plan to use in the house, we have constructed a prototype room in our lab. The room was constructed using standard house construction techniques to resemble the actual house as closely as possible. We expect to implement some systems immediately to test their effectiveness in the house. These systems include human position tracking through ultrasonic sensors, RF technology and video, recognition through floor sensors and vision techniques.



Fig. 1, 2. First and second floor plan, front elevation of house

2. Our background

Rather than provide an exhaustive survey on home automation, intelligent environments and other related work, we provide a brief summary of our research backgrounds and relevant work we have conducted at Georgia Tech on living laboratories for ubiquitous computing research and computing in the home.

1. Context Awareness and Ubiquitous Sensing

Humans, in general, are quite successful at communicating complex ideas to each other, due in part to an implicit shared understanding known as context. When humans interact with computers, there is very little shared understanding or context. However, it is becoming increasingly possible to build sensors that can help a computational environment to interpret and begin to understand the contextual cues of its occupants. In augmented environments, such as the Aware Home, we need to provide the capability for computational services to take advantage of these soon-to-be ubiquitous sensing capabilities. For example, we have built vision-based sensors to track multiple individuals in an environment (Stillman et al., 1999) and we are trying to use similar signal processing techniques to build a smart floor interface that can identify and track people walking across a large area. There are many compelling applications for these sensing technologies throughout a home, such as support for the elderly or finding lost objects, or in specialized spaces within the home, such as the front door or the kitchen.

However, progress in the sensing technologies needs to be matched by progress in supporting the rapid development of applications that use sensed information. These applications are what we call context-aware applications, and we are building a software infrastructure to assist in their rapid development (Salber et al., 1999). To date, the context-aware development infrastructure has been applied to controlled situations in an office environment and we see the Aware Home as a very valuable resource for exercising much of the capabilities we want to provide in a robust and programmable software infrastructure.

2. Individual Interaction with the Home

One interesting direction of this work occurs when we consider sensing on the body, as is done in wearable computing, in conjunction with sensing off the body, as is typical in an instrumented environment. In this environment, human-home symbioses becomes important as a means to provide as seamless interaction as possible with the home. Wearable computers and intelligent environments allow the delivery of convenient, personalized information and entertainment services at almost any time and in any context. However, there is very little work on how wearable computing and any computing infrastructure attached to the home environment should interact together on behalf of a user. By learning about users' habits and behavior, embedded systems in the home may perform complex, seemingly intelligent tasks automatically. Part of the technological and social challenge is determining where to put various interaction and sensor technologies for maximum benefit.

The Aware Home infrastructure is an excellent chance to obtain general information about a user while at home, and a wearable computer can gather data wherever the user may go. The home can contain a large amount of computation and infrastructure for sensing at a distance, while a wearable has the advantage of immediate and intimate contact with the user. The data gathered on the wearable might then be filtered and released to the environmental infrastructure as appropriate. On the other hand, the wearable may draw on the house's data resources to cache important information for the mobile user when away from the house. Thus, an automated wireless collaboration between the platforms seems appropriate, with the user placing limits on the type and level of information transferred between his personal and environmental infrastructure. We will develop such infrastructure interactions and explore some of the technical and social benefits.

3. The Smart Floor

In ubiquitous computing, knowing who is where and what they are doing is central to enabling intelligent behavior. In the Smart Floor project, we are addressing the who and where aspects of this problem: we have created a system to identify and locate a person based solely on his or her footsteps. In this system, we will place ten strategically sized and located force-sensitive load tiles throughout the Aware Home to gather footstep data from occupants. The tiles are flush with the floor and consist of a metal plate supported by four industrial load cells; the data we gather from these tiles are known as ground reaction force (GRF) profiles. We have gathered sets of training data to create footstep models for each person; we then compare each new GRF profile against these models and search for the best match. We have used two techniques to create models for each user: Hidden Markov Models (HMMs), and simple feature-vector averaging. Work similar to our HMM approach was described recently by Addlesee, et al. (1997). For a reasonably sized user population (on the order of 10 people), the GRF profiles are unique enough to correctly identify the user over 90% of the time. We are currently characterizing the system more fully, including examining the effect different shoes have on GRF profiles, and comparing the Smart Floor to identification technologies such as face recognition. We are also investigating the relevance and robustness of the particular features chosen for our feature-vector models, and are studying other methods for creating and evaluating the user models, such as neural networks.

As mentioned above, we will strategically size and locate our Smart Floor tiles throughout the Aware Home. In addition to the tiles, we are exploring other technologies to track users more finely throughout the house; we are evaluating systems based on grids of piezoelectric wires, grids of deformation sensitive optical fibers, and networks of vibration sensors attached to the underside of the flooring. In this tracking system, we will establish identity and a location landmark using the Smart Floor tiles and track the movement of users with the finer grained system.

 

4. Finding Lost Objects

Designing the next generation of applications for homes is different from designing for offices. In offices, time and how it can be used is determined by the rhythms and culture of the organization. Movement is restricted and often monitored. Tasks and activities are circumscribed and determined by the organization. Work is generally couched in terms of productivity, efficiency and profit, emblems of Tayloristic notions of work. But what are the frameworks that guide activities at home? Can we apply such notions as productivity and efficiency or are there home-based concepts that guide the way people use space and existing artifacts? There we are free to use time as we wish, to undertake the kinds of activities that we like, to come and go as we please, and use resources as we see fit. At home we are free to choose how space and time are structured, what activities are undertaken and who is involved. For these reasons, homes are what we call "free choice" environments. Because designing for such environments is challenging, it is critical that we develop methodologies that ensure that the latest technological advances are being funneled into useful applications. This thread of research will apply more qualitative techniques for uncovering applications for technology in the home.

Using qualitative techniques, one specific activity we will attempt to understand deeply is how people lose and find objects around the home. This study will be used to support our Frequently Lost Objects project mentioned above. We will use ethnographic techniques to study what people lose frequently, why these things become lost, how people go about finding these objects, and how other people in the household may assist in finding what has been lost. While we have already outlined a technical solution to the problem of finding lost objects in the Aware Home, we are not irrevocably committed to this solution and we hope that our qualitative study of this problem can help to direct modifications or our existing system or help inform the design of another solution altogether. We also hope to use the relationships with the families who take part in this narrow study as a jumping-off point for additional broader studies of home life.