PAC personalized adaptive control

One of the driving applications of ubiquitous computing is universal appliance interaction: the ability to use arbitrary mobile computing devices to interact with arbitrary appliances such as TVs, printers, and lights. Because of limited screen real estate and the plethora of devices and commands available to the user, a central problem in achieving this vision is predicting which appliances and devices the user wishes to use next so that interfaces for those devices can be made available.

With Personalized Adaptive Control we took seriously the idea that a user will not necessarily think of devices as separate entities; therefore, rather than focus on which device the user may want to use next, we developed methods for automatically discovering the user's common tasks (e.g., watching a movie, or surfing TV channels), predicting the task that the user wishes to engage in, and generating an appropriate interface that spans multiple devices.

The task is seemingly straightforward but requires extending semi-supervised machine learning techniques and extending those techniques into the realm of HCI.

At one level, PAC is about building a specific application that helps users by creating better remote controls. Traditional remote controls typically allow users to activate functionality of a single device. Given the exploding number of devices and the fact that users activate only a subset of functionality across devices to accomplish a particular task, it is attractive to consider a remote control directly supporting this task-specific behavior.

At another level, PAC is about representations that help in activity discovery. In this case, we represent low-level actions as distributions over the other low-level actions that follow them. First, this makes sense in that it means actions are distinguished from one another in what they enable or predict about the environment. Second it affords theoretically elegant and computationally feasible similarity metrics (e.g., KL divergence and histogram-matching techniques) and thus provides a mechanism for clustering and discovering useful activities or tasks.

So far, we have shown that it is possible to discover and cluster collections of commands that represent tasks and to use history to predict the next task reliably. In fact, we showed that it is an easier problem to solve. Moreover, we show that tasks vary from user to user. At present, we are following up on our work by implementing a full working system and continuing a set of user studies.