[Back to projects]

How do users understand intelligent systems?

Joe Tullio, Jason Chalecki, Anind Dey, James Fogarty

Intelligent systems must provide users with the ability to understand their actions and ultimately maintain a sense of control over them. To this end, we are studying how people come to understand an intelligent system as they use it in their everyday work.

Domain: Interruptibility

An important concern for office workers is control over interruptions. To achieve this control, they adopt conventions. For example, some of the participants in our study use ceramic hang-tags outside their office doors (left), where red means "do not disturb" and green means "it's ok to come in". Conventions such as these require diligence on the part of the office worker to maintain the proper state of the tag. Otherwise, their colleagues will learn not to trust it.

For this project, we are using statistical models of interruptibility (Fogarty et al., 2005) that are trained on self-reported interruptibility levels over the course of roughly three months.

The estimates generated by these models are displayed on touch-screens situated outside office doors. They show passersby how receptive the person inside the office is to interruptions. A simple color code is used to represent levels of interruptibility, and the most important reasons for the estimate are shown across the bottom of the display.

Four managers have trained models their interruptibility and consented to having displays installed outside their office doors. Meanwhile, nine of their coworkers have volunteered to report on their understanding of the system. Half of the participants are given reasons for the estimates, while the other half are not.

Assessing mental models

Data is being collected in a number of ways. First, weekly interviews are being conducted to establish, in detail, how participants think the system works. By asking them to identify possible inputs, the relative importance of these inputs, and diagramming their relationships, we hope to arrive at a description of the mental models that are possessed by each participant. Second, the office displays present a survey that measures to what degree users agree with the display's interruptibility estimate, whether that estimate was used to decide whether to interrupt, and the importance/urgency of the office visit. In this way, data can be collected in a situated manner, as the system itself is used. Lastly, follow-up emails ask users to elaborate on their answers to the door display survey. Here, they can list reasons why they agreed or disagreed with the displays, or provide any other comments about their usability and usefulness.

By analyzing this data over time and across our two groups, we hope to understand how our participants develop mental models of the system. Through this research, we hope to inform the design of improved user control and feedback for intelligent systems.

In the future, we would like to generalize the findings of this study to other intelligent systems.

Relevant publications:

Tullio, J., Dey, A.K., Chalecki, J., and Fogarty, J. (2007) How it Works: A Field Study of Non-Technical Users Interacting with an Intelligent System. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2007). [Local PDF]