Severity and impact of computer user frustration: A comparison of student and workplace users

1 Introduction

Everyone is familiar with computer problems and the ensuing frustration that results when, yet again, your program crashes with no warning, taking the last 30 min of work with it. Pop-up advertisements that mimic typical error messages can be both misleading and frustrating. Dialog boxes that are written in a confusing manner can also lead to lost work. Frustration can be defined as when the computer acts in an unexpected way that annoys users and keeps them from reaching their task goals. Frustration is a common theme among computer users who must deal with many annoying delays, incompatible files, and indecipherable menus. Frustrating experiences could be alleviated if more attention was paid to designing interfaces that typical users can understand.

These challenges are well-known by individual users, but less is known about the causes and effects of these frustrations. How much time is lost on a daily basis as we struggle with our machines? How do these experiences affect our mood, our days, our being? What role does our prior experience with technology play? Which factors reduce the level of frustration? In addition, how do these frustrating experiences impact on family, community or workplace?

We believe that user frustration is a significant issue that is closely tied to the digital divide. Even if universal access to technology is attained, users will still have to struggle with poorly designed computer interfaces (Kling, 2000). For the effective use of technology, careful attention must be given to documentation, tutorials, training, online user assistance, and helpdesk support (Lazar and Norcio, 2001). Kling (2000) recognizes that easy-to-use interfaces, user support, technical skills, and a network of people who can help, are part of the social access to technology, as opposed to technological access. Even with the most up-to-date hardware, software and network connections, users may still find poorly-designed technology hard to use (Kraut et al., 1996). Simply providing the technology to economically disadvantaged individuals is not enough; successful bridging of the digital divide requires improved designs (Kling, 2000). In addition, previous research has shown that the quality of the network connection plays a role in causing or reducing frustration (Ceaparu et al., 2004; Lazar et al., in press).

The community networking and software project developed at MIT for the residents of Camfield Estates, a low-income housing community in Roxbury, MA, is a good example of using technological resources to improve the economic situations and overall lives of people (Pinkett, 2002). Successes can only occur when users have well-designed systems that are not frustrating, and support to utilize the technology effectively.

This exploratory study examines the factors that influence the experience of frustration in computer usage. The computer frustrations of 157 users (107 students, 50 workplace users) are examined through modified time diaries. Individuals' prior experiences, psychological characteristics, level of computer experience, and social system are all examined to determine how they influence the frustrations that users face with their computers. In addition, factors such as the importance of the task that was interrupted, the frequency of occurrence (both of same and different frustrations), and the amount of time or work lost as a result of the problem are also examined to determine how they affect the experience of frustration. The existing psychological literature on frustration provides a foundation for the examination of the frustration process in computer use. The goal of this paper is to help elucidate the nature of the frustration experience with computer use. In addition, the outcomes of the study provide guidance to designers of more structured research experiments.

2 Background literature

2.1 Frustration

Frustration occurs when there is an inhibiting condition, which interferes with or stops the realization of a goal. The frustration level experienced by an individual clearly can differ, depending on the circumstances surrounding the frustrating experience, and on the individual involved. One major factor in goal formation and achievement is goal commitment, which refers to the determination to try for and persist in the achievement of a goal (Campion and Lord, 1982). Research on goal theory indicates that goal commitment has a strong relationship to performance and is related to two factors: the importance of the task or outcome and the belief that the goal can be accomplished. Individuals will have a high commitment to a goal when the goal is important to them and they believe that the goal can be attained (Locke, 1996). The importance of the goal to the individuals and the strength of the desire to obtain the goal (Dollard et al., 1939) will affect the level of goal-commitment as well as the strength of the subsequent reaction to the interruption. Self-efficacy, the belief in one's personal capabilities, can also affect goal commitment (Locke and Latham, 1990). The belief about how well a task can be performed (self-efficacy) when it involves setbacks, obstacles, or failures may also affect how committed individuals are to that goal (Bandura, 1986).

2.2 User frustration with computers

User frustration can be defined as when the computer acts in an unexpected way that annoys the user and keeps the user from reaching their task goals. There are many possible causes of user frustration with computer technology. For instance, a software application may crash, an error message may be unclear, a misleading pop-up advertisement may appear, or an interface can be confusing (Preece et al., 2002). Users have to respond to something unexpected and unclear that interferes with their task goals. Users can lose work and waste time. In the context of the social psychological research literature, frustration occurs when users cannot attain their task goals. The causes of the problem could include poor interface design, computer hardware or software failure, or even the users' lack of knowledge regarding the computer, but the result is the same: users cannot complete their tasks, and may have an emotional response (frustration!) to the inability to attain their goal. This is especially true as the Internet changes the nature of the computer user population, to include many more non-technical people and people with little computer experience (Shneiderman, 2000; Cummings and Kraut, 2002).

Frustration with technology is a major reason why people hesitate to use computers, or avoid computers altogether. For instance, in a recent study, it was reported that a large percentage of people (42%) do not go online, because they find the technology to be too frustrating and overwhelming (Pew Internet and American Life Project, 2003). This is not surprising; previous research on user frustration found that users wasted nearly one-third to one-half of the time spent on the computer, due to frustrating experiences (Ceaparu et al., 2004). Other research has found that when computers are frustrating, it can lead to lower levels of job satisfaction and even increased blood volume pressure and muscle tension (Murrell and Sprinkle, 1993; Scheirer et al., 2002). In addition, users frequently experience a sense of rage, where they want to do things such as smash a computer screen and throw computer parts out a window, and nearly 80% of users in one study had cursed out loud towards a computer (Norman, 2004).

Other research has focused on computer anxiety. For example, people with low computer self-efficacy may be more anxious (Brosnan, 1998; Meier, 1985) and more likely to view the computer suspiciously and react with great frustration when a problem occurs, especially when they have run into it before. Different levels of anxiety will affect performance when a problem unforeseen or unknown occurs, causing anxious people to become more anxious (Brosnan, 1998). On the other hand, the level of experience may temper this if the prior experience increases computer self-efficacy (Gilroy and Desai, 1986) by lowering anxiety and reducing frustration when a problem occurs. The perceived ability to fix problems on the computer, as well as the desire to do so may also affect levels of frustration. If instead, these problems are seen as challenges, they may not be as frustrating, which is most likely directly related to level of prior experience as well as computer self-efficacy.

There is a wide body of literature on interface usability (e.g. Nielsen, 1994; Carroll and Rosson, 2002; Shneiderman and Plaisant, 2005), covering both the interface design and the processes for ensuring good interface usability. Good interface usability prevents frustration. Frustration occurs when poor interface usability leads to time lost or failure to attain goals. For instance, if a user's web browser crashes, this is poor usability. However, knowledgeable users might avoid frustration if they know how to respond to the crash, they do not lose work, and they can attain their goal. By contrast, if a poorly designed dialog box appears, thus confusing the novice users, and causing them to lose important work and waste time, this could be very frustrating. The direct causes of frustration are the time lost, work lost, and the failure to attain the goal.

Frustration theory indicates that it is the interruption of a goal or task that causes individuals to become frustrated. Two categories of factors affect the level of frustration: the incident and individual factors. The incident factors experienced by end users include the level of goal commitment, the severity of the interruption, and the strength of the desire to obtain the goal. Goal theory tells us that experience, self-efficacy, and the importance of the goal all affect the commitment to the goal or task. When the goal interruption occurs, the level of goal commitment, measured in terms of the importance of the task to the user, will affect the severity of frustration experienced by individuals directly. Individual factors affecting the strength of the frustration include computer experience, mood and other psychological factors, and the cultural and societal influences upon the individual. A computing frustration model has been described in the literature to help synthesize these various theories, and more detail is provided in (Bessière et al., 2004).

3 Research methodology

We considered multiple methods to measure errors and frustrations. For instance, surveys and interviews could be used, however, users trying to recall frustrations from their past experiences often over-estimate or under-estimate the level of frustration and the time wasted (Fowler, 1993). Data logging can be used only to measure when the computer encounters an error state, which could be different from the user perception of a frustrating experience. There are many events that are frustrating for users (such as spam or pop-up advertisements), and occur when the system is operating in a correct state. A more attractive way to study frustration is a time diary, because it records immediate impression as users work on their everyday tasks. It also can be used to measure the time lost due to these frustrating experiences. Time diaries have proven to be more accurate because users record their time lost immediately as the losses occur, rather than attempting to remember the information at a later time (Robinson and Godbey, 1997).

While we based our work on a theoretical foundation, these first studies were exploratory or formative in nature. We debated using physiological measures of frustration such as heart rate, blood pressure, or facial muscle constriction, but to maintain a naturalistic setting we simply asked users to rate their frustration for each incident and for the overall session. We captured individual factors such as computer experience through pre-session survey questions and incident factors by a short time diary form when a frustrating experience occurred.

In order to obtain data reflecting typical computer usage, we asked users to work on the computer for a minimum of 1 h, on tasks of their own choosing. Because self-set goals are more meaningful to individuals, they may be better than assigned goals that may be unclear or be rejected (Locke et al., 1981). Users worked on tasks of their own choosing as opposed to assigned tasks. In order to be able to collect data on tasks that are important to the individuals, it was important that the tasks were not pre-assigned. Prior to the session, users filled out a short pre-session questionnaire assessing demographic information, computer experience and attitudes, and mood. After completing the 1 h long data collection period, users filled out a five question post-session survey.

The pre-session survey (Appendix A) asked respondents about demographic information, computer experience and attitudes, level of computer anxiety, and mood. Previous research indicates that level of computer experience or perception of computer self-efficacy can affect subsequent user behavior (Murphy and Coover, 1989; Brosnan, 1998). Questions were chosen after a careful review of previous research on the Computer Aptitude Scale, assessing computer attitudes, computer anxiety/confidence, and computer liking (Loyd and Gressard, 1984; Nash and Moroz, 1997). Stemming from this, it was hypothesized that prior experience and level of perceived knowledge will affect individuals' level of frustration as well. To assess, the overall state of the individuals, we included three questions dealing with overall life satisfaction, general mood, and how often they get upset over things. The post session survey (Appendix B) consisted of five questions to assess mood after the session, how frustrated overall the individuals were after the session, how these frustrations affect the rest of the day, and the frequency and typical nature of the frustrating experiences during the session. The frustration experience form which includes the diary questions related to this (Appendix C) primarily tracked data related to the causes and responses to frustration.

Both incident-level and individual-level factors were measured in the various questions. Incident-level factors were measured, such as the cause, level of frustration, and response. The severity of interruption was measured by combining the time to fix the problem with the time lost due to the problem (Ceaparu et al., 2004). For example, after a dropped phone line, it may take 6 minutes to reconnect and 10 minutes to rewrite a lost email draft. The strength of desire for the goal is also affiliated with how important the goal was, so importance is also used here as a proxy for strength of desire.

For individual-level factors, measures of mood included satisfaction with life, how often users get upset over things, and general mood. Also of interest are how the frustrating incident affected the users' day, and how frustrated users were, overall, after the session. Computer factors are separated into computer experience/self-efficacy and computer anxiety/attitudes. Computer experience can be measured as years of computer use, hours of computer use per week, and a subjective measure of experience—also effective as a measure of computer self-efficacy. Additional measures of computer self-efficacy include confidence about their ability to fix problems, how much users persevere when encountering a problem on the computer, and how much users thought about unresolved computer problems after being unable to fix them. Computer anxiety is measured with two questions, one on level of comfort with the computer and one on how users react to a problem with a computer. Cultural and societal influences are not measured, as they are expected to be a constant. Our sample is taken from American university and workplace settings, in which it is expected that most respondents will be American. While user frustration as occurs in different cultures is certainly an interesting question to pursue, it lies outside the scope of the current study.

There are two separate user populations within this study. The first population was largely students, recruited from computing classes at the University of Maryland and Towson University. Students self-reported their time diaries, and also observed other users and assisted them with their time diary reports. For the self-reported diaries, students completed a minimum of 1 h of data collection and filled out incident reports for their daily computer use. These students were also asked to observe another person not in the class and gather data on this individual. Users for the observations were enlisted by the students and followed the same methodology as the self-reports. Instead of filling out the frustration experience report forms themselves, however, the observers filled them out, asking the users to talk out loud and share their experience with the observer. Data was collected from a total of 107 users in this portion of the study. The second population was 50 workplace users completed the same data collection process. All of these users were professionals, who recorded their time diaries while using their computers in a workplace setting. Both populations were convenience samples, and therefore, we make not claims for generality. Future work might focus on specific population segments, age groups, professions, education backgrounds, etc.

The methodologies used for the student users and the workplace users were identical. The only difference is that the student users filled out the pre-session and post-session surveys online, whereas the workplace users filled out the pre-session and post-session surveys on paper. All groups filled out the frustration experience forms on paper, because if the computer was already causing frustration, to use the computer to record the frustrating experience would be a complicating factor, possibly causing an increased level of frustration.

Our first analysis of the data from student users (Ceaparu et al., 2004) examined the frequency, cause and the level of severity of frustrating experiences. The three task applications that were the cause of the most frustrating experiences (N=372) were web browsing (120 frustrating experiences), email (50 frustrating experiences), and word processing (44 frustrating experiences). The specific causes of frustration most often cited were error messages (35), timed out/dropped/refused connections (32), freezes (24), long download time (23), and missing/hard-to-find features (23). Most users indicated that the frustration experience had happened before (277), as frequently as several times a month (40), week (54), or even several times a day (60).

Our first analysis of the data from workplace users (Lazar et al., 2005) analyzed similar data. The 50 users reported 149 frustrating experiences, caused primarily by the task applications of word processing (34), e-mail (28), and web browsing (17). The specific causes of frustration cited most often were system crash caused by operating system (21), missing/hard-to-find features (19), and application crashes (11).

The most disturbing result from our student and workplace users was the large amount of time lost due to frustrating experiences. One-third to one-half (depending on whether outliers were included) of the time spent in front of the computer was lost due to frustrating experiences. This is clearly a large amount of time, possibly reducing productivity, and the impact may also harm interpersonal relationships and influence user mood. These social impacts of frustration for the student users were analyzed by Bessiere et al. (in press). They examined the role of instrumental factors (attributes of the incident situation) and dispositional factors (attributes of the individual) on frustration severity. Their results demonstrate the importance of self-efficacy in reducing frustration.

The first contribution of this paper is to present new data related to the emotional impact of frustration on workplace users of computers. The second contribution is to compare the impacts of frustration on student and workplace users. To accomplish this, we analyzed the data from student and workplace users by the same measures. Section 4.1 presents the analysis for student users, Section 4.2 presents the analysis for the workplace users, leading to Section 4.3, which compares the student and workplace data. We close with Section 5 that discusses the results and makes recommendations.

4 Results

We correlated the incident and session frustration with the individual and incident factors. Incident frustration was measured by time diary forms asking for frustration levels. Session frustration was measured by post-session survey asking for their mood and the effect of the session on the individuals' day. Incident factors included time lost, time to fix, and importance of the task. Individual factors examined are demographic factors, computer experience, computer anxiety, computer self-efficacy, and mood (See Appendices (A)–(C) for questions). Data will be presented in two separate sections, one for student data and one for workplace data.

4.1 Student data

4.1.1 Demographic information

Out of 111 total users that took part in this phase of data collection, four were discarded due to technical problems with the database. There were a total of 372 frustrating experiences reported (188 reported by females and 184 by males). The users were 54 females, and 53 males, and were composed mainly of college undergraduates (75.9%). Since a majority of these users were students, exactly half of the users were under the age of 22. The remaining half ranged from age 22 to 80. Respondents also reported a high level of self-reported perceived computer experience (see Fig. 1, N=107, mean=6.88, sd=1.88), and 40% of the respondents reported either being a computer professional or student.

Age is negatively correlated with experience in our study (r=−.278, p<0.001) indicating that younger users seem to have more experience with computers, an expected result.

4.1.2 Incident frustration level

Incident frustration level is measured on a scale of 1 (not very frustrating) to 9 (very frustrating) for each incident occurring in the study (N=372, mean=6.74, sd=2.13) Fig. 2 displays the frustration levels for the 372 frustrating experiences.

Incident frustration level is quite significantly skewed towards the high end, indicating that users are often very frustrated by problems that they encountered with their computer. Approximately 75% of the incidents reported in the sessions resulted in higher than neutral frustration. In addition, almost 50% of the incidents (45.7%) resulted in a frustration score of 8 or 9, the highest levels of frustration possible.

In addition to frustration level, we asked users to record their feelings after the incident, in an effort to determine what kind of reaction was elicited from them after a problem with the computer (see Table 1 ). Note that for two frustrating experiences, users did not respond to this question.

Here, we see that 42% of the users in the study have a resultant feeling of being mad at the computer after a frustrating experience. Only 4% of the users reported being angry with themselves, but 12% of them reported feeling helpless or resigned, indicating that some users do in fact experience a sense of loss of control when faced with computer problems. On the other hand, 23% of the users were determined to fix the problem, which may be mediated by the high level of experience reported by the users.

4.1.3 Session frustration level

After the session was completed, users answered a few questions on the post-session survey designed to measure how the frustrations that they experienced affected them overall. Overall frustration about the session, how it affected their day, and their mood after the session were all measured (on a scale of 1–9), as well as a question on whether they experienced more or less frustrating incidents in the study, as compared to a typical day (see Table 2 ). Table 2 also includes data on the mood before the session began, which was collected in the pre-session form (Appendix A).

Overall, it does not appear that the session as a whole produced much of an overall affect on the individuals. It would appear that the frustrating incidents encountered were about average for these users. The score for whether they encountered more or less frustrating experiences in an average day was just about 5. The mean for overall frustration was much lower than the mean for incident frustration, indicating that the individual incidents as a whole did not produce a feeling of high overall frustration. Users reported a low mean score for whether or not the incidents taken together affected the rest of their day, as well. Mood scores went down slightly on average after the session, from 6.10 to 5.67.

4.1.4 Correlations

To determine the factors that influence both incident specific frustration and the overall effects of frustration in computer use, we ran correlations with time, computer anxiety, computer self-efficacy, mood, and goal importance (See Table 3 ).

4.1.5 Incident frustration factors

Incident frustration in our users has a high positive correlation with the amount of time it took to fix the problem, the amount of time or work lost due to the problem, and the importance of the task. Computer experience does not seem to impact the amount of frustration experienced, although there is a slight significant negative correlation between the number of hours worked per week and the frustration level, indicating that the number of hours worked per week could lessen the amount of frustration experienced. Whether or not the users would stick with a problem encountered on the computer is negatively correlated with the level of frustration experienced, with a p value of <0.01, meaning that those who are less likely to stick with problems have a higher level of frustration.

4.1.6 Session frustration factors

Overall frustration level after the session on the computer is highly correlated with whether users would stick with a problem until it was fixed, and also with the difference in pre and post session mood. A small but significant negative relationship is also found between the number of years the individual has been using a computer and level of overall frustration.

The individual's, mood after the session has a high positive correlation with level of computer anxiety, perceived level of experience, mood before the session, level of comfort with the computer, perceived ability to fix problems on the computer, and willingness to stick with the problem until it is solved.

There is a negative correlation between the effect on the day and number of years of computer use, level of perceived experience, perceived ability to fix problems on a computer, willingness to stick with the problem until it is solved, and the mood swing between the start and end of the session.

4.2 Workplace data

4.2.1 Demographic information

Fifty users participated with 149 frustrating experiences reported. The convenience samples of users included 37 females and 13 males and were composed mainly of college graduates (78%). Respondents also reported a high level of self-reported perceived computer experience (see Fig. 3, N=50, mean=6.52 sd=2.01) and 48% had used a computer 10 or more years. Females reported 105 frustrating experiences while males reported 44 frustrating experiences. Age is negatively correlated with experience in our study (r=−0.081, p<0.001) indicating that younger users seem to have more experience with computers, an expected result

4.2.2 Incident frustration level

Incident frustration level is measured on a scale of 1 (not very frustrating) to 9 (very frustrating) for each incident occurring in the study. The frequency table shows that 70% of the incidents resulted in high levels of frustration, 7–9 (N=149, mean=7.10, sd=2.15). Fig. 4 displays the data on incident frustration level.

Incident frustration level is quite significantly skewed towards the high end, indicating that users are often very frustrated by problems that they encountered with their computer. Seventy-six percent of the incidents reported in the sessions resulted in higher than neutral frustration. In addition, more than half of the incidents (55%) resulted in a frustration score of 8 or 9, the highest levels of frustration possible.

In addition to frustration level, we asked users to record their feelings after the incident, in an effort to determine what kind of reaction was elicited from them after a problem with the computer (see Table 4 ). Note: in Table 1 (student users), the frequencies add up approximately to the number of frustrating experiences reported, because those forms were entered online, and users were limited to one choice by the electronic form. In Table 4 (workplace users), this data was collected on paper, so users in some cases marked more than one feeling, so the total frequencies add up to more than 149, the number of frustrating experiences cited.

Here, we see that a large number (58) of frustrating experiences in the study result in the user feeling angry at the computer. Only a small number of frustrating experiences (15) caused the user to feel angry at themselves, but in 24 frustrating experiences, the users reported feeling helpless or resigned, indicating that some users do in fact experience a sense of loss of control when faced with computer problems.

4.2.3 Session frustration level

After the session was completed, users answered a few questions on the post-session survey designed to measure how the frustrations that they experienced affected them overall. Overall frustration about the session, how it affected their day, and their mood after the session were all measured (on a scale of 1–9), as well as a question on whether they experienced more or less frustrating incidents in the study, as compared to a typical day (see Table 5 ). Table 5 also includes data on the mood before the session began, which was collected in the pre-session form (Appendix A). There was a statistically significant decrease between mood scores at the beginning of the session and at the end of the session. The mean mood dropped from 7.14 at the beginning of the session to 6.27 at the end of the session (p=.008).

4.2.4 Correlations

To determine the factors that influence both incident specific frustration and the overall effects of frustration in computer use, we ran correlations with time, computer anxiety, computer self-efficacy, mood, and goal importance (See Table 6).

4.2.5 Incident frustration factors (first column)

Incident frustration (measured by a subjective rating) in the workplace users has a highly positive correlation with the amount of time it took to fix the problem (0.263), the amount of time or work lost due to the problem (0.270), and the importance of the task (0.456). There were no other highly significant correlations for incident-level frustration.

4.2.6 Session frustration factors

Overall frustration level after the session on the computer is negatively correlated (−0.368) with computer anxiety. This is surprising, since it would be predicted that users with higher levels of computer anxiety would also have higher levels of frustration. The average importance of the task is positively correlated with overall frustration level, which makes sense, since the higher the overall importance of tasks, the more likely that the frustration level will be high.

The individual's mood after the session has a positive correlation with mood level before the session. This is expected, because despite the average drop in mood before and after the session in the workplace users, if the mood is higher at the beginning of the session, the mood is expected to be close to that previous level at the end of the session.

There is a positive correlation between Time to Fix (Total) and whether the user's day was effected (Affect Day). There is also a positive correlation between Time Lost (Total) and whether the user's day was effected (Affect Day). These two results mean that if more time was wasted due to responding to the original causes of frustration or making up for lost work, then the impact on the individual's day was more severe. This is an important finding.

4.3 Comparison of student and workplace data

When comparing the student and workplace data, only three factors were highly correlated (p<0.01) with frustration in both the student and workplace data: time lost, time to fix, and importance of the task. This shows that the greatest cause of frustration, regardless of the user population, is how important the task was, and how much time was wasted due to the frustrating experience (time wasted caused by both time to fix and time lost). More important tasks and more time wasted led to higher frustration levels. The bad news here is that factors such as years of computing experience, anxiety, or training do not impact the frustration level. Regardless of experience level, when an important task gets delayed for a long time due to a frustrating experience, frustration levels will be high.

Another interesting finding is that there is a correlation (p<0.05) between affect day, with both time to fix and time lost, but only in the workplace user population. The more time wasted due to a frustrating experience, the more likely that the user's day will be affected. These correlations are not significant in the student user population. This might mean that these frustrations have more of an impact on the day of workplace users, rather than student users. This makes sense, as time is generally more limited for workplace users than it is for student users.

On the other hand, self-efficacy is highly correlated for student users in half the categories examined but self-efficacy was not a factor for the workplace users. Experience level, ability to fix the problem, and determination to stick with the problem impacted on frustration, affecting the student's post-experience mood and their outlook for the rest of the day.

While workplace users were frustrated by anxiety over completing the task (p<0.05), student users did not record anxiety over the frustration, but rather their post experience mood was affected. This could be explained for the workplace users by tight project deadlines that must be met. As expected, both the workplace and student users' mood before the incident correlated with the mood after the frustrating experience.

5 Discussion

In the original data collection effort related to user frustration (Ceaparu et al., 2004), users reported high levels of frustration, and large amounts of time lost. In responding to that first study, some skeptics felt that frustration was only a concern for students. They conjectured that workplace users, with more training, better equipment, and more support (including documentation and help desk service), would not be severely impacted and would be less frustrated. However, the data from workplace users (Lazar et al., in press) showed similar levels of frustration to student users, as well as large amounts of time lost due to these frustrations. The data indicates that frustrations do impact on both user populations, and they cause large quantities of time to be wasted.

This paper presents new data on the frustrations of workplace users. Then it applies a common method of analysis to compare student and workplace user frustration. By going beyond examining the causes of frustration and the time lost, this analysis points to the factors that lessen (or aggravate) the level of frustration. The analysis also shows how these frustrations impacted the user's day.

In comparing, the student and workplace users, there were three factors that were strongly (p<0.01) correlated with frustration levels: the time lost, time to fix, and importance of the task. These correlations appeared in both the student and workplace users. Tasks with higher importance that suffer from larger amounts of wasted time will lead to higher frustration levels. This makes sense, and it seems that there is no way around the fact that, when a task is important, and a large amount of time is lost, it will be highly frustrating. Pre-mood was also correlated (p<0.05) in workplace users and highly correlated (p<0.01) in student users, with post-mood. The higher the mood going into the session, the higher the mood was likely to be coming out of the session. Despite this correlation, it is sobering to discover that overall mood levels dropped from beginning to end of session in both student and workplace users. It is sobering to realize that using computers is not as relaxing and pleasurable as technology suppliers suggest in their advertising.

There were some differences that appeared between the workplace and student users. Only in the workplace population were there correlations (p<0.05) between affect day, and both total time to fix and total time lost. That means that the more time lost due to frustrating situations, the more that the user's day will be impacted upon. These frustrations have more of an impact on workplace users than on student users. On the other hand, there were many correlations between frustration levels and both self-efficacy factors and anxiety factors in the student population, which did not appear in the workplace population. This might mean that self-efficacy and anxiety levels do have more of an impact on student users.

6 Implications and conclusions

In previous papers, we addressed the implications for designers, managers, policy makers, users, and IT staff (Lazar et al., in press; Bessiere et al., 2004). We see strong opportunities for redesign of interfaces to provide better dialog boxes, clearer instructions, better wording, more consistent terminology, and improved menus (Shneiderman and Plaisant, 2005). Other improvements include better training, documentation, and online help. Pressure on software developers through market forces is also important. Research advances and government influence also have a role.

In this paper, we address the implications for researchers. The participants in these studies included only two populations of users: student users and workplace users. It is important to continue this research by focusing more on specific user populations with well-documented differences (such as younger users, older users, and users with disabilities). Examining differences within workplace users, such as the amount and type of training, support staff, and patterns of responses, could provide useful feedback for designers. Even expert users, who have a great amount of experience and are not easily fazed, tend to underestimate the large amounts of time that they waste in dealing with frustrating experiences.

Another approach to expanding this research is to examine the frustrations that users face with specific software applications. For instance, some studies have examined the errors that occur while browsing the web (Lazar et al., 2004) and using spreadsheets (Panko, 1998). The frustrations that occur primarily in operating systems have been examined (Mentis and Gay, 2003). Such fine-grained research helps reveal the exact problems in specific applications, so that narrowly-focused design suggestions can be made.

The compelling results from these two user populations justifies further studies with a more well-formed hypothesis testing approach. A next step is to conduct design-oriented studies that lead to suggestions for improvements. Certainly, more research is needed into the frustrations of specific user populations and in specific software applications. Finally, more research is needed into usability engineering methods that assess frustration levels during development of new software applications.

While we can never eliminate computer user frustration, there clearly is an opportunity to do better. The goal of reduced frustration is attainable and the benefit will be widespread; users will be happier and more productive. And there is reason to believe that higher quality software applications will expand the community of willing users.

Acknowledgements

We appreciate partial support from National Science Foundation grant for Information Technology Research (#0086143) Understanding the Social Impact of the Internet: A Multifaceted Multidisciplinary Approach. We thank Katie Bessiere, Irina Ceaparu, and John Robinson for their efforts on the student data collection. We appreciate the devoted efforts of Prof. Shirley Anne Becker of the Florida Institute of Technology and her students Ali Al-Badi and Madhan Thirukonda in preparing the website for data entry, and Kent McKay of Adonix Systems for help in transferring and converting the system for use at the University of Maryland. We also thank Cheryl Schroeder-Thomas for comments on an earlier draft.

The lead author of this article was partially supported by Training Grant No. T42/CCT310419 from the Centers for Disease Control and Prevention/National Institute for Occupational Safety and Health. The contents are solely the responsibility of the author and do not necessarily represent the official views of the National Institute for Occupational Safety and Health.

Appendix A Pre-session survey

Appendix B Post-session survey

Appendix C Frustrating experience report

References