Lakhasly

?Eye tracking for usability evaluation ?Hypotheses

8

Evaluation Through User Participation

..Cont'd

Observational techniques o Think aloud and cooperative evaluation ?Computer logging o Automatic protocol analysis tools

9

Evaluation Through User Participation

..Cont'd

Query techniques ?the evaluators can clarify points of confusion at the time they occur and so maximize the effectiveness of the approach for identifying problem areas.Clearly, it is not usually possible to perform extensive experimental testing continuously throughout the design, but analytic and informal techniques can and should be used.Cont'd

Heuristic evaluation o (Jakob Nielsen and Rolf Molich)

1.Laboratory studies: In the first type of evaluation studies, users are taken out of their normal work environment to take part in controlled tests, often in a specialist usability laboratory.7

Evaluation Through User Participation

..Cont'd

Empirical methods: experimental evaluation ?Ideally, evaluation should occur throughout the design life cycle, with the results of the evaluation feeding back into modifications to the design.Evaluation Through User Participation

4

Evaluation Through Expert Analysis Cognitive walkthrough 1.Field studies: The second type of evaluation takes the designer or evaluator out into the user's work environment in order to observe the system in action.Questionnaires Evaluation through monitoring physiological responses ?Assessment of our designs and test our systems to ensure that they actually behave as we expect and meet user requirements.to assess the extent and accessibility of the system's functionality ?Evaluation Through User Participation Styles of Evaluation ?3

Evaluation Criteria

1.Participants ???2.3.4.2.3.4.5.6.7.8.9.10.1.2.3.4.?

➢ Ideally, evaluation should occur throughout the design life cycle, with the
results of the evaluation feeding back into modifications to the design.
Clearly, it is not usually possible to perform extensive experimental testing
continuously throughout the design, but analytic and informal techniques
can and should be used.
➢ Assessment of our designs and test our systems to ensure that they actually
behave as we expect and meet user requirements. This is the role of
evaluation.

2

Goals of Evaluation

Evaluation has three main goals:

➢ to assess the extent and accessibility of the system’s functionality
➢ to assess users’ experience of the interaction.
➢ identify any specific problems with the system.

3

Evaluation Criteria

1. Evaluation Through Expert Analysis


2. Evaluation Through User Participation

4

Evaluation Through Expert Analysis
Cognitive walkthrough

1. A specification or prototype of the system.


2. A description of the task the user is to perform on the system.


3. A complete, written list of the actions needed to complete the task with the proposed
   system.


4. An indication of who the users are and what kind of experience and knowledge the
   evaluators can assume about them.

5

Evaluation Through Expert Analysis ..
Cont’d

Heuristic evaluation
◦ (Jakob Nielsen and Rolf Molich)

1. Visibility of system status.


2. Match between system and the real world.


3. User control and freedom.


4. Consistency and standards.


5. Error prevention.


6. Recognition rather than recall.


7. Flexibility and efficiency.


8. Aesthetic and minimalist design.


9. Help users recognize, diagnose and recover from errors.


10. Help and documentation.

6

Scale 0-4
0. I don’t agree that this is a usability problem at all.

1. Cosmetic problem only: need not be fixed unless extra time
   is available on project.


2. Minor usability problem: fixing this should be given low
   priority.


3. Major usability problem: important to fix, so should be
   given high priority.


4. Usability catastrophe: imperative to fix this before product
   can be released.

Evaluation Through User Participation
Styles of Evaluation
➢ Laboratory studies: In the first type of evaluation studies, users are taken out of their normal
work environment to take part in controlled tests, often in a specialist usability laboratory.
➢ Field studies: The second type of evaluation takes the designer or evaluator out into the
user’s work environment in order to observe the system in action.

7

Evaluation Through User Participation

..Cont’d

Empirical methods: experimental evaluation
➢ Participants
➢ Variables
➢ Hypotheses

8

Evaluation Through User Participation

..Cont’d

Observational techniques
• Think aloud and cooperative evaluation
✓ the process is less constrained and therefore easier to learn to use by the evaluator
✓ the user is encouraged to criticize the system
✓ the evaluators can clarify points of confusion at the time they occur and so maximize the effectiveness of
the approach for identifying problem areas.
• Protocol analysis
✓ Paper and pencil
✓ Audio recording
✓ Video recording
✓ Computer logging
• Automatic protocol analysis tools

9

Evaluation Through User Participation

..Cont’d

Query techniques
➢ Interviews
➢ Questionnaires
Evaluation through monitoring physiological responses
➢ Eye tracking for usability evaluation
➢ Physiological measurements