1.Usability and User-centered Design in Biomedical Informatics:Using computers to help patients! Not hurt them.
Katrina M. Romagnoli
Dr. Harry Hochheiser
7/26/13
CoSSBI
2.Who knows how to make a peanut butter and jelly sandwich?
Write down the instructions for making a PB&J.
3.Not so simple, right?
Try doing that for a complicated computer system used by a hospital.
Or a radiation delivery system.
Or a medication delivery system.
4.Challenge
Computers in medicine can help people.
They can also kill them.
How do we build computer systems in healthcare that help clinicians better care for their patients?
5.“Radiation Offers New Cures, and Ways to Do Harm”Walt Bogdanich, NY Times, January 23 2010
“As Scott Jerome-Parks lay dying, he clung to this wish: that his fatal radiation overdose — which left him deaf, struggling to see, unable to swallow, burned, with his teeth falling out, with ulcers in his mouth and throat, nauseated, in severe pain and finally unable to breathe — be studied and talked about publicly so that others might not have to live his nightmare.”
“A New York City hospital treating him for tongue cancer had failed to detect a computer error that directed a linear accelerator to blast his brain stem and neck with errant beams of radiation. Not once, but on three consecutive days.”
6.“Radiation Offers New Cures, and Ways to Do Harm”Walt Bogdanich, NY Times, January 23 2010
“Shortly after 11 a.m., as Ms. Kalach was trying to save her work, the computer began seizing up, displaying an error message. The hospital would later say that similar system crashes “are not uncommon with the Varian software, and these issues have been communicated to Varian on numerous occasions.””
“An error message asked Ms. Kalach if she wanted to save her changes before the program aborted. She answered yes.”
“When the computer kept crashing, Ms. Kalach, the medical physicist, did not realize that her instructions for the collimator had not been saved, state records show. She proceeded as though the problem had been fixed.”
7.Therac-25
Medical linear accelerator
Radiation Delivery for Cancer Treatment
Therac-25 - Third iteration
First that was completely computer controlled
8.Therac-25
Treatment suspend – must reboot
Treatment pause – single key restarts
Up to 5 times
“Error messages provided to operator were cryptic, and some merely consisted of the word MALFUNCTION followed by a number from 1 to 64 denoting an analog/digital channel number”
Manual did not supply or describe these codes
Operators did not understand errors
One clinic: average of 40 dose-rate malfunctions/day
9.Therac-25 causes
Many – bad software, poor practices, lack of defensive design
“Safe versus Friendly User Interfaces: Making the machine as easy to use as possible may conflict with safety goals. Certainly, the user interface design left much to be desired, but eliminating multiple data entry and assuming that operators would check the values carefully before pressing the return key was unrealistic.”
Speed of performance should not always trump error rate
10.Fluorouracil Incident Root Cause Analysis
Chemotherapy medication incident
4-day dose of fluorouracil delivered in 4 hours
28.8 mL entered as rate. - per hour
3/5 participants entered incorrect data when programming pump
5/5 confused with one or more aspect of pump operation
11.Fluorouracil
12.Who do we blame?
“User error”: the user just shouldn’t have made the error.
Simple, right?
….Or, the manufacturers should make them easier to use!
That’s simple, right?
13.What are the goals?
Understand the problems that a system must solve
what does it have to do?
Understand the users who will use the system
What do they know?
What can they do?
14.Fundamental Theorem of Biomedical Informatics
15.Human-Computer Interaction
Definition: “the discipline concerned with the design, evaluation, and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them”
- ACM SIGCHI Curriculum Development Group, 1992
Understand human abilities, needs, goals in order to design systems that will help meet those goals and needs
16.Usability
Usable: one can use a system to successfully complete goals and tasks.
Unusable: a system is unusable if it interferes with the successful completion of goals and tasks.
A system that has crashed is unusable.
17.What usability is not
Not “user-friendliness”
Are you guys old enough to remember this creature?
He was “friendly”
Not usable!
Being “cute” is not the same
as being helpful.
18.Human Factors
Ergonomics (or human factors) is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance.
--International Ergonomics Association
19.Human abilities
We are limited by
Perception
Attention
Memory
Language (reading, speaking, listening)
Problem solving (planning, reasoning, decision making)
Learning
Mental models
20.Visual Perception
Color
Contrast
What is the time?
What is the time?
What is the time?
What is the time?
What is the time?
What is the time?
21.Pre-attentive Processing: Color selection
22.Pre-attentive processing: Shape Selection
23.Not pre-attentive: conjunction of shape and color
24.Motor abilities
How quickly we can move around, with what parts of our bodies
Cost of various actions
Mouse vs. keyboard
Cut down on number of steps required
25.Attention
http://www.youtube.com/watch?v=47LCLoidJh4
How do you alert user of a problem?
Each claim on attention has a cost
Context-shift takes time, loses focus
When do you interrupt the user?
How do you interrupt the user?
26.Short term Memory
Temporary recall
7+/- 2 ‘chunks’ of information retained
George Miller- strings of numbers
Rapid access, rapid decay
Repeat to send to long-term
Don’t overtax!
27.User-centered Design
Incorporate the user(s) (needs, goals, limitations) into the design and development process
At every step, not just at the end, or just at the beginning!
28.It’s not as obvious as it sounds.
Requires an investment of time, money, and effort
Many types of studies at various phases of development
29.Seems like a pain! Why bother?
Reduces implementation issues, which are more costly to fix later
Reduces costs over the long run
Every dollar invested in usability returns $10-100! (Karat, 1994)
30.How does it happen?
A variety of studies performed at various points during development
Starts before you even start writing code!
Continues the whole way!
Even after it’s implemented, you can still work on usability!
I know it sounds Sisyphean, but remember the previous slide: it’s worth it.
31.Most important step
Identify your users!
Remember: users are not just the people who will be directly interacting with the software.
They could also be the people paying for it.
The people who will be keeping it running (IT support staff)
Identify roles versus users
A clinician taking her child to the pediatrician is interacting with the EMR as a patient, not a clinician.
32.First Steps
Contextual Inquiry
What task(s) will this resource support?
What task(s) do the users want/need it to support?
What is the current environment?
Stakeholders?
Current workflow?
What works currently?
What doesn’t work currently?
33.Contextual Inquiry
Observations
Watch your users in the wild.
Semi-structured interviews
Talk to your users, sort of in the wild.
Surveys
Talk to your users, at a distance.
Focus groups
Talk to groups of your users, sort of in the wild.
34.Let’s try it!
35.Build models
Stories, use-cases, scenarios
Diagrams, pictures
Describe the work flow
Who does what? When? How?
What can be improved? Where are the points that things can go wrong?
36.Next step: Prototyping
No, you may not begin programming yet. Soon, I promise.
Draw a prototype.
Could be as simple as a paper drawing of the possible interface, or Power Point slides.
Draw another prototype, and maybe another.
Multiple options are better than no options.
37.Prototyping, cont.
Show your prototypes off to some users!
Have them mimic doing some tasks.
Can they find the menus? Can they figure out where they need to go, what they need to do?
Ask them what they like.
Ask them what they do not like.
Incorporate changes into new prototypes!
Rinse and repeat, until you (and your users) are satisfied enough.
38.Interface Design in 5 min
Art, not science
Interaction Style
“First, do no harm”
Build on familiar models
Metaphors
Don’t mess with convention!
Less is always more
Get the basics right.
Avoid 3D
Unless necessary
39.Harry’s 8 Golden Rules (via Ben Shneiderman)
Strive for consistency
Cater to universal usability
Offer informative feedback
Design dialogs to yield closure
Prevent errors
Permit easy reversal of actions
Support internal locus of control
Reduce short-term memory load
40.Is this a good interface?
41.Now you may program.
Go ahead.
42.Ok, stop!
That’s plenty.
Don’t get too far into development without showing it to some users and doing some evaluation.
The bigger it is, the harder it will be to make changes.
43.Lab evaluations: No users!
Heuristic analysis (Nielsen, 1994): look at these throughout!
Learnability: should be easy to learn
Efficiency: an experienced user should be very productive using it
Memorability: features should be easy to retain
Errors: System should minimize errors, support error detection and recovery
Satisfaction: the user experience should be subjectively satisfying (and not aggravating)
44.Let’s do it!
45.Lab evaluations: no users!
Cognitive Walk-through
Characterize the cognitive processes of users when engaged in a task
Analysts (NOT users)
Go step by step through a task from beginning to end
Track mouse clicks, cognitive actions, responses from the system, and screen transitions.
46.Lab evaluations: no users!
Questions to ask during a Cognitive Walk-through:
Are there too many mouse clicks?
Are the responses from the system clear?
What does the user have to know to reach their goal?
Where could they go astray? How likely is it that that could happen?
How can you improve these things?
What can you minimize to make it even simpler?
Who should do lab analysis?
Ideally, not the person/people building the system. Having fresh, objective eyes is a good thing.
Not always possible. But do your best to be objective and critical of your own work if you have to do it yourself.
47.Rinse. Repeat.
Make improvements.
Do it again.
Make improvements.
Do it again.
This is known as iterative design.
48.Analysis in the Wild
It’s time to show your software off to the people who will use it!
Find out if they CAN use it.
Can they do their work on it?
Is there an improvement compared to the current environment?
What problems do they experience?
49.Task Analysis
Recruit users from each stakeholder group
Give them a series of tasks that your software supports.
Record what they do!
Screen recording
Video recording
Audio recording
Ask them to “Think Aloud”.
50.Think Aloud
Think aloud means vocalizing what is going through your head while you do a task
You want to know:
What they are thinking
Why they are doing what they are doing
Why did they click that? What did they expect to happen? What are their assumptions about your system?
So if they have a problem, you can know better what caused it.
51.Analyze the data
Someone (probably you) gets to watch a lot of video.
Code every step, every mouse click, every selection, every error, every success.
Measure the error rate.
How many users fail at doing a task? What tasks? Where do they go wrong? Why?
52.Experimental Deployment
Deploy your software in the wild on a small scale
Randomly (if possible) select some users to use that software for their work, and others the current software
Example: compare a new computerized physician order entry (CPOE) tool to the old one
Measure the difference!
53.Example: Avansino & Leu, 2013
Effects of CPOE on Provider Cognitive Workload: A Randomized Crossover Trial
54.Rinse. Repeat.
Do this iteratively, too.
Make improvements again.
Evaluate it again.
55.So which evaluations should you do?
In an ideal world, all of them, repeatedly, with as many users as possible.
We do not live in an ideal world.
We have to make compromises.
56.Lab studies (heuristics, CW)
Pros:
Inexpensive
Easy
Will cover the vast majority of problems
Do not have to recruit or entice users to participate
Not time consuming
Cons
Do not use real users
Will miss some problems
Will not get as rich of data
Data might not be objective, especially if evaluation is done by developers
57.Non-lab studies (Task analysis, experiments)
Pros:
Rich data
Will find problems that lab studies might not
More objective
Use real users
Cons:
Expensive
Time-consuming
Difficult to recruit users
Lots of data to analyze
58.At the very least…
Do user needs inquiries.
Do the inexpensive lab studies.
If you can, do some evaluation with real users.
59.Whatever you do…
Try to understand the environment first. You are not the user, so you cannot know what the user needs intuitively.
Don’t NOT evaluate. You will doom yourself to failure if you skip evaluation.
60.Next up….
Dr. Harry Hochheiser
Our local usability guru! (and source of much of the material in this lecture!)
Let’s do some evaluation together!
OpenMRS