A Usability Test Script – George Schneiderman

This is a script that I produced for paper prototype usability testing with teachers. I wrote the script, created the necessary paper prototypes, administered the testing, and updated the designs based on the results.

The “Goal” column states the goal the test subjects were asked to accomplish. The “Interaction” column indicates the “correct” steps in order to accommodate the goal. The “Screen requirement” column describes the screen mockups required for that step of the test.

Scenario:

User is a 6th grade math teacher.
District gives benchmark assessments 4 times a year.
It is currently early in the school year, shortly after Benchmark #1.
Teacher needs to create an initial instructional plan, based on the data from BM#1.

Starting point:

It is about six weeks in the school year. Benchmark assessment #1 has been administered and scored, as have several ad hoc assessments. Default to the user’s class and the desired assessment (BM#1), so usability testing can focus on creating a plan, rather than on setting report parameters.

	Goal	Interaction	Screen requirement
1	“For this scenario, it is about six weeks into the school year, and you have recently administered the first benchmark assessment. Results for this assessment are now available. You want to explore this assessment data, and ultimately use it to create an instructional plan. Tell me what buttons or links you might try clicking, and what you would expect to happen when you did so?” If necessary: “You are now ready to begin creating an instructional plan based on data from the first benchmark assessment. How would you proceed?”	With this very open-ended direction, we are looking to see if the user tries any of the following: 1. Choosing a different assessment, to see standard-level results for that assessment 2. Drilling down on a particular student, to see assessment results for that student 3. Drilling down on a particular learning objective, to see student and item level data. Ultimately, we want the user to click “Create Plan”, preferably while still looking at a class-level learning map for the same class and assessment	1a. Need a class honeycomb screen with: 1 BM assessment, which is selected a few ad hoc assessments ideally, realistic data, which probably means that only a relatively small fraction (~20%) of the standards have data a “Create Plan” link in the left nav
2	[continuation] If necessary: “You are now ready to begin creating an instructional plan based on data from the first benchmark assessment. How would you proceed?”	Read text and understand that the plan will be for the selected class and based on the selected assessment. Type a plan name and optional description. Click “Save”.	2a. “Create a Plan” dialog box, on top of screen 1a.
3	[continuation]	Read “Created Plan” text and click “Close”	3a. “Created Plan” dialog box, on top of screen 1a.
4	[continuation] If necessary: Can you tell which objectives have been designated for whole class reteach and for group reteach?	Look at icons. Or, just look at color of hexes, since no changes have been made yet	4a. Screen 1a with “Choose Students” widget in left panel
5	What would you do if you wanted to add another objective to the “whole class reteach” list, or remove an objective from this list?	Two options: 1. Right click on an objective and click in checkbox 2. Left click on an objective (to get the objective overlay), click in checkbox, and close overlay screen	Need screen 4a. Also need the right click contextual menu, and the left click overlay screen.
6	How can you choose students to receive small group instruction?	Check the box by the name of each desired student. Click “Save”.	Need screen 4a
7	If you were engaged in this process, how many different groups would you anticipate wanting to manage?	(just want to gather data on this point)	7a. Need class honeycomb screen with the new plan displayed. No students yet for individual tutoring.
8	You would like to review the performance of [name a student who is not in the group]. How would you do this?	1. click “Hide Plan (Show Student List)” 2. click the name of a student	Need screen 1a. 8a. Need a student honeycomb screen.
9	Pick a skill in which this student needs one-on-one instruction. How would you add this to your plan?	1. Click on a hexagon on the student honeycomb. 2. Check the “Individual Tutoring” checkbox in the center panel. 3. Type in the tutoring text field. 4. Click Save	9a. Need a student-standard overlay on top of a student-honeycomb screen. Need screen 8a.
10	You would like to add a second small group, to focus on a different set of skills. How would you do this?	1. Click “Show Plan” 2. Click “Choose students” 3. Click “Create Multiple groups 4. Mark the checkboxes next to the names of desired students. 5. Choose Group 2 from dropdown menu and click “Go”. 6. Click “Done” 7a. Right click a hex to get contextual menu. Click desired objective, and then click desired group. OR 7b. Click a hex to get overlay. Click desired objective, and then click desired group. Click Save.	10a. Need a student honeycomb screen with the plan displayed. 10b. Need a student honeycomb screen in “Choose Students” mode. 10c. Need a student honeycomb screen in “Choose Students – Multiple Groups” mode. 10d. Need a student honeycomb screen in “Choose Students – Multiple Groups” mode with the new students in Group 2. 10e. Need a student HC screen with standard overlay.