Use the Final Test as the Initial Activity
I use (and train others to use) a dozen important principles for faster-cheaper-better instructional design.
Here’s one these principles: Begin your design by constructing an authentic performance test.
What We Planned for the Final Test
I do not use paper-and-pencil tests as final tests — unless my training deals with some paper-and-pencil activity such as writing a business report. In most situations, my final tests present authentic scenarios and require actual performance.
Here’s an example:
We recently designed a training package for Customer Service Representatives (CSRs) on how to empathize with upset customers during telephone conversations.
This was the plan for our final test:
The test will involve scenario-based roleplays over the telephone. It will present unpredictable telephone calls from upset customers. The participant’s performance will be evaluated through an objective checklist.
To create the scenarios, we worked with experienced CSRs, subject-matter experts, and real-world customers. The final version of the test involved 10 different scenarios that represented different types of internal and external customers complaining about different products and services. We created five different personality profiles for the customers (ranging from the meek and compliant to the abusive and aggressive). By using these five personalities with each of the 10 situations, we ended up with 50 different scenarios.
To assess the competency of each participant, we asked him or her to roleplay in two randomly selected scenarios. To ensure the authenticity of the test we hired improv actors to make the telephone calls. These actors were provided with the scenarios but not with scripts. They were asked to improvise the conversation in response to the test-taker’s questions, statements, and reactions. They were also asked to push the participant to a medium level of difficulty.
What We Actually Did
Because of time and budget constraints, we did not precisely implement the plans for our final performance test. Instead, we adapted aspects of this planned test to design our training faster and better.
We used the planned test as the operational definition of our training objectives. We created several scenarios according to the original plan and incorporated them in different parts of the training session. We created the checklist for rating participants’ performances and used it to define the content and objectives for our training.
The Training Design
Here is the sequence of activities in our training session:
Walk-through. The facilitator walks the participants through the rating checklist and explains appropriate behaviors for demonstrating empathy during a conversation with an upset customer.
Roleplayed demonstration. The facilitator conducts a roleplay (inviting a participant to play the role of an upset customer) to demonstrate appropriate empathic behaviors.
Team practice. The facilitator assigns different scenarios to teams of participants. These teams analyze the scenarios and prepare an outline for a simulated telephone conversation. They enact this conversation as an audio drama. After each enactment, participants from the other teams (and the facilitator) provide feedback about which empathic behaviors were demonstrated effectively and which behaviors could be improved in future conversations.
Individual practice. Individual participants use other scenarios to conduct rapid roleplays to improve their fluency with empathic behaviors.
Final test. As a final activity, each participant is given a different scenario. A randomly selected participant roleplays a CSR while the facilitator roleplays an upset customer (like the improv actor in our original plan).
Our Recommendation
Early in the training design process, construct an authentic performance test. Use components of this test to operationalize the training goal, objectives, and content. Design your training to enable participants to perform effectively in this test – even if you don’t actually use it in the original form.