Go to top of page

Appendix 1c: Learning and performance

Reaction

Participant reaction to learning and potential for application

Download the participant reaction to learning template

Participant reactions

Download the participant reactions template

Sample participant reaction evaluation questions for a complex program

Download the sample evaluation questions

Capability

Techniques to evaluate learning and performance

The following highlights a variety of techniques which can provide evidence of learning and improved capability or competence.

Generic techniques

generally used after the learning:

  • Surveys
  • Interviews
  • Structured observations
  • Work samples
  • Document reviews

Needs based techniques

generally used during and after the learning:

  • Repeat needs assessment/feedback
  • Critical incidents
  • Assessment centres
  • Focus group process
  • Individual impact maps
  • Individual learning agreements
  • Performance reviews

Performance techniques

generally used during the learning:

  • Simulations and role plays
  • Workplace trials
  • Apprenticeships

Assessment techniques

generally used during and after the learning:

  • Tests and assessments
  • Instructor ratings and reports
  • Supervisor ratings and reports

The following suggests how instruments can be matched with domains of learning so that learning is evaluated in a valid manner. For these domains consider these instruments.

Cognitive

  • Tests/assignments
  • Learning agreements and personal impact maps
  • Work samples

Affective

  • Role plays
  • Structured observations
  • Surveys of supervisors and direct reports/clients
  • Focus groups
  • 360/180 degree feedback

Psychomotor

  • Simulations
  • Apprenticeships
  • Workplace trials and assessments
  • Instructor ratings
  • Tests

Accountable learning agreement

Download the accountable learning agreement.

Planning guide for evaluating learning and development

This tool is designed to provide evaluators with an overview for evaluating performance. It does not seek to be comprehensive, but focuses instead on strategic action for cost-effective and useful evaluation.

Evaluation element

Performance

Objective

Assess how learners perform on the job following learning interactions

Data collection method

  • Refer to evidence gathering techniques and measures of learning highlighted in the Techniques table.

Possible data sources

  • Test results
  • Reaction sheets
  • Performance review data;
  • 360/180 feedback data;
  • Learning agreements and plans;
  • Insight from key stakeholders
  • Workplace assessors
  • Facilitators
  • Participants
  • Supervisors
  • Team members and reports
  • Clients and customers

Analytical tools

  • Application of learning
  • Questionnaire
  • Application of learning
  • surveys for participants and supervisors

Reporting tool

  • The learning and performance scorecard

Timing

  • Conducted after learners have the opportunity to apply learning.

Stakeholders and their roles

Evaluator
  • Oversight and conduct of the evaluation
  • Communicate findings to the broader organisation
Key programme sponsors
  • Provide support to the evaluation
Key line managers in areas where learning is to be applied
  • Participate in the evaluation and analysis and use findings to improve workplace support
Relevant management and other reference committees
  • Provide input, guidance and approvals
Learning designer
  • Incorporate findings into the design

Individual learning impact map

Download the impact map.

Sample application of learning survey

Download the sample learning survey

Capability and performance

Structural observation log

Download the observation log.

Reporting on learning and performance evaluation

Download the report on performance and evaluation learning.

Tips when evaluating learning and performance

1. Don’t just assess learning, measure factors that influence learning as well

  • Finding out what encourages and hinders learning and its transference is as important as measuring what learning actually occurred.
  • Use data from participant reaction evaluations to identify and analyse potential barriers and enhancers to learning.
  • When assessing actual learning, use “before and after” techniques. These can include: written, oral and workplace assessments, 360 interviews and surveys, structured observations and work samples etc.
  • Allow time for behaviour change to take place. This may take months.
  • Use a generalisable sample when large groups need to be evaluated
  • Use control groups where practical

2. Learning and development facilitates capability, the workplace facilitates actual performance

  • Help the organisation recognise that learning and development has only limited direct influence on an individual’s ability to perform in the workplace. The workplace environment, its culture and the support that managers and supervisors provide are critical factors that also shape actual performance on the job.
  • Follow up with learners and their supervisors to evaluate if learning has been successfully applied and the visible difference this has made on their work, and their teams.
  • When evaluating performance, factors which have a positive, negative or indifferent impact on performance need to be identified, analysed and understood.
  • Use the information you obtain form performance evaluations to help managers and supervisors better understand how to enhance and support their people’s performance.
  • When resources are limited, focus on learners who are successful in applying their learning and who seem to perform well. Find out why and share this information with others. Build from success.

References and resources—evaluating learning and performance

Caffarella, RS 1994 Planning programs for adult learners, Jossey Bass, San Francisco.
Presents useful information, ideas and tools on transfer of learning, evaluation of learning and reporting evaluation data.

Gilley, JW & Maycunich, A 2000, Beyond the learning organisation, Perseus Books, Cambridge, MASS.
Provides a useful albeit brief section on evaluating learning and the importance of supporting transference and application. Also discusses the importance of individual accountability in successful learning and application.

Knowles, M 1986, Using Learning Contracts, Jossey Bass, San Francisco.
A classic text which provides a wealth of information on learning agreements and how these can be used to evaluate learning across a wide spectrum of capabilities.

Laird, D 1985, Approaches to Training and Development, Addison Wesley, Reading, MASS.
A seminal text of the field with a useful section of measuring learning in cognitive, affective and psychomotor domains.

Mayne, J 2004, ‘Reporting on outcomes: setting performance expectations and telling performance stories’, The Canadian Journal of Program Evaluation, vol. 19, no. 1.
An innovative approach to evaluating performance through the use of a structured performance story or narrative that weaves qualitative and quantitative data to paint a holistic picture of performance.

Machin, A 2002, ‘Planning, Managing and Optimizing Transfer of Training’, in Kraiger, K. ed, Creating, Implementing and Managing Effective Training and Development, Jossey Bass, San Francisco.

Parry, SB 2000, Training for Result, ASTD, Alexandria, MASS.
Useful information on how the use of appropriate learning plans can influence the transfer of learning and how these plans can be used for evaluations.