Evaluation is essential for improving capability development initiatives. It strives to uncover the impact on learner capability and the benefits for the APS.
We evaluate the courses we have designed and delivered, consistent with our Evaluation Framework. This Framework allows us to adapt our approach for each of our courses, while ensuring we can compare evaluation data across iterations of a course, across courses, and across the entire suite of courses, as well as gaining valuable insights into programs which are not course based.
Under this framework, we use a range of methodologies and activities to collect evaluation data on five key areas:
Evaluation in practice
We use a scalable approach to evaluation, determined by the requirements of course stakeholders and course investment. Every course we deliver includes some evaluation – from a ‘light touch’ approach for our one-day courses to a more comprehensive approach for our multi-day, multi-module senior executive suite.
Depending on the evaluation needed, we will use questionnaires, interviews, focus groups and case studies (or a combination of these) to collect evaluation data. The data is then analysed to determine the effectiveness of the courses and inform ongoing design and delivery.
We also undertake research, analysis and reporting to drive continuous improvement and inform the future of capability development across the Australian Public Service.
We have started using a more comprehensive evaluation approach for assessing the implementation and impact of participant learning, as well as using some techniques from behavioural economics to improve individual capability. We capture information from participants using questionnaires about their capability gains, and their views of the relevance and value of the program immediately after the program. Our course participants also set an action plan they will implement in the three months following their course. After three months, we follow-up with a short interview to determine the barriers and enablers participants experienced, and any impact they have seen from implementing their learning. This is generating new insights into the participant experience, the factors that enhance or inhibit behaviour change in the workplace, and the implications for course design.
Sharing with others
We are committed to sharing our findings and experience with colleagues in the Australian Public Service, State and Territory public service, educational institutions and others who are interested in our work.
Contact us at Evaluation [at] apsc.gov.au or 02 6202 3881 if you would like to discuss evaluation of capability development.