Appendix 3 Evaluation methodology

During 2006–07, the Commission conducted an evaluation of agency approaches to the recruitment and management of employees with disability. It also undertook two case studies as part of a project which examined complex policy problems and behavioural change. One case study looked at the National Landcare Programme while the other examined the National Tobacco Strategy. Information on the methodologies used is provided below.

Evaluation of agency approaches to recruitment and management of employees with disability

The Commission received funding in the 2006–07 Budget to develop ‘practical approaches that can be used by APS agencies to facilitate the recruitment and retention of people with disability’. The project involved an evaluation of strategies being used by agencies to attract, recruit and retain people with disabilities in the APS, to enable the Commission to prepare practical good practice guidance, and to underpin ongoing advice to agencies. The evaluation built on the Management Advisory Committee (MAC) report, Employment of People with Disability in the APS, ¹ which was launched on 30 August 2006.

Six agencies were included in the evaluation: Centrelink, the Department of Families, Community Services and Indigenous Affairs, Defence, the Family Court of Australia, the Royal Australian Mint, and the National Science and Technology Centre (Questacon). These agencies were chosen to represent a range of business focuses, proportions of employees with disability, size, and regional presence.

The evaluation drew on a range of data sources and methodologies, including:

a review of the material collected by MAC in the preparation of its report
analysis of APSED data at the agency and APS levels and State of the Service agency survey data
researching approaches to the employment of people with disability in other Australian public sector agencies and a limited number of comparable overseas organisations
collecting information through interviews with key personnel and the analysis of relevant policy and procedural documents
focus groups of employees with disability, and of managers with experience in the supervision of employees with disability, in the participating agencies
consultations with peak disability bodies and private sector agencies that provide employment support to people with disability.

The material gathered during the evaluation, and the conclusions reached, informed the structure and content of the Commission’s publication, Ability at Work: Tapping the talent of people with disability,² launched in June 2007.

Case studies looking at complex policy problems and behavioural change

Two case studies were undertaken by the Commission in 2006–07 as part of a project examining complex policy problems and behavioural change. One case study looked at the National Landcare Programme (NLP), which is managed by the Department of Agriculture, Fisheries and Forestry. The NLP is part of a suite of government programmes addressing Australia’s very complex natural resource management issues. As part of the solution to these complex problems the NLP focuses on influencing landholders’ and community groups’ behaviour by ‘enabling natural resource managers to identify, develop and implement improved management practices’.

The other case study examined the National Tobacco Strategy which is managed by Health. The National Tobacco Strategy was chosen as a case study because tobacco control is a difficult and enduring problem where the main focus is on achieving behavioural change.

The objectives of the two case studies were to:

identify and comment on the ‘wicked’ problem³ characteristics of the issues that the NLP and the National Tobacco Strategy aim to address
identify the range of measures used in the NLP and the National Tobacco Strategy to influence behaviour
identify any additional insights for the NLP and Australian tobacco control policy from the theories and empirical evidence about behavioural change.

The methodology adopted for the NLP case study was to review the substantial number of existing evaluation reports on the NLP and to conduct a range of interviews with Landcare groups and networks, industry farming groups, personnel from a number of Catchment Management Authorities, and a number of key government employees involved with the NLP and other natural resource management programmes.

The methodology adopted for the National Tobacco Strategy case study was to review a number of publications on tobacco control from Australia and overseas and to conduct a range of interviews. Interviews were held with NGOs, including the Cancer Council, the National Heart Foundation and QUIT organisations, and with academics and a number of State and Australian Government employees.

The case studies informed two discussion papers the Commission published in October 2007 as part of its goal to stimulate debate about future directions for the public sector. One paper discusses how the APS should go about addressing ‘wicked’ problems. The other related paper provides insights into how the APS can encourage sustained behavioural change to assist in addressing a range of policy issues such as tobacco control, natural resource management and obesity.

1 Management Advisory Committee 2006, Employment of People with Disability in the APS, Commonwealth of Australia, Canberra, <http://www.apsc.gov.au/mac>

2 Australian Public Service Commission 2007, Ability at Work: Tapping the talent of people with disability, <http://www.apsc.gov.au>

3 ‘Wicked’ problems are very complex policy challenges such as climate change and Indigenous disadvantage.