Appendix 3: Evaluation methodology

Abbreviations

A list of the abbreviations used in this report is available in the Glossary

During 2005–06, the Commission conducted an evaluation of agency approaches to the attraction and retention of Aboriginal and Torres Strait Islander employees and an evaluation of APS agencies’ remuneration policies as part of its evaluation programme. Information on the evaluation methodologies used is provided below.

Evaluation of agency approaches to attraction and retention of Aboriginal and Torres Strait Islander employees

The evaluation of agency approaches to the attraction and retention of Aboriginal and Torres Strait Islander employees was aimed at assessing existing agency measures in order to identify the strategies that are most effective in recruitment (including the use of identified positions and special employment measures), skills development and career advancement, and workplace support and retention, as well as identifying impediments to the retention of Aboriginal and Torres Strait Islander employees.

Six agencies were included in the evaluation: Centrelink, the Department of Agriculture Fisheries and Fostery (DAFF), the Department of the Environment and Heritage (DEH), the Department of Education Science and Training (DEST), the Department of Employment and Workplace Relations, (DEWR) and the National Archives of Australia (NAA). These agencies were chosen to represent a range of business focuses, proportions of Aboriginal and Torres Strait Islander employees, size, regional presence and classification profiles. The evaluation largely adopted a case study approach. Information was collected through interviews with key personnel, analysis of relevant documents, particularly Indigenous employment strategies and workplace diversity programmes and related policy documentation, and agency data on achievement against relevant performance indicators as well as data from agency staff surveys and exit surveys. It was also obtained through analysis of APSED data at the agency level, of State of the Service employee survey and agency survey data, and via focus groups with both Indigenous and non-Indigenous employees from each agency.

A census survey of all Aboriginal and Torres Strait Islander employees across the APS was also conducted. Topics covered by the survey included views on agency support provided, career intentions, impediments to Aboriginal and Torres Strait Islander employees remaining in the APS, and reasons why this group may consider leaving the APS. Detailed information on the methodology of the census survey is available in the Aboriginal and Torres Strait Islander APS Employees Census Report.¹

Evaluation of APS agencies’ remuneration policies

The objective of the evaluation of APS agencies’ remuneration policies was to examine the content of agencies’ written remuneration policies (explicit policy documents separate from Certified Agreements (CAs)) as they relate to Australian Workplace Agreements (AWAs) for non-SES employees, and to assess the efficacy of their interaction with other strategic corporate policy, for example, workforce planning, performance management, productivity initiatives and certified agreements.

The evaluation focused on the guidelines determining which employees are offered AWAs and how remuneration levels for existing and new employees are determined, where overlapping salary ranges exist between classifications, how these are managed, links with performance pay arrangements, issues relating to productivity initiatives, and the scope for individual tailoring of AWAs. It also focused on agency structures used to monitor the use of AWAs, communication and consultation arrangements included in remuneration policies, if AWAs are used to address regional differences (e.g. cost of living, job complexity), how these arrangements for AWAs broadly compare to those under CAs in the same agency, and an assessment of how effective remuneration policies have been in assisting agencies’ to achieve corporate objectives.

The evaluation drew on data from the 2003 and 2004 State of the Service agency and employee surveys, together with relevant written material provided by agencies. Published data, such as remuneration surveys conducted for the Department of Employment and Workplace Relations and various earnings data published by the Australian Bureau of Statistics were also examined. A literature search was conducted and various publications relating to performance management and/or performance pay arrangements in the APS such as reports prepared for Management Advisory Committee and the Australian National Audit Office audits were examined. Interviews with five agencies, selected on the basis of the content of their written policies and agency characteristics (eg. size, regional operations) were also conducted.

Australian Public Service Commission 2006, Census Report: Aboriginal and Torres Strait Islander APS Employees, pp. 105–111, <http://www.apsc.gov.au>

Next page: Factor analysis