Go to top of page

Survey methodology

Appendix B

Survey Methodology

In 2010 with the exception of Ministerial and Parliamentary Services, all APS agencies were mandated by government to participate in the 2010 APS Remuneration Survey. Agencies were invited to attend a survey briefing in January 2011. The briefings informed agencies about the project and prepared them for the data collection phase. Detailed instructions and information were distributed to all participating agencies, including those agencies unable to attend the briefing session.

Data Collection

Participating agency staff completed all of the following questionnaires:

  • SES Remuneration Input Spreadsheet
  • Non-SES Remuneration Input Spreadsheet
  • HR Policies and Practices Questionnaire
  • Data Clarification Questionnaire.

To ensure the accuracy of data collected from agencies, Mercer performed several integrity checks prior to including agency data in the database. Checks were conducted for the following:

  • face validity: to ensure the data was consistent and had no obvious errors or omissions
  • range validity: to check that the data fell within realistic ranges
  • visual data checking: for accuracy and consistency, using our detailed knowledge and experience acquired through many years of remuneration analysis in the public sector.

Data Analysis and Validation

Agencies submitted a dataset on spreadsheets provided by Mercer. Each dataset was analysed and validated using the following steps:

  • each record was checked for valid input for each compulsory field
  • each field was checked to ensure the entry corresponded with the instructions
  • outlier analysis was undertaken for Base Salary using Boxplots (Tukey method). Note: records were classified as outliers if they were greater than the upper whisker or less than the lower whisker. These ranges were set at 2.5 times the inter-quartile range either side of the median
  • records that were flagged as outliers or were outside expected ranges in other collection fields or data input errors were returned to the agencies for verification and/or correction
  • outliers that were verified or corrected by agencies were then included in the overall dataset for analysis
  • Mercer calculated the full value of each individual’s remuneration package
  • Mercer submitted a sample of all key data to the statistics department of the University of New South Wales for review of all calculations performed. In total, nearly 2,000 calculations were checked, with 100 per cent of calculations being confirmed as accurate to the precise dollar
  • individual agencies were provided with basic agency-specific data tables for review and sign-off
  • the overall dataset was finalised only after all verifications and corrections were received.

The majority of the statistical analysis undertaken is descriptive in nature (i.e. frequencies, averages, medians, percentiles). Tests of significance between groups were also performed for the analysis of data by various parameters.

All analyses and report formats are based on requirements as specified by APSC. Prior to releasing the final report, Mercer and APSC agreed on all items to be included in the analyses. Mercer presented a summary of initial findings and then draft reports to APSC for review. However, Mercer retains full responsibility for the accuracy and integrity of all data presented in this report.

Please note that where a range is provided for ‘n’ (the number of agencies or individuals providing data for a specific question/item), it reflects that not all respondents provided data for each possible option. This is most likely the result of some participating agencies not having staff at all classifications, resulting in the non-applicability of particular response categories.

Missing Data

The following approach has been used to represent missing, suppressed or ‘zero’ returns:

  • where fewer than three agencies provide a component (i.e. data is suppressed to ensure confidentiality) the symbol ‘--’ is used
  • where no data is provided, a ‘-’ is used
  • where a zero value is returned, the number zero (‘0’) is shown.

To ensure confidentiality of information for small sample sizes, statistics will not be published unless the following criteria for both case numbers and agency numbers are met or exceeded:

  • average and standard deviation – at least three records from three or more agencies
  • Q1, median, Q3 – at least four records from four or more agencies
  • maximum and minimum – more than 10 records from four or more agencies.

Motor Vehicle Calculations

Mercer has utilised two methods to determine vehicle costs. Where agencies have provided full costs, Mercer has used those figures in the analysis. Where agencies were unable to provide full vehicle costs, Mercer has costed the vehicles using the Mercer Car Formula 2010/2011, based on the purchase prices of the motor vehicles and the degree of usage the employee has of the vehicle. Mercer’s 2010/2011 Car Formula is detailed in Appendix C.

The Mercer Car Formula is a market-based methodology for valuing the package cost of a motor vehicle, taking into account purchase price, lease costs, running costs and FBT. The formula applies a number of consistent assumptions to all vehicles to enable data comparisons. Purchase prices were calculated by Mercer using the make, model and year as provided by each agency.

Mercer has costed each vehicle according to the level of availability of the vehicle to the employee to whom it is provided. For example, a car will only be costed to a package at full market value if the employee has full private and unrestricted usage of the car. Where an employee is required to make the car available for use by other employees at certain times, the car has been costed at less than the full market value.