Home > Building APS capability > Learning and development

LEARNING AND DEVELOPMENT

A key component of the performance management process is identifying the development needs of individual staff and the capability requirements of the agency as a whole. Responding to capability needs through effective strategies for managing learning and development is integral to the process for building organisational capability.²⁷ The ANAO’s 2001—02 performance audit report on the management of learning and development across the Service found that agencies had made significant efforts to ensure that learning and development strategies were properly aligned with the business needs identified in their corporate plans.²⁸ As a direct response to the ANAO’s findings the APS Commission and the ANAO collaborated to produce a better practice guide, Building capability–A framework for managing learning and development in the APS.²⁹ Released in April 2003, the guide encourages and supports agencies in developing a more strategic approach to planning, delivering, and evaluating learning and development to meet organisational goals and deliver best value for money.

The 2003 MAC report Organisational Renewal found that structured learning and development is important to building capability and that strategies require adjustment to ensure changing capability needs are met.³⁰ The 2004 MAC report Connecting Government also examined issues of learning and development in the context of building organisational and wider APS capability.³¹ The report highlighted the need for agencies to support whole of government activities by becoming more responsive to whole of government demands, in particular, ‘through learning opportunities for middle and senior managers in the skills relevant to whole of government activities, including project management, communications, and relationship management’.³² (Whole of government capabilities are discussed in more detail later in this chapter.)

Learning and development in the APS has also been the focus of an inquiry by the Senate Finance and Public Administration References Committee.³³ While the committee noted that there had been many positive developments in learning and development in the APS, aspects of the committee’s report also reinforced many of the findings of the ANAO audit, including:

• the need to ensure learning and development planning and activities are aligned to the agency’s business directions
• the paucity of data on the level and cost of learning and development across many agencies
• the limited evaluation of whether learning and development investment is achieving its purpose.

The committee was critical of the fact that the limited data available on learning and development in the APS hampered its ability to explore trends and assess value for money. The report made a number of recommendations directly related to learning and development in the APS. The Government has yet to respond to the Senate report.

ALIGNING LEARNING AND DEVELOPMENT TO BUSINESS GOALS

The APS Commission and ANAO’s better practice guide states that ‘agency business objectives and capability requirements should drive the subject matter and content of learning options’.³⁴ The agency survey explored how agencies aligned learning and development with organisational business goals.

Consistent with the ANAO’s 2002 findings, agency responses indicated a strong focus on this issue. Figure 9.9 shows that almost all (95%) agencies identified learning and development priorities as part of their performance management system, while 80% discussed priorities in their annual report.³⁵

Figure 9.9: Measures used to align learning and development with business goals

Source: Agency survey

The proportion of agencies using each measure was largely consistent between 2002—03 and 2003—04. However, a higher proportion of agencies in 2003—04 were identifying priorities as part of their performance management system (95% compared to 90% in 2002—03) and in their corporate plan (59% compared with 54% in 2002—03). In contrast, in 2003—04 a smaller proportion of agencies were including priorities in CAs and AWAs (66% in 2003—04 compared to 75% in 2002—03).

Some measures used to align learning and development priorities differed depending on agency size. While at least 90% of small, medium, and large agencies identified priorities as part of the performance management system, large agencies (90%) were more likely than medium (63%) or small (49%) agencies to identify priorities in work group plans. All large agencies discussed priorities in their annual report, compared to 89% of medium and 64% of small agencies. Furthermore, large and medium agencies (76% and 70% respectively) were more likely than small agencies (41%) to include learning and development priorities as part of their corporate plan.

In addition, the majority of large (90%) and medium (85%) agencies used at least four alignment measures, compared to only 41% of small agencies. It appears that large and medium agencies were more likely than small agencies to perceive the need for formal alignment across a broader range of measures.

DIMIA implemented a national learning framework in 2003—04, part of which involved an annual planning cycle to prioritise investment in learning and development according to the department's business priorities. The five national priorities endorsed under this framework were: induction, client contact, quality of decision-making, supervision and management, and contract management.

IDENTIFYING LEARNING AND DEVELOPMENT NEEDS

The agency and employee surveys collected information about how learning and development needs were identified in 2003—04.³⁶

The agency survey results suggest that learning and development needs were most commonly identified at the work-unit level. Figure 9.10 shows that the most common methods used were: individual self-identification, manager identification, and manager and individual identification through the performance management system (all 97%).³⁷ Three-quarters of agencies also indicated that the manager and the individual were responsible for identifying learning and development needs outside of the performance management system.

As indicated in Figure 9.10, the proportion of agencies in 2003—04 using each measure to identify learning and development needs was largely consistent with last year’s results. An encouraging result, however, is that agencies appear to be gradually strengthening the link between their approach to learning and development and broader workforce planning issues. Creating such a link is essential in integrating learning and development with HR and other business processes, and in building longer-term capability.

Figure 9.10: Identification of learning and development needs

Source: Agency survey

The identification of learning and development needs appeared to be related to agency size. While all agencies were most likely to identify learning and development needs at the work unit level, large and medium agencies were more likely than small agencies to use more formal methods. For example, well over half of large and medium agencies (67% and 59% respectively) used analysis of organisational capability, compared to only 33% of small agencies. Large (48%) and medium (41%) agencies were also more likely than small (21%) agencies to use workforce planning in identifying learning and development needs.

The employee survey shows that that the majority (94%) of employees had their learning and development needs identified in the last 12 months. The most commonly reported methods for identifying learning and development needs were self-identification (60%), and identification together with manager as part of the performance management process (51%).³⁸ Only a minority (16%) of employees reported that their manager (in isolation) identified their learning and development needs. Comments were provided by some employees on the identification of their learning and development needs. Negative comments generally related to a lack of funding for learning and development, or that work priorities meant there was not enough time for learning and development. Although there were fewer positive comments, some of these included:

I have just completed my Masters and the agency has been very supportive both in terms of time and financial support.

I feel that the learning and development I have received has been very good. Under the direction of my direct supervisor I feel that I have learnt quite a lot in the short time of my employment.

These results are generally consistent with the findings of the ANAO’s report on performance management in the APS.³⁹ The ANAO found that all responding agencies were identifying learning and development needs as part of the performance management process, and around half (53%) of these agencies were aggregating this information to provide an overall agency perspective of their needs. However, the ANAO found that only 39% of responding employees thought that the performance management system was effective in identifying their learning and development needs.

Overall, there is some disparity between the agency and employee data on this issue, as almost all agencies reported that they were using management identification (in isolation) of needs, but only a small proportion of employees reported that their learning and development needs were identified this way. Some of this disparity may be explained if the agencies’ learning and development priorities are relayed informally by supervisors to employees within the context of performance management discussions. The Commission suspects nonetheless, that there would be stronger alignment between the agencies’ business needs for skills and knowledge and employee learning and development if the needs were more formally articulated and conveyed to staff.

LEARNING AND DEVELOPMENT DATA

The ANAO’s 2002 report found that across the APS, the collection of data on learning and development expenditure and staff development days was patchy. The better practice guide published by the APS Commission and ANAO highlights the need for agencies to collect meaningful data on inputs, as well as outcomes, to ensure that agencies have the necessary information to assess value for money arising from learning and development investments. It recommends a minimum data set for agencies, including:

• the number of days of formal learning and development per person per year
• expenditure on formal learning and development as a percentage of running costs
• expenditure on outsourced providers
• expenditure on learning and development consultants
• qualitative views (by key stakeholders such as the executive, management and others) on changes in organisational and individual capability and performance.⁴⁰

For the second year in a row, the agency survey found that the collection of data on learning and development expenditure against some of the key items in the data set was problematic, with no evidence of improvement. Although the data is not directly comparable, the proportion of agencies that could estimate their investment in off-the-job learning and development in 2003—04 (63%) was almost identical to 2002—03 (64%).⁴¹ In 2003—04 only 51% of agencies could supply information on the approximate number of days of formal off-the-job learning and development per person, while 62% provided figures on overall off-the-job expenditure.

DFAT and OPC were the only two agencies that had developed a process to measure the cost of on-the-job learning and development, with an additional 18 agencies (21%) developing such a process. Of the 21 agencies in 2002—03 that reported they were developing a process to measure the cost of on-the-job learning and development, none had developed, and only six continued to develop, such a process.⁴² These findings suggest that measuring the cost of on-the-job learning and development remains a challenge for almost all agencies.

OPC uses quality, quantity, and price measures against the outputs from its portfolio budget statements to calculate the overall cost of learning and development. OPC has aligned the cost of both on-the-job and off-the-job learning and development with its portfolio budget statements output costings, and developed a detailed method to measure the cost of on-the-job learning and development.

ACCESS TO LEARNING AND DEVELOPMENT

The employee survey results shows that in the last 12 months, half of all employees participated in either 3—5 days (28%) or 1—2 days⁴³ (22%) off-the-job learning and development activity. A further 14% participated in 6—10 days, whilst an additional 14% participated in 11 or more days of learning and development activity. However more than one in five (22%) employees did not participate in these activities at all in the last 12 months. Roughly, this suggests an average of about four days’ off-the-job learning and development a year, or two per cent of employee working days a year.

Ongoing employees (79%) were more likely than non-ongoing employees (68%) to have participated in at least one day off-the-job learning and development activity. This is understandable given that ongoing employees are a long-term investment for agencies.

A lack of access to learning and development was also reflected in some of the employees’ comments, for example:

No specific training budget. Very limited access to anything other than in-house training.

Difficult to do any due to resource constraints and workload pressures.

The organisation is big on identifying needs and requirements, but consistently fails to deliver quality training/development/assistance for various reasons.

The agency survey also showed that of the 44 agencies that could supply information on the number of days of off-the-job learning and development provided to employees, most were likely to report that they provided either 3—5 days (55%) or 1—2 days (27%). These results are similar to those reported last year. Results of the employee and agency surveys are not directly comparable (as agency results are not weighted by employee numbers).

DELIVERY OF PROGRAMS

One of the key principles identified in the APS Commission and ANAO’s better practice guide is that agencies provide appropriate learning and development delivery options. This includes systematically planning learning interventions appropriate to the characteristics of the organisation.⁴⁴ The agency survey asked about the most commonly used learning and development delivery options adopted for different classification levels.

For most classification levels, external conferences, courses or seminars (other than APS Commission programs) were most commonly used by agencies. More than 70% of agencies nominated this as one of their most commonly used activities at all levels. The proportion of agencies using this activity increased with classification level: 72% of agencies used this activity for APS 1—4 compared to 90% of agencies for SES employees. Other commonly used activities at all levels included: in-house programs delivered by non-APS Commission external providers (between 49% and 83% of agencies) and APS Commission programs (between 49% and 71% of agencies).⁴⁵

In-house programs delivered by non-APS Commission external providers were most likely to be used at the APS 1—4 (83%) and APS 5—6 (80%) levels, while APS Commission programs were most likely to be used at the SES (71%) and EL (67%) levels. There was also considerable variation in the use of other activities (including agency organised guest speakers, coaching, formal networks, and university) between classifications, which suggests that agencies are taking into account the needs of different groups in determining appropriate learning and development options.

EVALUATION OF LEARNING AND DEVELOPMENT STRATEGIES

The 2002 ANAO report found that, in general, agencies were not evaluating learning and development strategies. Consequently, they were not in a position to determine how successful learning and development outcomes had been in assisting them achieve their business outcomes. Performance indicators were generally used as measures of activity rather than effectiveness. Responding to this finding, the better practice guide recommended that agencies maintain a minimum data set covering outputs and outcomes of learning and development programs, as well as inputs.⁴⁶

The agency survey showed that in 2003—04 there was still a substantial focus on evaluating the content of programs and the effectiveness of their delivery, with the amount of evaluation activity in this area increasing.

Evaluating the benefits of learning and development to the individual and agency performance remained a challenge for agencies, although an increasing proportion of agencies were evaluating value for money in 2003—04 (59%) compared to 2002—03 (44%). Table 9.7 shows other areas in which evaluation had significantly increased.

Table 9.7: Aspects of learning and development evaluated by agencies

Source: Agency survey

Of the agencies that reported last year that they were developing measures to evaluate the benefits of programs to individuals and the organisation, few were actually evaluating these benefits in 2003—04, and a substantial proportion were no longer attempting to develop these evaluation processes.

To provide further support in the area of evaluating learning and development strategies, the APS Commission is developing, in consultation with agencies, a learning and development evaluation guide. The guide is intended to assist APS learning and development practitioners to effectively evaluate learning and development strategies in ways that are appropriate to their agencies’ culture and business requirements. It will have two sections:

• a printed 'framework' that outlines how to get started, and key decisions to make in the planning stage for learning and development evaluations
• a set of web-based resources for practitioners to access and modify to suit their circumstances. These resources consist of tools, templates, tips, checklists, outlines of approaches for each of the six elements in the evaluating learning model from the better practice guide.

It is expected that this guide will be released in early 2005.

The development and release of the learning and development evaluation guide by the APS Commission should assist agencies in developing evaluation programs in these areas. It is also important to note that some agencies are having success in developing evaluation strategies, for example, DAFF.

27 Learning and development as discussed in this section refers to learning activities on-the-job as well as more formal off-the-job activities. Seminars, conferences, classroom training courses, leadership programs, academic study, in-house programs are all included.

28 ANAO, Management of Learning and Development in the Australian Public Service, Performance Audit Report 64, June 2002, http://www.anao.gov.au

29 APS Commission and ANAO, Building capability–A framework for managing learning and development in the APS, April 2003, http://www.apsc.gov.au

30 MAC, 2003, op.cit.

31 MAC, 2004, op.cit.

32 ibid., p. 5.

33 Senate Finance and Public Administration References Committee, Recruitment and Training in the Australian Public Service, tabled 18 September 2003.

34 APS Commission and ANAO, op. cit., p. 16.

35 Agencies were asked to indicate all measures used to align learning and development with the business goals of the organisation.

36 Data in the agency and employee surveys is not directly comparable, as the wording of the question varied slightly between the surveys.

37 Agencies were asked to report all methods used to identify learning and development needs in their agency.

38 Employees were asked to report all methods used to identify their learning and development needs.

39 ANAO, 2004, op. cit.

40 APS Commission and ANAO, op. cit., p. 22.

41 In 2002—03, the qualifier ‘off-the-job’ was not included in the relevant question– ‘Can your agency estimate its investment in learning and development in 2002—03?’

42 For comparative purposes, ATSIC responses have been omitted from the 2002—03 data.

43 Includes part day.

44 ANAO and APS Commission, op. cit., p. 16.

45 For APS Commission programs the 2004 agency survey did not differentiate between ‘in-house’ and ‘external’ delivery.

46 ANAO, 2002, op. cit., p. 24.