Measuring employee engagement in the Australian Public Service
Last updated: 11 Feb 2013
This page is: current
Cotton A.J. 2012, Measuring Employee Engagement in the Australian Public Service, Australian Public Service Commission Staff Research Insights, Canberra.
An overview of employee engagement
The concept of employee engagement is widely used in the human resources literature.2 While the exact origin of the concept is difficult to determine, it is often considered to have evolved from the concepts of job satisfaction, organisational commitment, and organisational citizenship behaviour. Multiple definitions of employee engagement have emerged in the literature which presents particular challenges for both managers and human resource (HR) practitioners:
For an HR executive, getting a handle on employee engagement can be like trying to catch a greased pig at a country fair. Just when you think you've got it, it slides right out of your hands. 3
One of the contributors to this confusion is the fact that the concept of engagement developed initially from organisational practice rather than from academic research and many of the authors in the field lack a degree of independence in their writing because they are in the business of delivering employee engagement solutions to organisations.4 As a result, there has tended to be limited collaboration or information sharing on the concept as many of the authors are, in fact, competitors genuinely seeking to protect their intellectual property.
While this pattern of development from practice to academia may not have been conducive to developing a unified understanding of the concept of engagement, it is not unique. The concept of burnout, for example, was well developed in the practitioner literature before being embraced by academia. Similarly the initial writing on emotional intelligence was almost entirely in the practitioner sphere.
Despite this lack of cohesion in the literature there are some general conclusions that can be drawn about the concept of employee engagement:
- It is a much broader concept than either that of ‘commitment’ or ‘motivation’ that one finds in the academic or practitioner literature.
- The considerable amount of research and analysis that has been conducted on engagement in the last 10 years suggests that it is more than a passing management fad.
- The quality of the thinking and measurement methods has improved significantly in the last 10 years.
Also, there are a number of features of employee engagement have been consistently identified in the literature5:
- Employee engagement is generally seen as a two-way interaction between the employee and the organisation.
- There are many characteristics of an ‘engaged’ employee including: commitment, pride and advocacy of the organisation, a connection with the organisation’s strategy, and a stated preparedness for discretionary effort in the workplace.
- While the literature has identified many ‘drivers’ of engagement, differences exist across different organisations and even among different elements of the same organisation and, as a result, there is no ‘template’ solution for engagement.
- Although there have been considerable claims made about the positive benefits of increasing employee engagement on tangible organisational outcomes, these tend to be based on correlation or co-incident results rather than any research demonstrating causality.
Approaches to measuring employee engagement
There are many “models” of employee engagement in the literature and they tend to fall into two broad categories. The first is often found in the practitioner literature and conceptualises engagement as a single, omnibus construct that can be measured as such. These models often are associated with a particular provider, commonly use self-report attitudes such as loyalty and advocacy as indicators of engagement and often cite the pursuit of engaged employees as an end in itself. Advocates of these models will often refer to a global relationship between the number of engaged employees and organisational performance but without any examination of the elements or nature of this relationship. One of the most widely known examples of this type is the Gallup Q12.6
The second type of employee engagement model is more common in the academic literature and argues that engagement is a multi-dimensional concept and more comprehensively examines the links between the antecedents7 of engagement and the consequences8 of engagement. A good example of this type of model is provided by Saks.9
Much of the practitioner literature on employee engagement takes a highly data-driven approach with often-limited formal theoretical consideration of nature of the link between employee engagement, employee behaviour, and subsequent work performance. The Gallup Q12 model is a good example of the use of rigorous statistical methods on a vast amount of data to identify a set of survey questions that:10
... link powerfully to key business outcomes. These 12 statements emerged as those that best predict employee and workgroup performance.
And while the empirical rigour behind such models is not in question, this data-driven approach can lead, at times, to a lack of understanding, or even critical examination, of the theoretical foundations underpinning the concept of employee engagement. As one author concludes: ‘...the relationships among potential antecedents and consequences of engagement as well as the components of engagement have not been rigorously conceptualized, much less studied.’11 And without any understanding of the relationship between the antecedents and consequences of employee engagement it is impossible to understand how this relationship might influence organisational outcomes.
Many published models fail even to attempt to identify a link between engagement and actual organisational outcomes but focus instead on self-report attitudes or opinions as the “outcome” of engagement. One very well regarded model, in a manner that is not uncommon, cites being engaged, being positive about customer focus, and being positive about organisational values as valid organisational outcomes of employee engagement.12 All of which are important attitudes for employees to possess but the model fails to provide any clear link between these attitudes and actual behaviours that result in organisational outcomes. Moreover, while many authors claim evidence that organisations with high engagement have better organisational performance than organisations with lower engagement scores, they offer no explanation of why this might be occurring.
The evidence used to support many of the assertions made about employee engagement in the practitioner literature in particular, does not allow examination of the nature of the relationship between engagement and its outcomes. Yet many authors speak about engagement with impressive hyperbole, for example ‘…engagement remains the ultimate prize for employers…’13
Engaged and disengaged employees
Another element common among many models of employee engagement is that they segment the workforce into ‘engaged’ and ‘disengaged’ staff. This is consistent with the idea that engagement is a unidimensional concept and that there is some absolute benchmark against which one can identify an engaged, or disengaged, employee. Many authors speak about ‘disengaged’ staff in highly emotive language:
…disengaged employees erode an organization's bottom line while breaking the spirits of colleagues in the process.14
The combination of such extravagant claims in the absence of an agreed definition of employee engagement, or a more complete understanding of the nature of engagement with clear links to organisational outcomes, has contributed to a degree of confusion and a subsequent loss of confidence by managers in the concept of engagement. It has been noted that:
…senior HR leaders and their colleagues in the executive suite have grown increasingly frustrated, questioning the concept of employee engagement itself, as well as the ultimate return from investments made to drive the engagement of their workforces. 15
The failure to establish a common definition of the concept of engagement with a clear link between the antecedents and consequences of engagement is a deficiency of many of the extant models of employee engagement. Without a clear understanding of the nature of employee engagement it is impossible to measure which:
... is a matter of particular significance to those who develop and conduct employee surveys in organizations because the end users of these products expect interpretations of the results to be cast in terms of actionable implications. Yet, if one does not know what one is measuring, the action implications will be, at best, vague and, at worst, a leap of faith. 16
And it is this point that is particularly germane when considering the concept of employee engagement in the APS.
The APS context for employee engagement
Any consideration of employee engagement in the APS must recognise the organisations business context. The APS is a complex organisation with over 160,000 employees, working in more than 100 organisations that range in size from less than 10 to almost 40,000 employees. The nature of work done in agencies varies substantially though can be classified into four broad functional types: policy, operations, specialist, and regulatory each of which has different workplace characteristics.
As well as this diversity, there are a number of enduring features of the public sector that are also common across the APS and that serve to separate it from the private sector. The concept of public service motivation17 has an entire academic literature devoted to it for example. Understanding that there are difference between the public and private sectors is important because most of the engagement research is conducted in the private sector, and has led some to conclude that there is little difference in employee engagement across the two sectors18, which seems presumptuous given the lack of engagement research done in the public sector.
Employee engagement as an element of human capital
One key driver in rethinking employee engagement in the APS has been the adoption of the concept of human capital as a fundamental way of thinking about APS workforce capability. Ahead of the Game: Blueprint for the Reform of Australian Government Administration19 (the APS Reform Blueprint) quite deliberately introduced the term human capital to drive a shift in APS thinking to a more strategic and forward looking approach to workforce capability.
A fundamental principle of a human capital approach is that the workforce is an important contributor to organisational capability and, as a result, employee engagement in the APS is meaningful only if it can be related to the workforce’s contribution to organisational outputs and outcomes. This is echoed in some of the more recent engagement literature; for example, Ellis and Sorenson20 state that employee engagement should ‘...always be defined and assessed in the context of productivity’. Key to this is whether it is possible to develop an explainable and measureable model of employee engagement in terms of the relationship between its antecedents and consequences in the APS context.
Understanding engagement in the APS
The starting point for developing an explainable and measureable model of engagement in the APS is the generally held view that engagement is a two-way, reciprocal, relationship between the employee and the workplace.
As a concept, employee engagement represents an evolution in thinking about people and work from concepts such as job satisfaction. Omnibus measures of job satisfaction have been shown to be substantially influenced by the personality of the employee so that “...organizational efforts to improve long-term job satisfaction in employees are probably futile.” 21 Moreover, the relationship with job performance has been shown to be so low that efforts to improve performance through job satisfaction may in fact be counter-productive.22 And, while the relationship between job satisfaction and organisational withdrawal behaviours such as turnover is stronger, there are many confounding variables in these relationships.
One key feature of engagement as a concept is the idea that there is a reciprocal nature to the relationship between the employee and their work. The relationship is based on an exchange between the employee and their work; the employee gains from their participation in work (remuneration, prestige, social connectedness, etc.) and reciprocates by understanding and meeting workplace needs (effort, timeliness, etc.), remaining in the workplace, and applying effort to their work. The reciprocal nature of the relationship between the employee and their work is fundamental to employee engagement.
As in any human relationship, trust is a key part of this relationship. An employee will form a set of expectations about their workplace, when these are met, their trust is justified and this will have a positive impact on the employee. But when expectations are not met, or there has been a breach of trust, this can have substantial negative consequences for the employee23. Trust in the workplace is a prominent feature in the literature related to the psychological contract24 and workplace bullying25; and it is a key element of the employee engagement relationship.
A final and important consideration in understanding engagement in the APS it the multi-dimensional nature of “work” in the modern workplace; the concept of “work” has evolved substantially from a means of obtaining economic subsistence26 to something much more complex. In the modern workplace employees are likely to engage with many different aspects of the workplace and therefore employee engagement is almost certainly a multi-dimensional concept. A number of authors, particularly in the academic literature have theorised on the multi-dimensional nature of engagement.27
A model of APS employee engagement
Given this thinking, a number of parameters were identified that needed to be satisfied before a model of engagement could be considered suitable for the APS; specifically, the model needed to:
- provide clear links between the antecedents (or drivers) of engagement and the organisational outcomes of engagement; and
- reflect the multi-dimensional nature of the modern APS workplace.
An extensive examination of both the practitioner and academic literature identified that the Corporate Leadership Council (CLC) employee engagement model28 satisfied both of these parameters.
The CLC model of employee engagement
The CLC model is underpinned by a definition of ‘purposeful’ engagement that clearly links engagement with organisational outcomes:
The Council has defined engagement as the extent to which employees commit—both rationally and emotionally—to something or someone in their organization, how hard they work, and how long they stay as a result of that commitment. By using this outcomes-focused definition, we can measure the tangible benefits of engagement, as opposed to focusing on ‘engagement for engagement’s sake.
The model is also multi-dimensional being based on the idea that staff commit, in varying levels, to the following elements of their workplace: their job, their team, their immediate supervisor, and their organisation.
As well as having a basic conceptual framework, the CLC Engagement Model has a strong empirical basis having been developed on a large sample of employees surveyed across a number of private and public organisations.29 It integrates key concepts from the organisational commitment literature,30 in particular the idea that commitment to the workplace can have both a rational and emotional component31 and it provides a clear link between organisational outcomes and engagement including improved employee performance and retention.
The CLC claim three key findings from their research:
- Successful engagement strategies begin with an outcome focussed definition of engagement
- The majority of employees are neither fully engaged nor fully disengaged
- Employee engagement has a significant impact on performance and retention
The outcomes-focussed definition means that the CLC model allows for the examination of the nature of the relationship between engagement and its consequences rather than simply saying that a relationship exists. The finding regarding the number of “engaged” or “disengaged” employees is consistent with a view that there is no absolute point beyond which an employee is (dis)engaged but rather that employee engagement should be considered in more relative terms. Finally, the link between employee engagement and important organisational capability inputs such as performance and retention is entirely consistent with the APS context and a necessary requirement for any model of APS employee engagement.
The model also allows a more sophisticated examination of the nature of the relationship between the employee and their work. It includes elements of engagement identified in other research, (e.g., job and organisational engagement32) as well as adding engagement with two important elements of the workplace: one’s team and immediate supervisor. Because the model hypothesises a relationship between each individual element of engagement and specific organisational performance outcomes (in this case in terms of employee performance and retention) it allows one to consider the possibility that there is a compensatory characteristic to engagement where, for example, high levels of job engagement compensate for lower levels of organisational engagement with a result that two employees both produce similar work performance.
If this is the case, organisations could develop tailored engagement strategies depending on the nature of the engagement in their workplace, or even within a specific workgroup. Also, it might be possible to develop different engagement strategies for different segments of the workforce, for example, senior employees often have different attitudes towards work than more junior employees. The multi-dimensional and (possibly) compensatory nature of engagement in the model allows for a more sophisticated and more useful understanding of the workforce than does the more typical unidimensional view of engagement.
Overall, the CLC model provides a sound conceptual start point for developing a model of employee engagement for the APS; it meets the parameters defined for consideration and it allows for a more sophisticated examination of the nature of the relationship between the antecedents and consequences of employee engagement. However, given the unique aspects of the APS business context it is necessary to test the theoretical construct of the model in the APS context and, where necessary, adapt it to suit the APS.
Developing the APS Employee Engagement Model
The APS has routinely collected a wide range of information from its employees through the State of the Service Employee Survey (the Survey). The Survey is administered annually to a sample of APS employees and includes a broad range of items ranging from specific aspects of the workplace to more broad employee attitudes and opinions.
Given the breadth of the Survey, it was determined that a useful initial approach would be to identify what aspects of employee engagement were already being measured in the Survey and whether a model of employee engagement for the APS based on the CLC model could be developed from the existing APS data.
Examination of the contents of the Survey suggested that there were a number of item sets that mapped closely to the elements of engagement in the model.33 These were:
- Job - Q16 Please rate your level of agreement with the following statements regarding your current job.
- Job – Q26 Please indicate your level of satisfaction with EACH of the following workplace attributes in your current job
- Team – Q18 Please rate your level of agreement with the following statements regarding your immediate work group
- Supervisor – Q19 Please rate your level of agreement with the following statements regarding your supervisor
- Organisation – Q20 Please rate your level of agreement with the following statements regarding the senior leaders (i.e. the Senior Executive Service [SES]) in your agency
- Organisation – Q21 Please rate your level of agreement with the following statements regarding aspects of your agency’s working environment
Data from the 2010 Survey was used for the analyses. The Survey was administered to a stratified random sample of 9,083 staff of which 5,607 responded. This data set was split into an Experimental sample (60% of the sample or 3,427 cases) on which the exploratory analyses were conducted and a validation sample (the other 40%, or 2,180 cases) on which the final models were validated.
Using the Experimental sample, a classical test theory approach was used to determine the structure of the existing item sets; principal axis factor analysis (PFA)34 was used to determine the dimensionality of each item set. Factor extraction was determined using a combination of scree plot and parallel analysis35 and the resulting factor solutions were rotated to simple structure by oblique rotation (oblimin). Items loading greater than 0.4 on each factor were included for consideration in the factor make up and final scales were refined using reliability analysis to make them as parsimonious as possible while retaining the psychometric properties of the original scales.
The final scales36 and overall model were validated using confirmatory factor analysis (CFA; using the confa procedure in Stata/IC 10.1)37 on the validation data set. Goodness of fit of the final models was assessed using the following fit indices:
- The Comparative Fit Index (CFI) – acceptable model fit is generally indicated by a CFI value of 0.90 or greater.38
- The Root Mean Square Error of Approximation (RMSEA) - acceptable model fit can be indicated by an RMSEA value of up to 0.10.39
- The Root Mean Square Residual (RMSR) – acceptable model fit is typically indicated by an RMSR less than 0.0840 although some authors suggest that a value less than 0.05 is preferable.42
For each final scale a “marker” item was identified; this was the item with the highest item-total correlation and these were used to develop a “short form” of the item set that could be used to approximate the full model more efficiently.
Results: First Order Analyses
Item 16: Your current job
This is one of the larger item sets in the Survey and had 24 items. Factor analysis indicated a four-factor solution, however, after rotation only one item loaded on factor four at the cut-off of 0.4. The data was re-analysed as a three-factor solution and rotated to simple structure using an oblique rotation. Confirmatory factor analysis showed a good fit to the three-factor structure (CFI = 0.97, RMSEA = 0.06, RMSR = 0.05).
The three factors identified from this analysis were labelled: Job Identification, Job Recognition, and Discretionary Effort.
This factor reflects the level of identification an employee experiences with their job; the initial factor had nine items loading greater than 0.4 with very high internal consistency (alpha = 0.91) indicating redundancy in the item set. Refinement of the scale yielded a four-item scale with very good internal consistency (alpha = 0.85), and a very good correlation with the original set of nine items (r = 0.96). The items that comprise the scale42 are:
- My job allows me to utilise my skills, knowledge and abilities
- My work has become a fundamental part of who I am.
- Overall, I am satisfied with my job
- My job gives me a feeling of personal accomplishment*
The second factor extracted reflects the broad level of recognition that an employee receives for their performance. It initially had six items load on it that showed high internal consistency (alpha = 0.81) and substantial item redundancy. Refinement of this scale yielded a three-item scale with improved internal consistency (alpha = 0.86) and good correlation with the original item set (r = 0.92):
- I receive adequate recognition for my work contributions and accomplishments
- I receive adequate feedback on my performance to enable me to deliver required results
- I am satisfied with the recognition I receive for doing a good job*
The next factor reflected the self-reported discretionary effort that an employee might exhibit. There were only three items that loaded onto this scale at greater than 0.4 and three items is considered the minimum number of items needed to create an adequate scale for measurement purposes. This scale had unsatisfactory internal consistency (alpha = 0.61) and will not be considered further. An example of an item loading on this scale:
- When needed, I am willing to put in the extra effort to get a job done.
Item 18: Your immediate workgroup
The original scale was shorter than the previous with nine items and reflected the workplace behaviours of other members of their immediate team. Factor analysis confirmed that the item set was unidimensional and reliability analysis showed very good internal consistency (alpha = 0.88) and substantial item redundancy. Scale refinement yielded a four-item scale with very good internal consistency (alpha = 0.86), and a strong correlation with the original set of items (r = 0.92) and a marker variable with a very good correlation with the scale (r = 0.88). The questions in the final scale were:
- The people in my work group cooperate to get the job done
- The people in my work group share job knowledge with each other
- The people in my work group are honest, open and transparent in their dealings*
- The people in my work group treat each other with respect
Confirmatory factor analysis showed reasonable fit to a single factor (CFI = 0.99, RMSEA = 0.11, RMSR = 0.02).
Item 19: Your immediate supervisor
Supervisor behaviour is represented by another shorter scale that had only ten items that describes the behaviour of the employee’s immediate supervisor. Factor analysis confirmed this set of items was unidimensional. The internal consistency of the scale was extremely high (alpha = 0.93) suggesting redundancy in the items. Scale refinement yielded a four-item scale with very good internal consistency (alpha = 0.88), a very good correlation with the original set of ten items (r = 0.95) and a strong marker variable (r = 0.87). The questions in the final scale were:
- My supervisor demonstrates honesty and integrity.
- My supervisor shows concern for the welfare of his/her staff
- My supervisor encourages and manages innovation
- My supervisor provides effective feedback*
Confirmatory factor analysis conducted on the validation sample showed reasonable fit to the model (CFI = 0.98, RMSEA = 0.12, RMSR = 0.04).
Item 20: SES
Agency leadership was a shorter scale with only six items and was also clearly unidimensional with consistently high factor loadings (0.75-0.85), it reflects the quality and behaviour of the Agency’s senior leaders. The internal consistency was very high (0.92) and indicative of substantial item redundancy. Scale refinement yielded a three-item scale with very good internal consistency (alpha = 0.85), very good correlation with the original item set (r = 0.97) and a strong marker variable that correlated well with the overall scale (r = 0.89):
- In my agency, the leadership is of a high quality
- In my agency, communication between senior leaders and other employees is effective*
- In my agency, the most senior leaders are sufficiently visible (e.g. can be seen in action.
CFA showed reasonable fit to the model but due to the small number of items the traditional goodness of fit tests could not be conducted. However, when the internal consistency analysis was replicated on the validation sample it produced very similar results (alpha = 0.84, marker variable correlation with total r = 0.89).
Item 21: Agency working environment
The Agency Working Environment was a larger item set with 21 items. Factor analysis indicated three stable factors that were extracted from the data and rotated to simple structure. Confirmatory factor analysis showed some support for this structure (CFI = 0.92, RMSEA = 0.09, RMSR = 0.05). The three factors in this data set were: development, agency behaviour, and agency identification.
The first factor extracted related to the developmental opportunities provided in the workplace. It had five items loading on it and good internal consistency (alpha = 0.82), refinement of this scale saw one item removed which caused a minimal drop in internal consistency (alpha = 0.81) but retained a very high correlation with the original scale (r = 0.98) and included a strong marker variable (r = 0.85):
- My agency places a high priority on the learning and development of employee.
- My workplace provides realistic performance expectations
- My workplace provides increased knowledge and/or experience in the job
- My workplace provides access to effective learning and development*
The second factor reflected employee’s perception of the organisational behaviour observed in the agency. It initially had five items load on it greater than 0.4, respectable internal consistency (alpha = 0.79) and some item redundancy. Scale refinement removed one item which reduced the internal consistency a little but it still remains respectable (alpha = 0.76), it has a very good correlation with the original item set (r = 0.97) and a reasonable marker item (r = 0.79):
- In general, employees in my agency effectively manage conflicts of interest
- In general, employees in my agency appropriately assess risk
- Employees in my agency feel they are valued for their contribution*
- My agency deals with underperformance effectively
The final factor reflects the level of identification that the employee experiences with their organisation. It had three items load on it greater than 0.4. The scale had respectable internal consistency (alpha = 0.73) and as it only has three items no further refinement was done with this scale. It had a good marker variable (r = 0.84):
- Working at my agency is important to the way that I think of myself as a person
- When someone praises the accomplishments of my agency, it feels like a personal compliment to me*
- When I talk about my agency, I usually say “we” rather than “they”
Item 26: Job satisfaction
Job satisfaction was measured by a longer set of items that covered a wide range of factors that might contribute to satisfaction with one’s job. Factor analysis of this scale yielded a stable simple structure of two factors: workplace challenges and workplace conditions. Confirmatory factor analyses on these data showed reasonable fit for this model of the data (CFI = 0.96, RMSEA = 0.08, RMSR = 0.05)
Workplace challenges factor related to the challenges the workplace provided to the employee in a positive sense. It had six items load on it, but subsequent scale refinement produced a set of four items that had good reliability (alpha = 0.86) with a strong marker variable (r = 0.87).
- Chance to be innovative
- Opportunities to utilise my skills*
- Opportunities to develop my skills
- Interesting work provided.
The second factor reflected the conditions an employee experienced in their workplace. It showed reasonable internal consistency (alpha = 0.772) but inadequate fit. However refinement of the scale removed one item causing a minor decrease in internal consistency (alpha = 0.759) but improved fit (CFA = 0.99, RMSEA = 0.06, RMSR = 0.02). This scale has only a moderate marker variable (r = 0.66).
- Regular feedback/recognition for effort
- Good working relationships
- Flexible working arrangements
- Good manager*
Results: Higher Order Analyses
Having identified a possible set of factors or constructs that might contribute to a model of engagement from the existing survey items, the next step in testing how the model fit the APS data was to identify whether there was a structure in the data that was consistent with that in the model. To determine this, a confirmatory factor analysis of the hypothesised four-factor structure comprising the ten measures described above was conducted using the Validation data set.
This yielded an acceptable fit of the model to the data (CFI = 0.95, RMSEA = 0.09, RMSR = 0.04) and supported the conceptualised four-factor APS Employee Engagement Model with the following structure (see also Figure 1):
- Job Engagement: Job Identification and Workplace Challenge;
- Team Engagement: Team Behaviour and Recognition;
- Supervisor Engagement: Supervisor Behaviour and Workplace Conditions; and
- Agency Engagement: Agency Leadership, Development, Agency Behaviour, and Agency Identification.
Figure 1: The APS Employee Engagement Model
While the confirmatory factor analysis supports the existence of four separate factors, they are strongly43 correlated as shown in Table 1.
Using the APS Employee Engagement Model
Understanding engagement in the APS
The results of the analysis of the Survey have shown that employee engagement in the APS can be conceptualised as the relationship an employee forms with a number of separate elements of their work: the job they do, the people with whom they work, their supervisor, and the agency within which they work.
In terms of the job people do, engagement is a function of whether the job challenges the employee; whether it gives them an opportunity to be innovative, gives them an opportunity to use and develop their skills, and whether it stimulates a degree of interest in the employee. It also relates to the sense of identification an employee has with the job; do they see the job as a fundamental part of who they are? Do they get a personal sense of achievement from doing their job?
Team engagement revolves around the recognition that employees get for doing their job in a broad sense: do they receive feedback for what they do, are they happy with the feedback they get, and does the feedback allow them to do their job better. This is not limited to just feedback from employees’ supervisors, rather it is open to all forms of feedback. The other component of team engagement is the work-related behaviour of other team members, does the team share information, cooperate, and treat each other well; is it a workplace where the employee wants to come to work?
The behaviour of immediate supervisors in terms of honesty, concerns for their staff, and the provision of feedback is a key component of Supervisor Engagement. So too is the workplace environment and conditions created by the supervisor: the quality of the working relationships, the flexibility of the working conditions, and the feedback provided to staff.
Agency engagement is the most complex component of engagement encompassing the quality and behaviour of senior leaders, the developmental opportunities provided by the agency, the behaviours exhibited by agency employees and how well the employee identifies with the agency, including how well the individual identifies with praise for the agency.
A compensatory model
These results show that the elements of engagement are separate but related (Table 1). There is a degree of correlation among the elements that means that they can be aggregated in a meaningful way if needed, but that they can also be considered independently, and so can compensate for each other to some degree in contributing to organisational performance. For example, high team engagement may compensate for poor job engagement. The compensatory nature of the model can apply to individuals, workplaces or even agencies. For example a workplace where work is not particularly challenging for employees might still experience good organisational performance due to good agency leadership and hence high levels of agency engagement.
The possible compensatory nature to employee engagement in the APS requires further examination, but it has the potential to provide managers and HR practitioners a much more sophisticated understanding of engagement in their workforce and allow them to tailor engagement strategies for different segments of their workforce depending on their specific characteristics.
Presenting engagement results
As discussed earlier in this paper, one of the problematic features of many engagement models is the use of the percentage of ‘engaged’ or ‘disengaged’ staff in the workplace as the key engagement metric in an organisation. This suggests two things: first, that the concept of engagement can be measured with a degree of precision that some would argue is not possible to achieve with a self–report survey44 and second, that there is some fixed point at which an employee becomes engaged (or disengaged) and their workplace behaviour subsequently changes.
The idea of an absolute measure of engagement is unrealistic for two reasons. Firstly, engagement is a psychosocial concept and therefore highly subjective and highly individual. Secondly, almost all methods of measuring employee engagement use self-report survey that are, irrespective of the content, subjective and prone to some degree of error related to the method of the data collection. There are both statistical and methodological means of mitigating error so this should not discourage attempts to measure engagement, but it does substantially weaken any claims that might be made about absolute levels of engagement either in individuals or organisations.
Finally, as demonstrated here, employee engagement involves a set of complex relationships between the employee and the workplace. To suggest it can be described by a single index fails to recognise the complexity of these relationships. Engagement measures need to reflect the multi-dimensional nature of the relationship between an employee and their work and it is unlikely that overall engagement can be presented in something as singular as the percentage of engaged staff. Rather one is tempted to ask, “engaged with what aspect of work?”
While making absolute statements about levels of engagement has significant flaws, making relative statements are significantly less affected by the fundamental nature of engagement or the method used to measure engagement. As a result, the APS Engagement model eschews reporting percentages of (dis)engaged staff. Instead engagement levels are reported on a common zero to ten index for each component of engagement. This avoids some of the criticisms made regarding making absolute statements about engagement. Secondly, the zero to ten point index allows for employees to have varying levels of engagement with different components of their workplace. For example, one employee might be highly engaged with their job but less engaged with their team while another employee might be more engaged with their team than their job. Both might be considered ‘engaged’ staff, however, they are different in their engagement with their workplace. By reporting engagement levels on this common scale, meaningful comparisons (i.e., relative statements) can be made across the different elements of engagement.
By reporting engagement scores on a common scale, meaningful comparisons can be made either within an agency over time, across different segments of an agency workforce, across different segments of the APS workforce (e.g., age groups, sex, etc.), and across agencies within the APS. Making comparisons against “benchmarks” like these can be useful for APS managers and HR practitioners but they require context. Comparison across different agencies might be misleading if the natures of the agencies are substantially different or if the context for the agencies is different (e.g., if one agency had recently undergone a major restructure). Comparisons within an agency over time are particularly useful because they provide one way for APS managers and HR practitioners to test what effect their engagement interventions have had.
Applying the results of the APS Engagement Model
The purpose of any workforce research in the APS is to improve our understanding in order to guide managers and HR professionals in the development of strategies that will improve some aspect of the workforce or workplace that will lead to improvements in productivity and therefore better outcomes for APS clients.
Because the Model provides a multi-dimensional view of employee engagement, it allows managers to develop a more sophisticated understanding of the engagement issues in their organisation. Similarly, identification of clear antecedents of engagement supports the development of tailored strategies for addressing engagement in the workplace; an agency with low levels of job engagement due primarily to low job challenge might invest in job enrichment strategies, whereas another agency with high levels of job challenge, but lower agency engagement due to limited learning and development opportunities might gain greater value from investing in a learning and development strategy.
The identification of a suite of measureable consequences of employee engagement in the form of observable organisational behaviours will allow agencies to tailor engagement strategies that target these specific outcomes. For example, if workplace psychological injury claims were identifiable as related to specific elements of employee engagement, managers in an agency with high claim rates might develop an engagement strategy that targets these elements of engagement specifically with the aim of reducing the claim rate. Work remains to be done to identify these consequences and the relationships between the antecedents of engagement and their consequences. However, once relationships and consequences have been identified they will provide APS managers and HR practitioners with a very sophisticated model for addressing key organisational capability issues in their workplace.
Limitations and further research
There are some limitations to this work; primary among these is that while the Survey provided good coverage of the aspects of the hypothesised engagement model, it wasn’t developed to measure this specific model of employee engagement. As a result, the work reported here represents an exercise in fitting an existing theoretical model of employee engagement to a different existing set of data. The fact that the model fits the data well suggests there is a degree of robustness to the model that provides a high degree of confidence for adapting it as the APS Employee Engagement Model. It does, however, also suggest that there is potential for future refinement of the model as described in this paper.
The drivers and outcomes of engagement
Having identified a model of engagement that gives a greater understanding of the nature of engagement in the APS and shows the relationships among its elements, the next key is to identify the drivers (or antecedents) and outcomes (or consequences) of engagement. What aspects of working in the APS lead to higher levels of job identification, for example, and hence job engagement? And what impact will improved job engagement have on employee performance and availability (retention, use of sick leave, etc.)
To deliver a useable outcome for APS managers and HR practitioners, the APS Employee Engagement Model must be able to show a clear link between those aspects of the workplace that influence or drive engagement and the subsequent impact on workplace outcomes. Figure 1 above shows this link graphically, but there is now a need to test this empirically. This will also test the theoretical strength of the APS Employee Engagement Model and identify its utility to the APS.
The compensatory nature of engagement
The nature of the relationships among the elements of engagement suggests that there may be a compensatory nature to engagement in the APS that would help managers and HR practitioners better tailor engagement strategies to their specific context. The hypothesised compensatory nature of engagement needs further investigation to determine whether it exists and, if so, how it can be used to better understand and improve engagement in the APS.
Trust and engagement
Trust has been hypothesised as a key element in the APS Employee Engagement model and although there is a substantial body of literature on the importance of trust in the workplace, it’s contribution to engagement in the APS needs to be examined to determine what role it plays and how the APS might respond to trust in the workplace.
Other elements of engagement in the APS
There is also scope for future work examining other elements of employee engagement. The APS Employee Engagement Model has been adapted from one developed primarily for the private sector and there is good argument that substantial differences exist between the two sectors. As a minimum, the relationship between the Model and the concept of public service motivation would warrant further examination.
The APS Employee Engagement Model was developed after an examination of the literature on employee engagement identified an existing model of employee engagement that met the needs of the APS and was subsequently adapted for the APS. This model is based on the assumption that employee’s relationship with work is multi-dimensional and characterised by reciprocity and trust. It recognises that these relationships can vary across the APS, and gives APS managers and HR practitioners a sophisticated understanding of the engagement issues specific to their workplace so that they can develop engagement strategies tailored to their specific context.
There are still a number of steps required to complete the development of the model. The first is to identify drivers of engagement in the APS and the subsequent impacts on APS organisational performance. Second, the compensatory nature of the model will need to be tested: will different segments of the APS workforce demonstrate different levels of engagement on the different elements of engagement? The relationship between trust and engagement also needs further examination.
Finally, possible extensions to the model need to be considered. The current model was developed using existing item sets from the APS Employee survey, and while the approach taken has been psychometrically rigorous, it has not had the benefit of the exploration of other concepts that might contribute to the understanding of employee engagement. For example, the concept of public service motivation is one that warrants further consideration for how it relates to the concept of employee engagement in the public service.
Appendix A: APS Employee Engagement Model initial item sets
Items related to the Job component of the APS Employee Engagement Model
Please rate your level of agreement with the following statements regarding your current job:
- I enjoy the work in my current job.
- I am motivated to do the best possible work that I can.
- I do my work for the satisfaction I experience from taking on interesting challenges.
- When needed, I am willing to put in the extra effort to get a job done.
- My job allows me to utilise my skills, knowledge and abilities.
- My work has become a fundamental part of who I am.
- Overall, I am satisfied with my job.
- My current job will help my career aspirations.
- My job gives me a feeling of personal accomplishment.
- I have a clear understanding of how my own job contributes to my work team’s role.
- I stay in my current job for the salary it provides me with.
- I receive adequate recognition for my work contributions and accomplishments.
- My job gives me the opportunity to work on the tasks I do best.
- I clearly understand what is expected of me in this job.
- I have the authority (e.g. the necessary delegation(s), autonomy, level of responsibility) to do my job effectively.
- I want to succeed at my job so others will think highly of me.
- I receive adequate feedback on my performance to enable me to deliver required results.
- I am constantly looking for ways to do my job better.
- I am satisfied with the recognition I receive for doing a good job.
- I am fairly remunerated for the work that I do.
- I frequently try to help others who have heavy workloads.
- I am motivated at work because it helps me achieve my career goals.
- I would be more productive if there was less ‘red tape’ (e.g. regulatory or administrative processes).
- I receive appropriate training and/or have access to information that enables me to meet my recordkeeping responsibilities.
Please indicate your level of satisfaction with EACH of the following workplace attributes in your current job:
- Duties/expectations made clear
- Regular feedback/recognition for effort
- Chance to be innovative
- Chance to make a useful contribution to society
- Seeing tangible results from my work
- Opportunities to utilise my skills
- Opportunities to develop my skills
- Good working relationships
- Appropriate workload
- Remuneration package (e.g. salary, superannuation)
- Opportunities for career development
- Interesting work provided
- Flexible working arrangements
- Good manager
- Appropriate level of autonomy in my job
Items related to the Team component of the APS Employee Engagement Model
Please rate your level of agreement with the following statements regarding your immediate work group:
- The people in my work group feel they are valued for their contribution.
- The people in my work group use time and resources effectively.
- The people in my work group cooperate to get the job done.
- I feel that my own ideas are genuinely considered when strategies, goals or tasks are being set for my work group.
- The people in my work group share job knowledge with each other.
- The people in my work group are honest, open and transparent in their dealings.
- The people in my work group treat each other with respect.
- The people in my work group resolve conflict quickly when it arises.
- I have a clear understanding of how my work group’s role contributes to my agency’s strategic directions.
Items related to the Supervisor component of the APS Employee Engagement Model
Please rate your level of agreement with the following statements regarding your supervisor:
- My supervisor ensures fair access to developmental opportunities for employees in my work group.
- My supervisor encourages me to build the capabilities and/or skills required for new job roles.
- My supervisor appropriately deals with employees that perform poorly.
- My supervisor demonstrates honesty and integrity.
- My supervisor works effectively and sensitively with people from diverse backgrounds.
- My supervisor delegates work effectively.
- My supervisor shows concern for the welfare of his/her staff.
- My supervisor draws the best out of his/her staff.
- My supervisor encourages and manages innovation.
- My supervisor provides effective feedback.
Items related to the Agency component of the APS Employee Engagement Model
Please rate your level of agreement with the following statements regarding the senior leaders (i.e. the Senior Executive Service [SES]) in your agency:
- In my agency, the leadership is of a high quality.
- My agency is well managed.
- In my agency, communication between senior leaders and other employees is effective.
- In my agency, senior leaders are receptive to ideas put forward by other employees.
- In my agency, senior leaders discuss with staff how to respond to future challenges.
- In my agency, the most senior leaders are sufficiently visible (e.g. can be seen in action).
Please rate your level of agreement with the following statements regarding aspects of your agency’s working environment:
- My agency is a good place to work.
- In general, employees in my agency effectively manage conflicts of interest.
- In general, employees in my agency appropriately assess risk.
- Employees in my agency feel they are valued for their contribution.
- My agency places a high priority on the learning and development of employees.
- My agency operates with a high level of integrity.
- My agency deals with underperformance effectively.
- My agency encourages the public to participate in shaping and administering policy (e.g. seeks and uses feedback, consults and engages communities on issues affecting them).
- Working at my agency is important to the way that I think of myself as a person.
- When someone praises the accomplishments of my agency, it feels like a personal compliment to me.
- When I talk about my agency, I usually say “we” rather than “they”.
- My agency routinely applies merit in decisions regarding engagement and promotion.
- My agency promotes and supports good record keeping practices.
- My workplace provides realistic performance expectations.
- My workplace provides increased knowledge and/or experience in the job.
- My workplace provides access to effective learning and development.
- My workplace provides good working relationships with my manager and colleagues.
- Overall, my agency has sound governance processes for effective decision making.
- I am satisfied with my non-monetary employment conditions (e.g. leave, flexible work arrangements, other benefits).
- I am well paid compared to what I would receive in other agencies.
- My agency provides an ethical working environment.
1 Australian Public Service Employee Engagement Model by Australian Public Service Commission is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.
2 Scottish Executive Social Research Employee Engagement in the Public Sector: A Review of the Literature. Edinburgh, May 2007. This is a very comprehensive review of the literature in this area and is the source of much of this section of this paper.
3 Terms of Engagement Scott Flander (2008), http://www.hreonline.com/HRE/story.jsp?storyId=59866490.
4 Scottish Executive Social Research. Employee Engagement in the Public Sector: A Review of the Literature. Edinburgh, May(2007), p 3.
5 Scottish Executive Social Research. Employee Engagement in the Public Sector: A Review of the Literature. Edinburgh,, May(2007).
6 First, break all the rules: what the world’s greatest managers do differently. Buckingham, B., and Coffman, C., 1999.
7 Those characteristics of the workplace and the workforce that are required for engagement to occur, they are often referred to as “drivers” of engagement.
8 The attitudes and behaviours exhibited by “engaged” employees.
9 Saks, A.M. Antecedents and consequences of employee engagement. Journal of Managerial Psychology, 21 (7), (2006).
11 Macey, W.H., and Schneider, B. The Meaning of Employee Engagement. Industrial and Organizational Psychology, 1, 2008, p 4.
12 Employee Engagement underpins Business Transformation. Towers Perrin, 2009. http://www.towersperrin.com/tp/getwebcachedoc?country=gbr&webc=GBR/2008/200807/TP_ISR_July08.pdf
13 Working Today: Understanding what drives employee engagement. Towers Perrin, 2003.http://www.towersperrin.com/tp/getwebcachedoc?webc=hrs/usa/2003/200309/talent_2003.pdf
15 Corporate Leadership Council, Driving Employee Performance and Retention through Engagement. Corporate Executive Board, Washington, D.C. 2004.
16 Macey, W.H., and Schneider, B. The Meaning of Employee Engagement. Industrial and Organizational Psychology, 1, 2008, p 4.
17 Perry, J.L., Hondeghem, A., and Wise, L.R.Revisiting the Motivational Bases of Public Service: Twenty Years of Research and an Agenda for the Future. Public Administration Review, 70 (5), p 681-690.
18 Scottish Executive Social Research Employee Engagement in the Public Sector: A Review of the Literature. Edinburgh, May 2007, p2.
19 Advisory Group on Reform of Australian Government Administration, Ahead of the Game: Blueprint for the Reform of Australian Government Administration. Commonwealth of Australia, Canberra, 2010.
20 Ellis, C.M., and Sorenson, A. Assessing Employee Engagement: The Key to Improving Productivity. Perspectives: Insights for Today’s Business Leaders, 15(1), January 2007.
21 Muchinsky, P.M. Psychology Applied to Work: an introduction to industrial and organisational psychology. 10th Edition. Hypergraphic Press, Summerfield NC, 2012. p305.
22 Muchinsky, P.M. Psychology Applied to Work: an introduction to industrial and organisational psychology. 10th Edition. Hypergraphic Press, Summerfield NC, 2012. p306.
23 Helliwell, J. and Huang, H. Wellbeing and trust in the workplace. National Bureau of Economic Research Working Paper 14589. Cambridge, MA.
24 See, for example, Rousseau, D.M., Psychological Contracts in Organisations: Understanding written and unwritten agreements. 1995. London, Sage.
25 See, for example, Georgakopoulous, A., Wilkin, L, and Kent, B., Workplace bullying: a complex problem in contemporary organisations. International Journal of Business and Social Science, 2(3). 2011, pp 1-20.
26 Thomas, K. The Oxford Book of Work. Oxford University Press, 1999, p 23.
27 See, for example: Kahn, W.A. (1990). Psychological Conditions of personal engagement and disengagement at work. Academy of Management Journal, 33(4), 692Saks, A.M. Antecedents and consequences of employee engagement. Journal of Managerial Psychology, 21 (7), (2006).
28 Corporate Leadership Council, Driving Employee Performance and Retention through Engagement. Corporate Executive Board, Washington, D.C. 2004.
29 Over 50,000 employees, from 59 different companies or government organisations in 27 different countries.
30 Organisational commitment is one of the seminal inputs in the development of concept of employee engagement.
31 Meyer, J. P., & Allen, N. J. (1991). A three-component conceptualization of organizational commitment. Human Resource Management Review, 1, 61–89.
32 Saks, A.M. Antecedents and consequences of employee engagement. Journal of Managerial Psychology, 21 (7), (2006).
33 A full list of items is included in Annex A.
34 All analyses were conducted using Stata 10.1/IC.
35 Ender, B. fapara: command to conduct parallel analysis for factor analysis and principal components analysis. UCLA: Statistical Consulting Group. http://www.ats.ucla.edu/stat/stata/faq/parallel.htm
36 Scales constructed during these analyses used simple unit-weighted linear composites of item scores.
37Kolenikov, S. (2009). Confirmatory factor analysis using confa. The Stata Journal, 9(3), pp329-373.
38 Hu, L., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
39 Kenny, D.A. Measuring Model Fit. http://davidakenny.net/cm/fit.htm
40 Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newsbury Park, CA: Sage
41 Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioural Research, 25, 173-180.
42 Marker items are indicated throughout by an asterisk (*)
43 Cohen, J. (1988). Statistical Power Analysis for the behavioural sciences. (2nd Edition). New Your. Psychology Press.
44 The self-report survey is by far the most common means of measuring employee engagement.