Chapter 7 - Open data supporting open innovation
Opening up the APS innovation process requires making best use of available tools to access, assimilate and apply the knowledge that is available inside and outside the APS. Tools range from those that facilitate information transfer and knowledge management within agencies and across the APS to those that enable citizens to create and configure their own product with tools provided by the agency, or enable agencies to integrate external problem solvers (academics, businesses or citizens) into developing and delivering innovative services.
In October 2013, a joint project of the World Wide Web Foundation and the Open Data Institute studied 77 countries' open government data practices at a national level. Australia ranked seventh in the list of countries most advanced in open data readiness, implementation and impact. The United Kingdom (UK) was ranked first, followed by the United States (US), Sweden, New Zealand, Norway and Denmark.5
The UK Government places high priority on open data, launching its Open Government Data initiative in 2009. The emphasis, focus and importance it places on open data is best captured by Francis Maude who has been responsible for overseeing the UK Government's transparency policy. Maude sees open data as ‘a raw material for economic growth—just like iron and coal were to the industrial revolution’.6
In Australia, the Declaration of Open Government, the establishment of data.gov.au, and the publication of the Principles on Open Public Sector Information are all central to ensuring the data held by Australian Government agencies is more readily available for re-use. One report listed the value to be gained from open government data as7:
- reducing the cost of existing government and private services (enhancing efficiency, doing more with less)
- enabling new services and improved quality of services (enhancing innovation)
- improving transparency and accountability (enhancing consumer empowerment and governance).
These effects contribute to engendering greater trust in government, which enables further benefits, such as greater participation.
The central Australian public dataset repository (data.gov.au) has approximately 3,700 government datasets available for re-use.8 This case study is one example of how open government data enabled the development of a new information service.
Department of Industry: Energy rating app
The free energy rating app is designed to help households and businesses save money on their power bills by choosing energy efficient appliances. The app uses the data behind the well-known and trusted energy rating labels that are part of a highly successful industry and government programme. The labels appear on the more than 7 million appliances sold in Australia and New Zealand every year.
Manufacturers provide this data as they register their appliances for sale. The Department of Industry provides this data on the energy rating website (energyrating.gov.au) and data.gov.au. Making data freely available unlocks information for those who need it and creates opportunities for developers and industry.
Uploading data onto data.gov.au automatically creates a free application programming interface which makes the data machine-readable and easier for developers to use and combine with other datasets to create new apps.
This interface also makes it possible for retailers to automatically and accurately update the energy rating information on their websites every 24 hours to give customers the very latest information about their products.
Data.gov.au made it easy for the department, suppliers and developers to collaborate and create the energy rating app and create new business opportunities, promote energy efficient appliances and help households and businesses save money.
Open government data
Open data, big data, and open government data are related terms. A recent report defined open data sets as sharing these characteristics9:
- accessibility—a wide range of users is permitted to access the data
- machine readability—data can be processed automatically
- cost—data can be accessed free or at negligible cost
- rights—limitations on the use, transformation and distribution of data are minimal.
The same report provides this additional distinction between open data and big data:
Open data sets also are defined in relation to other types of data, especially big data. ‘Big data’ refers to data sets that are voluminous, diverse, and timely. Open data is often big data, but ‘small’ data sets can also be open. We view open and big data as distinct concepts. ‘Open’ describes how liquid and transferable data are, and ‘big’ describes size and complexity of data sets. The degree to which big data is liquid indicates whether or not the data are open.
Figure 7.1 shows the distinctions within each category.10
Figure 7.1. Mapping the distinction between open data, big data and open government data
Source: Adapted from http://www.opendatanow.com/2013/11/new-big-data-vs-open-data-mapping-it-out.
A map of the government data landscape, including projects, policies and community initiatives, demonstrates the variety of government data available for public re-use.11 The following examples provide insight into the way open government data is being made available by APS agencies for re-use. The first shows how open government data can be used to draw together people from government, industry, academia and the general public to mix, re-use and remix government data. The second shows how a small change in the way data is available and accessible can lead to efficiencies for the APS and those outside the service.
Encouraging engagement with government data
GovHack is Australia's largest civic hackathon. It represents an important shift towards collaborative and cross-disciplinary problem solving involving the public sector, private sector and citizens of all ages and skills. Its purpose is to find new ways of using government data and encouraging open government and open data.12
GovHack 2014 brought together more than 1,300 competitors and observers from across Australia to create more than 200 ‘hacks’ using data from all levels of government. The competition uses the term ‘hack’ in the positive sense. The idea is to bring data together in novel ways to create something new or to solve a technical or real-world problem. The overriding objective is to challenge participants to use government data to address real community needs and build social and economic benefits.
Competition entries cover a broad range of themes such as digital humanities, science, data journalism, open government, social inclusion and health. They include creative applications, crowdsourcing tools, data visualisations, detailed analysis, new ways to deliver government services, and functional improvements for open data platforms across the country. In 2014, many teams were extremely creative not only in how they used government data but also in addressing real community needs, and building social and economic benefits.
GovHack brings citizens face-to-face with policy makers to deliver first-hand perspectives of what the community needs from government and the data it holds, while showcasing the technical community in Australia.
The benefits of making government data available
On Budget Night 2014, for the first time, the tables and spreadsheets from the Budget were published in data form on data.gov.au. This collaborative project between the Department of Finance (Finance) and the Department of the Treasury (Treasury) focused on two key activities.
First, the tables from agency Portfolio Budget Statements (PBS) were made available in spreadsheet form. Historically, this information was only available in the publicly-available PBS reports as PDF documents. Significant additional manual effort was then required by data users to translate the data from PDFs to a form that could be accessed and used.
Second, key tables from Budget Paper 1 and Budget Paper 4 were made available in spreadsheet form. Finance undertook to make a number of these spreadsheets available as machine-readable files which allowed easier re-use for analysis and data visualisation.
In the lead up to Budget Night, Finance engaged with well-known Budget data users to ensure the data was published and made accessible in a machine-readable and beneficial format. Two weeks before publishing the data, Finance announced in a blog post that the data would be released. This ensured many Budget data users would not unnecessarily engage in the work of transforming data from the Budget Papers, as they traditionally had to do.
While this was a modest effort in providing open government data to users, it yielded good returns for the public sector, industry and the community. The Budget 2014 dataset on data.gov.au was the third most popular dataset viewed in May and June. It was also the second highest accessed dataset for May and June.
Feedback from users emphasised the time saved by accessing the data directly and in a useable form. For example, several government departments are required to analyse the aggregate budget data from Budget Paper 1 and Budget Paper 4. One department reported it saved eight hours over previous years. Outside the public sector, two organisations reported saving around 20 hours of data transformation time. Previously, it would have been necessary to manually extract data from the PBS PDFs by downloading PDF files individually from respective portfolio agency websites and translating the data into a machine-readable format for data visualisation and analysis.
The 2014 Budget data was used by agencies, industry, the media and the community. These links provide examples of how it has been used:
- The Open Budget (theopenbudget.org)
- BudgetAus (infoaus.net/budget)
- Public Sector Gains and Losses (ausviz.com/wordpress).
Finance and Treasury are reviewing lessons learned from 2014 to identify ways to improve the process and publish more data from the Federal Budget for 2015.
Open data toolkit
Finance provides whole-of-government infrastructure for discovering and publishing data, working closely with many departments and other government jurisdictions to share best practice, code, policy, planning and lessons learned. Finance and the Department of Communications (Communications) are working collaboratively to develop and maintain an open data toolkit that brings together Finance's responsibilities around open and big data policy, Communications' responsibilities around spatial policy, and expertise, best practices and guidance from across the public service, into a single location.
This collaborative approach across the APS and other government entities has created a resource that will be iteratively improved over time through shared resources, expertise and costs, for the benefit of the APS and other public sector organisations. Identifying mutually beneficial opportunities for collaboration is an important way to innovate and improve government services and efficiencies.
Geoscience Australia: Transforming open government with a network of open data
Geoscience Australia (Geoscience) has been committed to open government since 2008 when it began applying Creative Commons By Attribution (CC-BY) licensing to its data and products as the default.1 Open access licensing permits innovative use of Geoscience data and information so research, industry, other government agencies (at all levels) and the community can generate economic, social or other outcomes of benefit to Australians. To maximise re-use of information, however, government data and information needs to be free, discoverable, based on open standards and accessible online in machine-readable formats. This is required to enable uptake by the broadest range of potential users.
In 2013–14, Geoscience implemented the Data Stewardship initiative, creating a new business-as-usual section with a focus on improving data standards and governance, providing innovative data sharing tools and making data openly accessible through web services, to maximise stakeholder use of authoritative geoscience data. Machine-to-machine communication over the internet greatly enhances the rapid discovery, access and use of data and information. Geoscience has been creating spatial data web services since the late 1990s but only skilled spatial scientists could discover and make use of them. Recent advances in information technology and the implementation of broadband networks, however, have driven the development of online portals, mobile applications, and government ‘Globe’ map visualisation tools that draw on web services as the input to regularly updated information products. To satisfy this growing demand for geoscience data, Geoscience created and released approximately 90 new data web services in 2013–14.
In the ‘Government Open Data Network’ initiative, Geoscience works collaboratively with Communications and Finance to provide online tools and services that make it easy for the public to discover, share and visualise government data. In December 2013, for example, Geoscience collaborated with data.gov.au to implement a machine-to-machine harvest of Geoscience's product catalogue, which provides metadata with links to the data. This increased the number of government datasets that can be discovered through data.gov.au overnight from around 400 to more than 3,000. In April 2014, Geoscience launched FIND, the Australian Government's online portal that allows users to discover and access spatial data from agency ‘nodes’ across all levels of government and research data providers. The portal currently provides access to data from 143 organisations. This number will rapidly grow as agencies progressively develop their capability to enable machine-to-machine online access to their data. FIND differs from data.gov.au in that it provides spatial search functionality and simple map visualisation, but metadata is shared between the portals through machine-to-machine catalogue services to maximise the discovery of government data and information.
Geoscience is also working in collaboration with Communications and National ICT Australia on the National Map project, a web mapping application that provides users with intuitive access to a wide range of geospatial data from many Australian Government agencies, from federal to local level. A beta version of National Map was released by the Minister for Communications in the week leading up to GovHack in July 2014. Its architecture is based on open protocols and formats to allow straightforward incorporation of data web services from many systems and initiatives. As a key component of the Government Open Data Network, the National Map will be seamlessly linked to provide visualisation of government open data discovered through data.gov.au and FIND.
1 Government 2.0 Taskforce 2009, Engage: Getting on with Government 2.0, Government 2.0 Taskforce Report, Department of Finance, Commonwealth of Australia, Canberra, viewed 1 October 2014, http://www.finance.gov.au/publications/gov20taskforcereport.
5 Davies, T 2013, Open Data Barometer—2013 Global Report, Joint Project of the World Wide Web Foundation and Open Data Institute, London, viewed 19 September 2014, http://www.opendataresearch.org/barometer.
6 Maude, F 2014, Francis Maude welcomes France to the Open Government Partnership, GOV.UK, London, viewed 19 September 2014, https://www.gov.uk/government/speeches/francis-maude-welcomes-france-to-the-open-government-partnership.
7 Gruen, N, Houghton, J & Tooth, R 2014, Open for business: How open data can help achieve the G20 growth target, A Lateral Economics report commissioned by Omidyar Network, Omidyar Network, London, p. viii, viewed 19 September 2014, http://www.omidyar.com/blog/business-case-open-data.
9 Manyika, J, Chui, M, Farrell, D, Van Kuiken, S, Groves, P, Almasi & Doshi, E 2013, Open data: Unlocking innovation and performance with liquid information, McKinsey Global Institute, p. 3–4, viewed 19 September 2014, http://www.mckinsey.com/insights/business_technology/open_data_unlocking_innovation_and_performance_with_liquid_information.
10 Gurin, J 2013, ‘Big Data vs Open Data—Mapping It Out’, weblog post, 8 November, Datasphere, Open Data Now, viewed 19 September 2014, http://www.opendatanow.com/2013/11/new-big-data-vs-open-data-mapping-it-out.
11 Waugh, P 2013, ‘The Government Data Landscape in Australia’, weblog post, 20 October, AGCTO, Department of Finance, Commonwealth of Australia, Canberra, viewed 19 September 2014 http://www.finance.gov.au/blog/2013/10/26/government-data-landscape-australia.
Page ID: 64060 (Open data supporting open innovation)