Data61 revamps government data to make it more publicly accessible
National Map: Visualisation of patent applications in New South Wales
The Commonwealth Scientific and Industrial Organisation's (CSIRO) Data61 is looking to make government data more accessible to all Australians, revamping data.gov.au and nationalmap.gov.au, and working on the creation of government dashboards -- to name just a few projects currently underway.
The Australian government initially launched data.gov.au in 2010 as a tool allowing for the publishing of open data across all jurisdictions of government in the country. It then evolved in 2013 to see application programming interface (API)-style access to datasets.
Since then, more data has been collected and more government departments, enterprises large and small, and the average Australian citizen have become eager to get their hands on the information.
Last year, the Department of Prime Minister and Cabinet went to Data61 with a brief of building "world-leading" data infrastructure -- essentially a revamp of the data.gov.au portal.
Since July last year, Cam Grant, Data61 senior user experience designer, and his colleagues have been working on that challenge and as a result, have made available an Alpha version of its new platform: preview.data.gov.au.
"Governments are releasing more and more data and it enables all sorts of great stuff -- mainly better transparency and accountability across government. It is a good thing, but it also enables new economic possibilities, new businesses -- stuff we haven't dreamt of yet," Grant said.
"Along with that, it creates a lot of interesting challenges, privacy being one of the biggest challenges, but also just the challenge of infrastructure, how are we going to get all this data out there. If there are endless streams of data, how do we find the right data?"
Grant told ZDNet the team is currently "reimagining" this core piece of infrastructure, touted the current Alpha platform as "pretty cool", with some features not previously possible, such as filtering by location visually.
"We're trying to enhance the discoverability of data," he explained. "We're not trying to reinvent the wheel or take away the software that currently runs that service, but we're trying to make data more discoverable and more usable so that means raising the quality of data.
"We're also trying to make sure that we plug all these privacy technologies that we have into this pipeline to make sure that we don't breach individuals' or corporations' privacy and confidentiality."
In addition, Grant explained that Data61 is working on extending data.gov.au to enable people to work together on projects, allowing access to data set combinations others have created.
"We realised there was a lot of disconnect between the consumers and the providers of data and that gave rise to the idea of collaboration," Grant said. "We can bring the right people, the right experts, the right data, and the right tools and then when stuff happens, capture all of those linkages. So we have a whole bunch of ideas around collaboration that we're hoping to build out starting this year."
Another data-driven project Data61 is working on is around reporting dashboards for the government.
Using published data, Data61 is hoping to communicate the complexity of government programs to "ordinary Australians" so they can quickly see things like the current state of housing in Tasmania in order to get a really quick snapshot of things like pricing and auction hot spots; essentially, optimising the data so people can understand it.
Tight-lipped on the specifics of the federal government project, Hilary Cinis, UX group leader and principal designer at Data61, said it forms part of the government's transparency agenda.
Data61 has also been working on the federal government's National Map project alongside the Department of Communications and the Arts, Geoscience Australia, and other government agencies.
Launched in mid-2014, National Map enables the visualisation of any publicly available spatial dataset, made available by the federal government, state governments, and local governments.
Similar to Google Earth, users can search for places and things, as well as place available public datasets -- and a user's own datasets -- on top of the map.
"What's really important about this is that we do not actually cache any of the data so National Map just knows where this data lives and then it works in a federated manner," Simon O'Callaghan, senior research engineer at Data61, explained.
"So if the Department of Communications -- for example -- as the custodian of the data updates their data set, it gets updated on National Map."
Not just a gimmick, O'Callaghan told ZDNet that Data61 actively uses the map and the public datasets for its own research as well.
"We really love the idea that it's not a frozen snapshot of time. So every time that dataset updates, it's updated on the website so it's new, it's no longer historic -- it's continuous," Cinis added. "We can see that the dashboard, data.gov.au, and the National Map will become this big thing -- where all these platforms are fed by the data warehouse."
Cinis explained that Data61 is also currently investigating partnerships with large private sector companies, mainly around what she called privacy preserving analytics, which would allow the user to combine their dataset with someone else's.
"So we're using an interface design to allow people to explore the similar features and create models without ever actually seeing what's inside that dataset -- and that's not even deidentified data, it's just entirely masked but they're using algorithms and other things to represent the information."
According to Grant, Data61 is attempting to take a more "holistic" view about what data is actually needed by people. He said there is a need to somewhat resurrect it from being buried in a department somewhere and transform it into valuable insight for citizens.
Under laws introduced to the Australian Senate in October, intentionally re-identifying a de-identified dataset will become punishable by up to two years' imprisonment, with the laws to be retrospectively applied from September 29, 2016.
When announcing the proposed legislation in September, Australian Attorney-General George Brandis said open data was a vital part of modern government, and claimed "privacy of citizens is of paramount importance" to the government.
In February, the Senate Legal and Constitutional Affairs Legislation Committee recommended the legislation be passed through Parliament despite concerns about the scope of the law, its reversal of the burden of proof, exemptions under it, and the retrospective nature of it.
In tabling its report in Parliament, the committee's report outlined several key issues with the Bill: The release of de-identified information; the criminalisation of re-identifying data; the scope of the offences; the scope of the minister's exemption powers; the retrospective application of the laws; and the reversed burden of proof from the prosecution to the defendant.
According to the Senate committee, however, these concerns are all overridden by "the gap that was recently identified in privacy legislation", with the committee noting it was of the view that the Bill provides a necessary and proportionate response.