Can Australia get an open Open Data Catalogue?

I read over at Craig Thomler's eGov AU blog that Brisbane City Council has launched their own open data catalogue site. Great news, right? Not entirely.

Unfortunately it appears to be yet another Australian site reinventing the open-data-catalogue-wheel. So far we have:

Open Source Data Catalogues

How come none are using and contributing to an open source data catalogue like CKAN or Open Data Catalog? These projects are so much more advanced than the sites that have been deployed in Australia and it's no wonder - CKAN's maturity is because it has been developed for over four years.

Not only that but using an open source open data catalogue simply means it's cheaper for Australian governments to provide these services. If data.gov.au had used and contributed to an open source data catalogue we'd probably have had the vast majority of Australian local customisations done (if any were needed) and subsequent catalogues by Victoria, NSW and now councils would border on trivial to set up and deploy.

Doing It Right

The open source catalogues I mention above also get some key things right that Australian sites deployed so far don't - namely open and public data requests and an API (oh the irony of open data catalogues not having a good any API!).

Open and public requests are critical and would highlight the lacklustre data released to date in Australia. The Brisbane catalogue mentioned above is a prime example, there's a clear and demonstrable need for development application data to be opened and freed but it hasn't been chosen as one of the items to be included in the data that launched with the site. If there was a way for data developers to make public requests and not a glorified contact form we might see this data opened at some stage. As it stands, and while there's a scraper gathering the data that simply ignores the onerous conditions, I doubt we'll see this.

How do we fix it?

However there's no point in moaning without suggesting a way to fix things. One way the open data community in Australia could promote the use more advanced open source data catalogues would be to simply set up our own one. A system like CKAN can slurp in data from all of the existing catalogues, thereby providing a comprehensive centralised source of all Australian open data (not just one level of government and not just community projects like OpenAustralia). This would also provide an API to that data and a central place where everyone can log requests for open data and track what data is really in demand by open data hackers in Australia.

Maybe someone should take this on as a project for #odhd? ;)

Comments

These are imported from this site's old blogging software and are kept for archival reasons.

by

Thought you might be interested in this. Australika still a long way behind… http://blog.law.cornell.edu/voxpop/2011/12/05/opengovernment-org-researching-u-s-state-legislation/

by

Very much so - thanks Peter.

Interesting article. Of course the US has its own problems, “approval of the U.S. Congress down to 9% — lower than, yes, communism, the British Petroleum company during the oil spill, and King George III at the time of the American Revolution.”. Ouch.

Thanks again for posting the article.

by

Hi Henare,

Considering that by default data in data catalogues are open for reuse, why not build a community site that includes the data, plus rates it for openness (using Berner-Lee’s measures), includes crowd sourced data transformation (where data is released in problematic ‘open’ formats such as PDF) and a common API for any data?

This could include community dialogue tools for discussion and registering interest per datasets to help people find each other and share ideas for reuse and data visualisation tools built on some of those available in the open source community.

This would go far beyond the government’s efforts and open the door to asking governments for money to operate it on a not-for-profit basis.

I would be very happy to be involved on the funding sourcing, system concept, design & operations - but are not a developer.

by

Hey Craig,

Thanks for the response! Yes, this is exactly what I was suggesting in my post - set up a community site that pulls in data from the all the existing data catalogues and enhances it.

I really like your idea about rating data for openness. In fact one of the open source systems I suggested we use, CKAN, already has an extension for this - CKAN Quality Assurance Extension.

It wouldn’t take a lot of effort to get something like this off the ground. I was being serious when I suggested someone take it on for International Open Data Day last year :)

Maybe we should look at this together for the next OpenAustralia Hackfest? I’d be happy to get the site set up and then the bulk of the work would be importing the datasets, running the site and promoting it.

Given the recent coverage of what’s happening with data.gov.au - do you know of anyone else that might be interested in a project like this? If so we might be able to get it off the ground sooner rather than later.

Cheers, Henare

by

Sorry for late reply - not many people are active in this space in Australia, so sorry no :(

Pia is best bet - but she’s working with ACT gov on their data portal (socrata base).