Open data migration to CKAN

The players

Our Victorian government client works across a broad range of areas and manages over 200 open datasets. (Note: Due to a strict media/communications policy, unfortunately we can’t name this department.)

The problem

The agency is responsible for over 200 open datasets, including large open datasets on agriculture such as Victorian land use data and Farmbuild. The open datasets are extremely important to Victoria, and are used to gain insights and make better decisions with positive social impacts on citizens.

While 19 of the open datasets had been manually brought over to data.vic.gov.au and made publicly available, another 203 open datasets were on an old, outdated system built on an implementation of GeoNetwork Open Source platform. The system was difficult to use and making open datasets publicly available was complicated. The system’s clunkiness was also a barrier for department employees who manage the open datasets (often scientists), and so internal uptake was low.

The department needed to shutdown the legacy system and migrate the open datasets to CKAN, a more current and adaptable data portal platform. Moving to CKAN would also make it much easier to make the data publicly available in the future.

The solution

Salsa Digital was engaged to help with dataset migration to CKAN and to create a user interface (UI) that was modern, visually appealing, responsive and user-friendly.

The department already had a good handle on what they needed, so our initial meetings and workshops focused on implementation. We firmed up the three main streams of work required:

1. Customise the CKAN user interface for the department, using their branding.

2. Customise CKAN to map the dataset schemas from the old system to CKAN.

3. Migrate 19 open datasets from data.vic.gov.au and 203 open datasets from the legacy system into their new CKAN instance.

We then turned the three streams into a set of user stories in our project management software (JIRA). Once the user stories were approved, we started work on them straight away. The project was driven by a short time frame — the team wanted the new CKAN system set up in three weeks, with the first two streams completed before Christmas (2017).

We used the latest stable version of CKAN and set the platform up within the department’s private network. At this stage only some of the the open datasets are publicly available (on data.vic.gov.au) but this project will make it easy to bring the open datasets across in the future.

It was a small project, but our agile methodology was still in place, this time as a lightweight layer on top of the three main tasks. This helped us agree on things and have good points of reference for project work. Most communications were through JIRA, and the rapid response time from the department’s product owner ensured we could meet the tight deadline.

The benefits

The benefits of moving to CKAN are:

• A new, strong foundation reflecting an emerging platform built-for-purpose to expose and help discover open data sets.

• An open source tool (no license fees and no vendor lock-in) with a large and active global community.

• An easy-to-use platform, which will hopefully increase their internal users.

• A new, visually appealing and user-centric interface (again, lowering the barriers for internal users).

• The ability to quickly and easily present private open datasets to the public down the track.

• A widely used platform, which gives the department more opportunities in the future to integrate with other open data platforms.

• The ability to extend and customise as they need to (not restricted by the legacy system).

• Standardisation of dataset management.

In addition, the business unit that managed the CKAN project has experienced increased exposure internally and has delivered many benefits to the whole department.

Why Salsa Digital?

Salsa Digital was recommended to this department by the Department of Premier and Cabinet (DPC), based on our work on data.vic.gov.au, Victoria’s open data portal. With several more recent government projects in Victoria, we’ve established a reputation in the Victorian Government for our expertise and customer service. Our skill sets also matched the job requirements — the department had already decided they wanted to use CKAN, and Salsa had the in-house people to run the project.

In addition, Salsa is a contributor to the open government movement, specialising in key themes such as open source (e.g. GovCMS and Drupal), open data, open platform and all things related to digital transformation in government (DTIG). We’re trusted in the marketplace and know the technology and players extremely well.

Salsa Digital is looking forward to further strengthening the relationship with our Victorian government client to not only support their open data journey but also assist them with their open source and Drupal needs.