Building the second-generation GovCMS

The players

The Department of Finance (Finance) owns the GovCMS platform, a whole-of-government digital platform for use across all levels of government in Australia. GovCMS is built on Drupal, an award-winning, enterprise-grade CMS that’s easy to use, stable, highly secure and open source (no license fees).

The problem

The GovCMS contract was coming to an end and Finance approached the market with a request for proposals. The GovCMS team was looking for an innovative approach and a way to build a second-generation platform that took the existing GovCMS offering to the next level.

Salsa Digital won the three-year contract based on the pitch of a fully open source technology stack using implementation partner amazee.io’s open source platform Lagoon, as well as cutting-edge technology and disruptive open source tools. Other key factors in the contract win were: our track record in GovCMS, our focus and contribution to open government and digital solutions for government, and our successful delivery of the Single Digital Presence for Victorian Government.

The solution

After a thorough discovery process, four parallel streams were set up:

1. Platform setup — setting up an open source stack on AWS.

2. Site migration — building tools and automated scripts to move sites from the old platform to the new platform, including automated testing.

3. Security accreditation — going through our processes and policies to elevate security.

4. Program setup — getting all processes, models, service desk, and underlying aspects of the program ready for the cutover date.

On top of these four streams was an umbrella stream for overall project governance and management.

The whole project was run using agile, bringing together multidisciplinary co-located teams. Our blog Delivering the second-generation GovCMS using agile has more information about the approach.

The new platform introduces many cutting-edge web technologies such as continuous integration (CI), containerisation, self-healing, horizontal and vertical scaling, and significant automation. The platform uses open source tools such as Docker, OpenShift (Kubernetes) and GitLab. The OpenShift cluster is hosted in Amazon’s Sydney AWS region.

The migration process itself was fully automated, making it a one-click migration process for each site. However, in the background over 100 individual tasks ran per migration. Before the official launch date, many trial migrations took place to streamline the process. This meant that all sites could easily be migrated over one weekend.

While the GovCMS 7 distribution (based on Drupal 7) was the primary focus, Salsa and amazee.io also built out the GovCMS 8 (Drupal 8) tools. These streams ran in parallel so that from launch day both distributions were available. Agencies can now choose between Drupal 7 and Drupal 8 for their SaaS sites to leverage the new features of Drupal 8 while still benefiting from a fully managed platform.

The new helpdesk

The new GovCMS also features a dedicated and unified helpdesk run by Salsa and amazee.io.

For the first three years of GovCMS, there was no dedicated GovCMS Helpdesk. Issues were logged via a general helpdesk that serviced lots of Drupal customers and then often re-assigned to the GovCMS team to action (e.g. password resets).

The new helpdesk was set up specifically for the new platform, with significant effort spent on a clear understanding of processes and communication requirements across multiple levels of support. It also delivers full transparency, because the GovCMS team at Finance can now see all the tickets in the system, as well as customer satisfaction survey results.

Once a ticket/issue is raised via phone or the online system, the ticket is triaged by a Salsa service desk agent and assigned to the best team/person. As a dedicated GovCMS helpdesk, we have customised the helpdesk to Finance’s needs and can make changes — such as changing a field in the ticket form, or altering the workflow process.

The helpdesk also includes automation. For example if a ticket gets raised as ‘critical’ outside of business hours, it automatically triggers a PagerDuty incident for the infrastructure team. This initiates a phone call to the team member on call at that time. The helpdesk has also been set up so that if we’re unable to pick up the phone, a ticket is automatically generated with the time of the call, the phone number, and a link to listen to the voicemail message.

The helpdesk performance is also monitored at the strategic level, with fortnightly meetings between the Service Desk Manager and the Head of GovCMS to discuss what’s going well, what’s not going well and how we can improve the customer experience. Frequent standups with the whole team (Salsa, Finance and amazee.io) ensure operational blockers or challenges can be resolved quickly.

Since going live with the new platform, customer feedback has been excellent, with 96% of respondents citing a positive helpdesk experience. Survey responses have also included comments on the faster service desk performance. This positive feedback demonstrates the effectiveness of planning, and a dedicated support team.

The benefits

The new Salsa/amazee.io platform has delivered many benefits to the Department of Finance and the GovCMS platform itself. Benefits include:

• Reduced costs over the long term

• Stronger collaboration with other government — Salsa/amazee.io brought the best of the Victorian government to GovCMS and vice versa through our relationships (the GovCMS platform model is built on Victoria’s Single Digital Presence platform model)

• Fully transparent pricing model plus elastic and consumption pricing

• New paradigms in application architecture — the emerging concept of SaaS+ means GovCMS will be able to introduce new features at reduced costs, with an improved developer experience (if it works locally it will work on the server)

• Fully open source technology stack for the first time

• Lower maintenance and costs by consolidating SaaS and PaaS, made possible by containerisation technology (For more information on containerisation see our blog What is containerisation?)

• Implementation of the latest technologies to do things differently — faster, better and cheaper — such as:

• Containerisation — See our blog What is containerisation? for the key benefits of containerisation.

• Automated scaling, which automatically increases or decreases the amount of resources needed to power the platform (this helps respond to traffic spikes or periods of low activity).

• Automated/self-healing for automated recovery of individual sites or, in the most extreme case, entire physical data centres.

• Automated forklifting (migrations) — Building a reusable migration framework allowed for rolling, ongoing migrations. By the time the final migrations occurred the process had already run thousands of times, giving greater confidence to the process and ensuring there was limited room for the unknown. The framework was also built to support PaaS migrations, meaning the barrier to migrating non-SaaS GovCMS sites to the new platform was lowered considerably.

• Continuous integration was built into the process and tooling from day one. Any new GovCMS project benefits from CI pipelines that perform automated testing, linting, code-standards checks, configuration management validation and more; providing further confidence that deployments function as expected. GovCMS 2.0 introduced automated testing on a per-site basis for the first time, giving agencies and developers further flexibility to enhance their development workflows.

• A dedicated, customised helpdesk to better meet GovCMS customer needs

• Faster turnaround for customers requiring helpdesk services (based on survey results) and extremely high customer satisfaction ratings based on service desk feedback

• More transparency for the Department of Finance, who can see all helpdesk tickets and know what their customers are requesting

Why Salsa Digital?

Finance chose Salsa/amazee.io as the implementation partner for the next-generation GovCMS based on our innovative tender response. Our technical proposal showed innovation that would deliver many benefits to GovCMS. And while it was a cutting-edge technical solution, it had been ‘tested’ in Victoria through the SDP build. In addition, the use of amazee.io’s Lagoon platform meant the whole stack could be open source for the first time.

About Salsa Digital

We’re an open source digital agency focused on helping governments and enterprises become more open, more connected and more consolidated. We’re committed to the open government movement and to the many benefits this ethos provides including transparency, innovation, and sharing and improving problems and patterns solved by the pioneers before us.

The open source movement has played a key role in the evolution of Salsa. Over the years, as technology and practices have developed, this commitment to open source has grown to encompass a much broader base beyond open source CMSs — open data, open platform and open design. We contribute and deliver services in all these areas, with a strong involvement in specific open source initiatives such as Drupal, Wordpress, GovCMS, Single Digital Presence, Kubernetes/Lagoon and CKAN.