Launching Bay, Victoria’s open platform

The players

Salsa Digital worked with the Department of Premier and Cabinet (DPC) to bring DPC’s vision of a Single Digital Presence (SDP) for Victoria to life. DPC is responsible for several elements of Victoria’s digital engagement, including vic.gov.au, data.vic.gov.au and engage.vic.gov.au. (The Engage website was recently recognised as one of only two finalists in the global GovCX Awards — read our blog on the Awards for more information.)

The problem

DPC knew that there were significant problems with the way websites had been built across the Victorian Government, with most sites being built on an ad-hoc basis. The result was over 900 vic.gov.au domains (plus an unknown number of sites on .com, com.au, etc. domains), built using different content management systems (CMSs) and on different platforms. DPC described this fragmentation as a ‘technology debt’ with significant implications on costs, efficiencies and cybersecurity.

DPC knew the Victorian Government needed a unified approach, a whole-of-government digital platform, and put the Single Digital Presence (SDP) project out to tender.

DPC identified two requirements:

1. A single site where Victorians could go to get information — a site that was managed by one content management system (CMS).

2. A common platform for hosting and management of the sites.

The new, single whole-of-government digital platform also had to:

• Be robust, resilient and secure
• Be based on a public cloud
• Use an elastic pricing model
• Be open source

The solution

Once Salsa won the job, we kicked the project off with a thorough discovery process to review options for the solution. In terms of the platform, instead of building a customised platform for DPC, Salsa recommended Lagoon, an open source, Docker build-and-deploy system for OpenShift and Kubernetes.

Using an open source platform also meant we could deliver a fully open source solution, something that was impossible until recently (see our blog on The future of open platform). This cloud-based solution allows Victorian Government agencies to build, test and deliver websites via the cloud.

Key elements of the solution included continuous integration, containerisation and a decoupled Drupal component.

Background: SDP consists of three main products — Bay (an open-source platform based on Lagoon), Tide (a headless/decoupled Drupal 8 CMS distribution) and Ripple coupled with Nuxt (the frontend presentation layer).

We’re going to start to get technical here...if you don’t want all the tech stuff, jump ahead to the benefits.

Cluster setup

amazee.io was engaged to carry out the OpenShift cluster setup in the Amazon Web Services (AWS) Sydney region. The cluster was setup in a highly available configuration spread across all three availability zones for redundancy. This redundancy allows the cluster to withstand losing two of the three zones and still maintain service availability.

Lagoon was installed inside the cluster and used for continuous delivery of all projects linked to the platform. Any new branch created in a source repository automatically deploys the branch to OpenShift.

A load and penetration testing phase took place to ensure all services were prepared for production workloads before any sites were migrated to the new platform.

Scaling

All production sites are configured with a horizontal pod autoscaler to scale on demand, which allows production sites to make use of additional cluster resources as required.

Local development

Lagoon includes support for a local development environment based on Docker Compose. This allows developers to setup an exact production clone of the production platform because it’s reusing the same base images.;

Continuous integration

Circle CI was chosen for continuous integration, which was also configured to build out the site on every commit. Layer caching meant that build times were minimal and a full regression test took around five minutes including the build of the whole project. Behat and code sniffs were run on this clone, which again was an exact clone of the production environment.

Development workflow

The new development workflow was able to leverage test environments for every branch, which meant the QA team had an isolated environment for every feature. It also gave the ability to do rapid prototyping as new features/modules can be easily spun up in a new environment for review.

Enhancements

As part of the project we were also able to make some enhancements specifically for DPC, these included:

• Custom images — Building a set of custom images that allows DPC to specifically control packages and dependencies across all projects. Images are rebuilt and scanned daily by Clair, which is an open source project for the static analysis of vulnerabilities in application containers. Additional packages such as clamAV are included in the PHP image to allow Drupal to scan for vulnerabilities during file upload.

• Metrics — Prometheus and Grafana were included to provide a granular level reporting of metrics across the cluster.

• Automated Composer updates — A nightly workflow was created to check for Composer package updates nightly and create a PR on the project. This also triggers the CI pipeline to ensure updates have passed all regression tests.

The benefits

The benefits of Bay can be broken into the overall benefits of the SDP, and benefits of the open platform, Bay.

SDP design brief BAY.png

As part of the bigger picture, SDP delivers:

• A more citizen-centric offering, helping Victorians find government information more easily

• Easier content management via one CMS

• A truly open source, open platform solution

• Access to the Drupal community’s contributions to continue to grow SDP

• Responsive sites, so websites can be viewed on desktops, tablets and smartphones

• Accessible sites (built-in WCAG2.0AA compliance tools)

• A secure environment

The benefits specific to the Bay (platform) product are:

• A faster web experience for citizens

• Cost-effective at scale

• A secure platform

• Containerisation — Containers serve a similar function to virtual machines (VMs), however instead of hardware virtualisation (like VMs) containerisation shares the host’s operating system. Some of the containerisation benefits include:

• More portable (than VMs)

• Faster and more efficient (than VMs)

• Greater flexibility

• Reproducibility

• Simple packaging format

• Rapid and consistent deployment of workloads

• Can reduce the baseline

• Robust runtime environment for scaling and self-healing

• Standard management interface

• Horizontal pod autoscalers to handle spikes in traffic

• An automated testing and deployment continuous integration (CI) pipeline to standardise project development workflows, decrease regressions and reduce costs of manual, error-prone deployments.

Amber Benjafield, Senior Project and Engagement Manager at the DPC (and the SDP project manager), talked about some of the benefits of a single platform: “SDP provides value to the government in cost savings, by building a single platform that can be used widely but supported centrally,” Amber said. “It’s about all the benefits that you get when you consolidate to a single platform, such as only fixing breakages and security risks once, only maintaining one platform, and building in new features that everyone can use but only paying for them once. Everyone who leverages SDP will benefit once the foundations are laid because all of the future investment will be on continuous improvement...not just maintaining the current, but improving.” 

Why Salsa Digital?

Salsa won the SDP tender in 2017 on the strength of our response, which not only demonstrated our ability to build a distribution to meet their requirements but also showed innovative thinking around the open platform (which then became Bay).

“Salsa has been a true partner. They’ve shared the vision for open government solutions and they’ve communicated with authenticity, and frequently. They have a highly skilled workforce and also know when to bring in external consultants. They haven’t done what’s comfortable and familiar they’ve stretched their own innovation.”  Amber Benjafield, Senior Project and Engagement Manager, DPC

About Salsa Digital

Salsa is a highly specialised, enterprise-grade digital agency focused on open source for social good and innovation. We’re committed to the open government movement and to the many benefits this ethos provides including transparency, innovation, and sharing and improving problems and patterns solved by the pioneers before us.

The open source movement has played a key role in the evolution of Salsa. Over the years, as technology and practices have developed, this commitment to open source has grown to encompass a much broader base beyond open source content management systems (open CMS) — open data, open platform and open design. We contribute and deliver services in all these areas, with a strong involvement in specific open source initiatives such as Drupal, Wordpress, GovCMS, Single Digital Presence, Kubernetes/Lagoon and CKAN.

Read more about Single Digital Presence

SDP is a total digital solution, a digital landscape the Victorian Government can use to create a unified online presence. Below are more related Salsa publications covering everything from the value proposition to technical case studies:

• Creating a single digital presence for our citizens— an overview of the process, from tender to launch.

• SDP - delivering value— key insights into the value proposition for citizens, the Victorian Government and content authors

• Dissecting the SDP— a deep technical analysis of the three products within the SDP - Bay, Tide and Ripple

• Case study: Tide, the content distribution— Find out about Victoria’s new open source CMS

• Case study: Ripple, the presentation layer  — Find out about Victoria’s new presentation layer and Atomic LIbrary of reusable design elements

• Case study: Department of Treasury and Finance website — SDP pilot site

• Case study: Governor of Victoria website — SDP pilot site

Read Our Media Release