Dissecting the Single Digital Presence platform

What is SDP?

Single Digital Presence (SDP) is the new open platform created to make it easier for citizens to find, understand and use Victorian Government information. It also makes it faster, easier, and cheaper for Victorian Government agencies to build new sites or migrate existing ones when they choose to leverage the platform and its benefits.

A Department of Premier and Cabinet (DPC) initiative, SDP consists of three components — Bay (an open-source platform based on Lagoon), Tide (a Drupal CMS distribution) and Ripple (the frontend presentation layer). The project went to tender last year and Salsa won the job...and we’ve been pretty busy helping to bring DPC’s vision to life!

Below is a deep technical overview of all three products of the SDP. If you’d like to read more about why DPC wanted one platform and the benefits it delivers to citizens and the Victorian Government, please read our blog on the SDP value proposition.

What is Bay?

Bay is the platform itself, and forms the infrastructure layer on which SDP has been built. Salsa and our partner amazee.io built Bay using open-source platform, Lagoon. This cloud-based solution allows Victorian government agencies to build, test and deliver websites via the cloud.

Building Bay

Key elements of the solution include continuous integration, containerisation and a decoupled Drupal component.

Cluster setup

amazee.io was engaged to carry out the OpenShift cluster setup in the Amazon Web Services (AWS) Sydney region. The cluster was set up in a highly available configuration spread across all three availability zones for redundancy. This redundancy allows the cluster to withstand losing two of the three zones and still maintain service availability.

Lagoon was installed inside the cluster and used for continuous delivery of all projects linked to the platform. Any new branch created in a source repository automatically deploys the branch to OpenShift.

A load and penetration testing phase took place to ensure all services were prepared for production workloads before any sites were migrated to the new platform.

Scaling

All production sites are configured with a horizontal pod autoscaler to scale on demand, which allows production sites to make use of additional cluster resources as required.

Local development

Lagoon includes support for a local development environment based on Docker Compose. This allows developers to setup an exact production clone of the production platform because it’s reusing the same base images.

Continuous integration

Circle CI was chosen for continuous integration, which was also configured to build out the site on every commit. Layer caching meant that build times were minimal and a full regression test took around five minutes including the build of the whole project. Behat and code sniffs were run on this clone, which again was an exact clone of the production environment.

Development workflow

The new development workflow was able to leverage test environments for every branch, which meant the QA team had an isolated environment for every feature. It also gave the ability to do rapid prototyping as new features/modules can be easily spun up in a new environment for review.

Enhancements

As part of the project we were also able to make some enhancements specifically for DPC, these included:

• Custom images — Building a set of custom images that allows DPC to specifically control packages and dependencies across all projects. Images are rebuilt and scanned daily by Clair, which is an open source project for the static analysis of vulnerabilities in application containers. Additional packages such as clamAV are included in the PHP image to allow Drupal to scan for vulnerabilities during file upload.

• Metrics — Prometheus and Grafana were included to provide a granular level reporting of metrics across the cluster.

• Automated Composer updates — A nightly workflow was created to check for Composer package updates nightly and create a PR on the project. This also triggers the CI pipeline to ensure updates have passed all regression tests.

Benefits of Bay

Bay delivers the following benefits:

• A faster web experience for citizens

• Cost-effective at scale

• A secure platform

• Containerisation — Containers serve a similar function to virtual machines (VMs), however instead of hardware virtualisation (like VMs) containerisation shares the host’s operating system. Some of the containerisation benefits include:

• More portable (than VMs)

• Faster and more efficient (than VMs)

• Greater flexibility

• Reproducibility

• Simple packaging format

• Rapid and consistent deployment of workloads

• Can reduce the baseline

• Robust runtime environment for scaling and self-healing

• Standard management interface

• Horizontal pod autoscalers to handle spikes in traffic

• An automated testing and deployment continuous integration (CI) pipeline to standardise project development workflows, decrease regressions and reduce costs of manual, error-prone deployments.

What is Tide?

Tide is the Drupal distribution that manages content. It’s a customised build of Drupal 8 and forms a central repository where all content can be managed. Importantly, Tide allows content sharing — one source of content that can then be ‘served’ to different sites. It’s also part of a ‘headless’ Drupal, so this CMS doesn’t control the look and feel of the websites (see below, ‘What is Ripple?’ for more information on the presentation layer).

The Tide solution uses:

• An API-first headless distribution
• Multi-site content distribution
• Pick-and-mix features
• Loose coupling between modules
• Tested configuration and business logic
• Extensibility
• Relies on the core and contrib code as much as possible

API-first headless

We identified that our content clients may be written using different front-end frameworks, therefore we decided to standardise the REST API request/response format. We chose JSON API format as it’s one of the most defined standards for REST web services. It also supports querying for entities and collections, which makes the data exposure easier to maintain — there’s no need to create Drupal views or other ways of data representation on the Drupal side, because clients can do that on their own.

Multi-site content distribution

Tide is built with a multi-site configuration in mind. This means that each content or media piece can be assigned to appear in the dedicated site. To resolve an issue with duplicate content across multiple sites (how search engine sees it), a ‘Primary Site’ field was added. Clients must implement “<link rel="canonical" href="primary-site-url">” tag to make sure that search engines do not mark the content as duplicated.

Pick-and-mix features

To allow the ‘pick-and-mix’ features approach, we needed a system that ensured all modules integrated well with each other after first release and a mechanism to update specific features without making an update to the whole distribution. Composer supports hierarchical dependencies and version resolution. This means that each Tide module can be updated separately, increasing version numbers, and making sure that updates work with dependent modules rather than the whole profile.

Loose coupling between modules

Tide provides a list of modules to cover distribution features. To provide the ‘pick-and-mix’ option, we needed to design and track how configuration is stored in modules.

We split modules in three categories:

1. Core — A set of base configuration and business logic. It handles roles, text formats and other basic configuration.

2. Content types — Using a module-per-content type rule, we extracted configuration for each content type in its own module, and added tests and business logic as required.

3. Optional enhancement modules —These modules provide some additional enhancements, like API configuration or multi-site content distribution.

Tested configuration

It’s important to note that one of our goals was to rely on the functionality of core and contrib modules as much as possible to avoid the burden of custom code maintenance. So we decided to test configuration captured in Tide modules using a suite of simple, but thorough, Behat tests. They assert that configuration changes captured in the module indeed apply. For some modules, Behat tests are used to test custom business logic. We also use a PHPunit test to test services that implement custom business logic.

Benefits of Tide

Tide delivers the following benefits:

• An easy-to-use content management system (CMS)

• An up-to-date CMS, which minimises security risks

• A stable CMS—every change goes through rigorous manual review and automated build process.

• Flexible functionality

• One content repository, with the ability to ‘serve’ content to multiple sites/channels

• Centralised feature governance—DPC has a dedicated team that manages the backlog of features to cover the majority of CMS use cases.

Tide is also Composer-based (which means all the modern development tools are available out-of-the-box as dependencies) and compatible with Bay, the open-source container-based solution powered by Docker, Kubernetes, and Lagoon.

What is Ripple?

Ripple is the component library (called the Atomic Pattern Library) to control the look and feel of agency websites, and the framework and code to ‘render’ those components onto webpages. Ripple uses Vue.js and Nuxt to deliver a consistent look and feel across government websites, which increases usability.

The system is a decoupled solution, which means the content repository (Tide) is separate from the website’s frontend (Ripple). This decoupling presents many options for the future — for example, the content could be served to another channel, such as voice interface or even augmented reality.

Ripple will deliver big benefits to Victorian government agencies who choose to leverage SDP. This library of frontend components includes a starter kit so government agencies (or their website vendor) can build their website’s frontend quickly and easily. All elements have also been through user testing. This significantly reduces the time and costs that agencies need to invest to get a website up and running.

Benefits of Ripple

Ripple delivers the following benefits:

• Consistent site design to increase overall usability and make it easier for citizens to find information across sites

• A library of reusable components to increase knowledge sharing and reduce costs across the Victorian Government

• A starter kit to reduce the barrier for agencies that want to bring their site onto the SDP

• Better performance (the Salsa-customised interface delivers better performance than the standard Drupal theme)

• Components informed by user testing to maximise the citizen experience

• The ability to use the library of components on websites that aren’t on the SDP platform, because the use of Nuxt to render the visual components means that as long as other sites use Vue.JS they can access the components

Coming together

SDP design brief FULL ASSEMBLY.png

The three products in the SDP suite — Bay, Tide and Ripple — come together to deliver a cohesive platform that offers many benefits to the Victorian Government agencies who leverage SDP. It delivers big benefits to citizens and the Victoria Government and Salsa was proud to partner with DPC to realise this vision.

Read more about Single Digital Presence

Single Digital Presence (SDP) is a total digital solution, a digital landscape the Victorian Government can use to create a unified online presence. Below are more related Salsa publications covering everything from the value proposition to technical case studies:

• Creating a single digital presence for our citizens— an overview of the process, from tender to launch.

• SDP - delivering value— key insights into the value proposition for citizens, the Victorian Government and content authors

• Case study: Bay, the open platform — Find out about Victoria’s new open source platform

• Case study: Tide, the content distribution— Find out about Victoria’s new open source CMS

• Case study: Ripple, the presentation layer  — Find out about Victoria’s new presentation layer and Atomic LIbrary of reusable design elements

• Case study: Department of Treasury and Finance website — SDP pilot site

• Case study: Governor of Victoria website — SDP pilot site

Read Our Media Release