RFC - Ripple Tide API module

[dylankelly]

Summary

Move Tide request functionality in ripple-nuxt-tide module to new module making use of Nuxt 3 server routes to expose an API. Backend for frontend pattern to allow more straightforward data flow in Nuxt sites and open the possibility of reuse in other projects.

Motivation

Currently all requests for Tide Drupal CMS data needed to render a page in a Ripple frontend are made by calling the Drupal resource directly from the frontend via a proxy request to /api endpoints. A typical request to render a landing page might include the following requests

• Request to sites taxonomy to fetch global site data and site menus
• Multiple requests to menu endpoint to fetch menu items
• Request to /route endpoint to fetch information about route resource
• Request to the route resource including which subresources required

Making these requests in the Nuxt rendering layer has several disadvantages

• Payload is overkill, Drupal entity has too much data in it, 80% of landing page JSON response data delivered to browser is not relevant to rendering page.
• Super long includes query string for relationships, currently dynamic page query has 4000+ characters, may run up against browser character limits for urls
• Hard to understand for developers without Drupal experience. Difficult to trace the data flow from BE to FE because it is written in non idiomatic way and requires intimate knowledge of our custom middleware implementation. New developers are hard to onboard because of this.
• Hard to version: upgrades to Tide need to have corresponding upgrades to the entire Nuxt rendering layer instead of just the data layer.

Nuxt 3 has a concept of server routes used for API data endpoints only, which allows you to easily create an endpoint to serve JSON data. This ticket proposes moving the current requests from the Nuxt context to this layer.

Moving these requests to a dedicated server endpoint would allow us to wrap all the logic for massaging data from Drupal JSON API to a usable page data model. This would be accessed from Nuxt pages in SSR asyncData hook by a HTTP GET request to the custom API endpoint eg /api/routes/nameofpage or some agreed routing schema.

Proposal

Create a POC detailing proposed solution. This approach has already been implemented in the Victoria Together project in https://github.com/dpc-sdp/virtual-vic-gov-au/tree/develop/modules/tide-server-api .

An early proof of concept PR outlines how this works here - dpc-sdp/ripple#1025

The proposed solution does the following:

• Adds new NPM module in Ripple Ripple-Tide-Api. This module does the following:
  • Exports a function to create an Express based server with endpoints which return page and site data from Tide.
  • A Nuxt module which
    • adds a wildcard route page component which calls the middleware API passing the current url route and passes rendering to a content type specific page component (which is defined in module config)
• Adds a new NPM module Ripple-Nuxt. This module is responsible for setting up the following
  • Adds ripple global JS and styles
  • Adds page layout
  • Sets up opinionated project defaults for linting
• Seperate NPM modules for content type functionality eg: ripple-tide-landing-page. Each content type module adds the following:
  • Field mapping for serverMiddleware for the content type
  • Page component for rendering the content type
  • Optional Swagger definition for the API response

Proposed module API for creating a new content type

export default {
  // import function for the component to render the page
  component: () => import('./path/to/component.vue'),
  mapping: {
    // This defines the fields that are returned in the serveMiddleware based
    // API response. Each key is the name of the field returned and value is a resolver function
    // which is passed the Tide API page data and returns the field data.
  },
  includes: [
  // Array of required include values to pass to Tide API request
  ]
}

Benefits

• A single place to reason about data in the FE. Moving to serverMiddleware means we can use Express for API routes which has proven idiomatic ways of handling RESTful API routes
• Multiple requests to BE to build data model for a page could be replaced with a single cacheable GET request, massive redundant JSON API payloads could be replaced with just the data we need reducing data served. The work to massage data from Drupal JSON api to useable data for Ripple API would then only performed once instead of on each request, reducing server load.
• We would be able to offer this API to 3rd party non Ripple website consumers (As was implemented in the Legislation project) as it would be human readable and would abstract over the Drupal implementation details.
• If we did decide in the future that we wanted to move this to a dedicated instance (say we wanted to have a single API server for all sites, which should technically be achievable), we could just plop this into its own node / express app.
• Even if we decided to move this to the PHP side, we would have done the hard work of defining the business logic needed and it would be just a matter of translation, our Nuxt rendering side would not need any modification other than pointing to the new endpoint.
• Offering only the API module would allow for faster development on projects which want to use Tide without Ripple
• Requests to API would be able to be mocked which make test automation easier to implement and allow for faster development as we wouldnt have to wait for a working Drupal BE to create new features in the FE.

Drawbacks

• Breaking change for Ripple Nuxt Tide
• Significant work to migrate existing sites to the new structure when there is significant custom work in Tide folder
• Another request which requires cache invalidation, mitigated by passing cache tags but still adds complexity

Adoption strategy

• New modules (ripple-nuxt and ripple-tide-api)
• Migration requirements for existing sites summarised here Migration plan
• Sites which only have theme modification (21) would be able to be migrated with automated process using codemods / ansible.