Proposal on Decorations Processing

[KindsonTheGenius]

Backend Decorations Processing

This document provides a proposal on backend decorations processing in Morphir and follows from Proposal on How to Handle Decorations

What is Backend Decorations Processing?

In the context of this document, it means use of selected decoration(s) optionally for code generation by the current backend.
For example backend decorations processing

• for the json-schema-enabled decorator for the Json Schema backend means ‘invoke the Json Schema backend based on the types that has been decorated with json-schema-enabled decoration’
• for the precision decorator for the Spark backend means ‘generate spark code with support for valid DecimalType with precision”
• for the min-length and max-length decorators for the the Json Schema backend means “the generated schema would support these constraints for strings. In other words, extends the generated schema to support these features”
• for the manifest decoration applied to the manifest generator backend means “manifest files would be generated based on types (records) decorated with manifest decoration”

Current Fundamentals and Assumptions

• Decorations are defined as a Morphir model containing only types
• The decoration model is converted to a Morphir and available to the Morphir Web
• The Morphir web is used to add/remove decorations to/from nodes in the main Morphir IR
• The mapping between decorations and the Main IR is maintained in an editable file we refer to as the ‘decorations dictionary’
• The decorations dictionary is keyed by the NodeId (String) with the value as the serialized value of the decoration applied to the NodeId
• We currently have a number of backends which require the use of decorations
• Backend code generation are invoked via a cli command. For example morphir json-schema-gen
• We would maintain the use of cli commands for backend code generation

Objectives

• Make the backend generate code based on the decorations. For example, running the command morphir json-schema-gen should optionally generate code based on the types decorated with json-schema-enabled decoration
• Use of decorations in the backend should be optional
• Backend code generation would still be done using cli commands
• This should be achieve with minimal code change

Approach

• Provide a way to specify use of decoration the backend code generation
  • Provide a way to enumerate all the decorations that apply to the given backend
  • Provide a way to specify which decorations to use
• For usability, we should provide backend information to the decorations
• Generate code based on the user selection

Recommendations

There should be a particular standard structure for decorations:
A record type with three fields:

• field 1(id) – unique id of the decoration
• field 2(data) – decoration data
• field 3(metadata) – decoration metadata

For example

{
	id: String,
	data: Boolean,
	metadata: Maybe String -- additional relevant data to this decoration 
}

Benefits of this Approach
We have a high level of flexibility in two areas:

• we retain all existing established decoration creation/management workflow, but we now have more options in terms of scalability and aligning to various kinds of backend processing requirements.
• it also maintains the possible option of editing imported decorations mentioned in Proposal on How to Handle Decorations. So if is accepted that decorations can be imported and edited, then the Morphir author could modify the imported decoration to fit his requirements

The metadata field could be more complex data structure holding additional processing-related data, but for this iteration, it would be a string containing information about the backend it applies to
At the backend, we have access to the decorations dictionary and decorations package (if package management is finally implemented). The package would provide us the type information about the decoration for derialization as needed.

This way, the backend would only retrieve only decorations that apply to the specific backend.

Decoration Boundaries

Decoration boundaries refer to constraints or restriction that would necessarily need to be enforced as it relates to decoration. There are three types (maybe more) of boundaries to consider:

1. Decoration Creation boundaries – how decorations should be created. Examples are:
   • Decorations cannot be modeled as Morphir values (functions)
   • Decoration model must be a record type
   • You cannot have two decorations with same local name
2. Decoration Application boundaries – how decorations are applied. Examples are:
   • the json-schema-enabled decoration cannot be applied to a value
   • the interface decoration should apply only to record types
   • packages cannot be decorated
3. Decoration Processing boundaries
   • decorations should not cascade to nested nodes/objects?
   • decorations should not be altered at the backend