Morphir File based layout

[DamianReeves]

What would the structure of the file based morphir look like:

Distribution

{
   "packageName":"Root.GrandParent.Parent.Local"
}

Module

{
   "doc": "",
   "exposes": {
     "types":[]
   }
}

 Encoding distribution:
PackageName of Distribution:
distribution.json : contains PackageName folder

Module :
module.json

[AttilaMihaly]

I assume these are just some notes from our meeting, but we can use this discussion to work out the details.

From our discussions it sounded like one of the most important benefits we are hoping to get out of the new format is the ability to break the IR up into smaller files. The benefit of this would be that some of the IR's structure would be represented by a directory/file structure which is easier to navigate without specialized JSON tooling.

This would be very beneficial but it comes at the price of having to define both a directory layout and a JSON format for the file contents and the two should form a cohesive specification. Also, the Morphir tools will need to now navigate the file system as well as the contents of the JSON so they become more complex compared to the current state of just having to deal with a single JSON file.

As we can see, both approaches have benefits, so I started to think about how we could combine them, and I think there's a way to get the best of both worlds.

To demonstrate the approach let's start with a sample JSON file:

// main.json
{
  "foo": {
    "foo child 1": [ "a", "b", "c" ],
    "foo child 2": {
      "width": 12,
      "height": 34
    }
  },
  "bar": {
    "baz": {},
    "bat": 123
  }
}

What would be the easiest way of breaking this up into files? One simple approach would be this:

// main
// - foo.json
    {
      "foo child 1": [ "a", "b", "c" ],
      "foo child 2": {
        "width": 12,
        "height": 34
      }
    }
// - bar.json
    {
      "baz": {},
      "bat": 123
    }

Or you can go even further and break up foo.json as well:

// main
// - foo
//   - foo child 1.json
        [ "a", "b", "c" ]
//   - foo child 2.json 
        {
          "width": 12,
          "height": 34
        }
// - bar.json
    {
      "baz": {},
      "bat": 123
    }

The rules that I applied are simple:

• To break a JSON object up into files you should:
  • turn the file into a directory
  • take its fields and create a JSON file with the same name as the field
  • take the value of the field and use it as the content of the file
• You can do this with any JSON files where the root node is an object

With this approach we could have the best of both worlds:

• We could define just one JSON format for the whole structure.
• The structure of the JSON implicitly defines a directory/file structure as well.
• There is flexibility in how much you break up the files. This could be based on convenience or other factors.
• We could easily create a library that makes the file layout transparent to consumer applications and present the whole thing as a single JSON.

@stephengoldbaum, @DamianReeves, what do you think?

[AttilaMihaly]

One thing I should clarify is that this approach would not work with the current Morphir IR JSON format as we barely use JSON objects so the implicit assumption is that we come up with an easier to read format that primarily uses objects. Which is another reason why we might want to choose the object based representation for tagged unions.

[stephengoldbaum]

Looks sensible. Where would we draw the line on granularity? And where do you put the information associated with the folder? Using a simplified IR example, given:

• Package X
  • Module Y
    • Type A : Int
    • Type B : Record { f1 : Int, f2 : String}
    • Function AtoB

Which would be folders vs files and where would we put the module information for Module Y?

[DamianReeves]

I think if we adopted @context.json as the name of a context file in each folder, it would help. The @context concept I think could potentially work similar to @context in JSON-LD.

In this case what I am thinking is that the context could by default only require a context_type (or perhaps just kind) property, that says whether the folder represents the context for a package, module, or type, for example:

{ "kind": "package"}

[AttilaMihaly]

One of the benefits of this approach is that we would not have to decide where the format is split up into files.

Where we put module information depends on how we decide to represent a module. For example, if you represent modules something like this:

{
  "Foo Module": {
    "types": ...,
    "values": ...,
    "docs": ....
  }
}

Then we could have a Foo Module folder with 3 files in it: types.json, values.json and docs.json

[AttilaMihaly]

You wouldn't need a special way to identify what type of thing the folder represents. If we use the tagged union encoding we discussed in the other thread, then we have a standard already.

Say you need to identify if a folder represents a record type or a union. If the encoding is { "union": ... }, then turning that into files you get union.json. So you just have to check the name of the single file inside the folder, just like you would check the single field of the object if it was all in one JSON.