Simplification of the content of component folders

[ThibFrgsGmz]

Hello everyone,

What do you think about a homogeneous update of the content of the component folders?

Some information in the SDD.md or README of the components is completely out of sync with the content of the component and its FPP model. Some of these files have not evolved for 6 years for example.

Personally, I have always wondered about the relevance of the content of some sdd.md or README files.
They contain redundant information compared to the FPP model which is the source of truth. Keeping them synchronized represents a significant additional maintenance effort. When comparing the SDD.md and the models, I wonder about the legitimacy of the model and the source code of the components.

Wouldn't it be better to try to keep it as simple as possible and to include in these files only the information that cannot be contained in the model? For example, the functional description, the use cases, the explanations about the design, the limitations...

Let's take the example of the ActiveLogger component:

There is a README file (https://github.com/nasa/fprime/blob/master/Svc/ActiveLogger/README) that is no longer up to date and whose presence leads us to wonder.

Concerning the content of its docs/ folder, there are Excel files that are difficult to manage in version management.
For its SDD (https://github.com/nasa/fprime/blob/master/Svc/ActiveLogger/docs/sdd.md): it contains a "Requirements" section which is not presented in all components (like FileManager: https://github.com/nasa/fprime/blob/master/Svc/FileManager/docs/sdd.md°).
The "Ports" section is not synchronized with what is written in the FPP model.

If we also take the https://github.com/nasa/fprime/tree/master/Svc/FileManager component, not to mention the fact that its SDD is almost empty, we see a Model/ folder whose contents I understand to be a relic of the past.

Personally, I would keep it as simple as possible to make it as easy to maintain as possible:

• A README.md file that would be the old SDD.md and only contain a functional description. (Named README.md to get the GitHub preview when browsing the component folder).
• An FPP model file
• A pair of .cpp/.h files for the implementation class of the component
• A .test.cpp file for the component unit tests.

Depending on the complexity of the component, we can classify the files into folders. For example,

• a model/ folder for all the FPP files,
• a src/ file for all the implementation files and finally
• a test/ folder for all the unit tests (without creating a ut/ sub-folder).

[LeStarch]

We need to come to a decision as to what these SDD.md files should include. Some customers want full SDDs that detail everything, model algorithm, etc. Some want simpler representations.

In general, I prefer a flatter structure for the actual source (.fpp, .cpp, .hpp), and I'd remove all README.md files. Tests come in two flavors: test/ut and test/int so I'd keep that structure.

I welcome input from the larger community as to their needs.

[ThibFrgsGmz]

Unless I'm mistaken, I haven't seen any components with test/int yet. Do you have any examples of the content of these tests in the components?

From what I understood the integration tests are only done at the deployment level in Ref/test/int for example (which by the way could just be Ref/test).

[SterlingPeet]

I personally see the SDD and Readme as redundant. I don’t like having the main component folder without a readme because then it has no landing page, which is why I’d prefer the readme to be the SDD. The docs folder is still useful to contain assets (like images or diagrams) that are referenced by the SDD/Readme.

[SterlingPeet]

I also think that a number of things in the Readme can be autogenerated, but not everything. How to keep this in sync is definitely a question. We have used an in internal tool to generate most of the docs from a component’s Ai.xml file, but it was never complete, and certainly couldn’t describe implemented algorithms.

[timcanham]

I agree on the SDD/Readme being redundant. We shouldn't keep them both. I wouldn't want to get rid of the SDD document, however, since that means a specific thing to the JPL software development process, and we need to point to it. Could we say "see docs/sdd.md for design details" or something like that in the README? I also fervently agree that sections that describe ports, commands, etc, should be generated and that comments describing them should be put in the FPP and copied to the generated output. The old pre-cmake build system had targets like that, and it would be nice to get that back.

[EbenezerA99]

As someone who has used fprime for a few years in the context of university level CubeSat projects , I am in favor of more detailed SDD’s. I and the rest of my team find them extremely helpful for understanding the use cases and implementation of various components. However, this is only helpful when the documentation is updated and accurate, which, as @ThibFrgsGmz mentioned, is not always the case!

[ThibFrgsGmz]

In my experience, the more detailed a document is, the harder it is to maintain over time. That's what we're seeing here. 😸

That's why it's absolutely necessary to avoid redundant information and aim for a single source of truth. But it is also important that the information is as close to the source code as possible because it is the source code that is considered by the compiler - not comments or documents.

I'd much rather have a nice docstring in the component's .h than in a file on the side. But I agree that having nice Mermaid diagrams or a markdown document is more pleasant to read. That's why I would have put in the .md document only an overview, and sequence diagrams (in Mermaid) but without mentioning any design or implementation details, that's the role of the FPP and the source code.

Maybe one day, with AI, we will be able to generate source code (usable and without retouching) from a document and it would then be interesting to have much more information. 😆

[bocchino]

For component SDDs, tables of ports and dictionary entries (commands, channels, etc.) can be easily auto-generated from the FPP model. We are currently working on a tool called fpp-to-json that dumps the FPP model to JSON. Hopefully this tool will be available by the end of the summer. Once this is done, it should be straightforward to write a Python tool to convert the JSON model elements to Markdown tables.

[bocchino]

This addresses Tim's comment:

I also fervently agree that sections that describe ports, commands, etc, should be generated and that comments describing them should be put in the FPP and copied to the generated output. The old pre-cmake build system had targets like that, and it would be nice to get that back.