Add templating support with mj-var and mj-env tags

[mlacorte]

Is your feature request related to a problem? Please describe.

When I'm writing email templates, they usually require dynamic values to be inserted via ESP template code, but attempting to write naive templated MJML results in a variety of errors.

Problem 1 - It breaks syntax highlighting and throws editor errors.

Problem 2 - The CSS processing step sometimes alters the template code, potentially enough for the ESP to not render it properly.

Problem 3 - Quotation marks can cause issues with the generated HTML.

While MJML's mj-raw tag works nicely inside HTML blocks, it doesn't help us with inserting template code into HTML attributes or CSS blocks.

Additionally, a pre-processing step is necessary if you want to use dummy data while developing, otherwise the user's raw template tags are rendered in the browser, ranging from an 'ugly' to a 'broken' appearance.

This is solvable with custom processing pipelines, but that's beyond the ability of many designers to setup themselves, and MJML doesn't have an officially supported solution that designers can leverage.

Describe the solution you'd like

I have two solutions I'd like to propose. One is a partial fix that ensures MJML will always be able to parse and generate templated code, and the other is a fix that allows dummy data during development.

Solution 1 - Partial fix

Add an mj-var tag that allows the user to declare template %variables% which are recognized by the MJML parser and guaranteed not to be altered by any MJML processing steps.

Example variable declaration:

Example variable usage:

These variables should also be escapable via a \%backslash%.

By default, %variables% that have not been declared by an mj-var tag should be treated as if they were escaped for backwards compatibility with with templates that happen to have variable-looking code in them. The compiler should emit a warning for undeclared variable usage with the ability to opt-into a full error.

Solution 2 - Full fix

The mj-var tag solves the problem of parsing and generating templated code, but doesn't solve the issue of rendering during development.

For that, I propose adding an mj-env tag that only renders itself if the associated environment variable is set during the build step.

Example variable declaration with environment conditionals:

The code inside the DEV tag would show for local development, and PROD would show for the final build.

The combination of mj-var and mj-env provide an unopinionated solution to writing arbitrary templated code in MJML. Users won't have to setup custom pipelines to work with templates, the additional tags are backwards compatible with existing MJML code, and custom pre and post-processing pipelines will continue to work as before.

Describe alternatives you've considered

The other solution would be to go even further and pick a full-fledged templating language to be officially supported by the MJML compiler and associated tooling, but I agree with the conclusions you came to in issue #1630 that advanced templating solutions are best left to custom pre and post-processing pipelines, at least until the theoretical day where some templating language becomes broadly accepted as 'standard'.

Thank you for your work on MJML and taking the time to read this!

[iRyusa]

As we said multiple times, mjml can be plugged easily into any workflow so let's not reinvent the wheel and let other templating languages do their work.

Problem 1 - It breaks syntax highlighting and throws editor errors.

Those are linter errors so you'll free to ignore them if you know what you're doing. We can't tell the different versus a mispelled property & and var syntax for a non known templating lang

This is solvable with custom processing pipelines, but that's beyond the ability of many designers to setup themselves, and MJML doesn't have an officially supported solution that designers can leverage.

MJML team doesn't have enough bandwidth to support a "project" like that. A simple gulp workflow would work perfectly fine. In Rails you can already use Premailer to preview MJML template will dummy data for example.

Solution 1 - Partial fix / 2 full fix

This would bloat heavily an MJML file at the point where a document would be really hard to read. If we add var support then we have to implement for loop/if statement/ ... #1815 will even simplify the way you can hook with MJML as a developer where you can just alter the full XML string before being converted to the final JSON object

That being said, the issue you're describing with mj-style content being altered looks like to be an issue with beautifier ? If I check the minify option it looks like the style block render perfectly fines https://mjml.io/try-it-live/Sy2ObU74I let's open an issue on that so we can investigate.

I'm closing this one because I think discussion will conclude the same as #1630

[mlacorte]

Thanks for the consideration, even though we disagree! :)

I assure you that I'm deeply familiar with runaway configuration hell, and there's deep wisdom in restricting a grammar by deciding what it isn't, but that does beg the question, "What is and isn't this language for?"

I don't think MJML in its current incantation adequately answers the question, "How do I build a MailChimp template?", a question I should certainly hope falls in the scope of what MJML is.

MJML provides a truly excellent set of utilities to safely and confidently build email templates in the hostile world of twenty year old email clients, but at some point the typical MJML user will want to export *|BRAND:LOGO|* in their template code while still seeing a proper image during development.

The steps, according to your answer (and the steps that I took in my multiple MJML projects I might add), are as follows:

1. Research and decide on a secondary templating language to use for variable interpolation.
2. Learn enough JavaScript, NPM, and Gulp (or non-JS based alternative) to implement a custom pipeline through whatever template pre-processor you picked in step 1.
3. Disable the convenience features of the default MJML toolset
4. Build your MailChimp template

Reasonable minds can disagree about what the 'best' solution to this is, but I hope we're in agreement that this is a higher burden than the average email template designer would like to endure.

I'm sure you've hashed this out many times internally, so I understand your desire to brush past the thoughts of yet-another-GitHub complainer, but with respect, there are a number of replies you gave in your answer that are fundamentally incorrect.

We can't tell the different versus a mispelled property & and var syntax for a non known templating lang

You've implemented a compiler. A small compiler, but a compiler none-the-less. Tracking variable declaration is very much a solved problem, one that shouldn't introduce much complexity to MJML at all. I think you might have been overly hasty to include this point.

This would bloat heavily an MJML file at the point where a document would be really hard to read. If we add var support then we have to implement for loop/if statement

Again, with respect, MJML already supports the variable insertion of code via the css-class and mj-class attributes, so I disagree that the concept of adding an mj-var tag necessitates adding a full templating language.

And I've rarely found the introduction of 'variables' to introduce bloat. Quite the opposite actually.

A simple gulp workflow would work perfectly fine. In Rails you can already use Premailer to preview MJML template will dummy data for example.

You've described two programming ecosystems in this answer, neither of which I'd call 'simple', and both of which require a competent level of knowledge about a programming language and its associated toolsets to implement custom pre-processor pipelines.

That may not be a huge burden to people like you and me, but there are many non-expert programmers who also want to make custom email templates. This is a huge burden to the kinds of people who are savvy enough to tweak WordPress templates and use the *|FNAME|* merge tag in MailChimp, but otherwise don't know how to code.

MJML team doesn't have enough bandwidth to support a "project" like that.

You absolutely do! I proposed the addition of two tags that would solve the problem entirely. :)

No need to open an issue about the beautifier, I'm running "htmlbeautify" in my custom pre/post-processor pipeline which is probably the issue, so I'll make sure to look into that. I will also enthusiastically follow the progress of #1815!

I would still implore you to provide a better solution to the question, "How do I develop a MailChimp template with MJML?" I'm not deeply attached in any way to the solutions I proposed, and I would gladly accept alternatives, but "implement a custom pre-processor pipeline" is a disappointing answer coming from what I believe is the world's most robust email development framework.

Again, thank you for your consideration, and I'm sorry for being a burden!

[iRyusa]

I don't think MJML in its current incantation adequately answers the question, "How do I build a MailChimp template?", a question I should certainly hope falls in the scope of what MJML is.

It's your standpoint here, I feel we do what we can while keeping our vision of MJML : a simple markup language.

The steps, according to your answer (and the steps that I took in my multiple MJML projects I might add), are as follows: (...)

I don't get why this would be wrong, when I'm building an APP I take some time to ensure that I'm using the right tool for the right job. When you commit to a workflow then you'll use it for a while, it can still be improved so it's not a huge deal if you have to work a bit to have something working.

Everyone isn't a developer I think we all get it. But there's enough ressource over proven solutions like Gulp to have the feature you're asking for with a minimal effort. Even more workflows can be shared, and improved (and we're even on slack to help too :)).

You've implemented a compiler. A small compiler, but a compiler none-the-less. Tracking variable declaration is very much a solved problem, one that shouldn't introduce much complexity to MJML at all. I think you might have been overly hasty to include this point.

It starts with string replacement, then comes other data structures, then ... if we start opening this pandora's box then we'll have to implement the whole set of feature.

Implementing variable into core/parser means that we have a lot of questions to answer (where, at what stage, what drawbacks, ...) and those have already answered by multiple templating language.

You absolutely do! I proposed the addition of two tags that would solve the problem entirely. :)

I'm not so sure about this, those tags cannot be implemented without modifying how core+parser interact each other given the current code base. And let's face it, it solves a minimal part of the whole dynamic email issue.

Again, with respect, MJML already supports the variable insertion of code via the css-class and mj-class attributes, ...

You're taking some shortcuts here, this is not really a variable insertion here. mj-class is a convenient way to override default values for components. You can certainly hack mj-class :

b-company.mjml 
<mjml>
  <mj-head>
      <mj-attribute>
        <mj-class name="logo" src="./file.png" />
       </mj-attribute>
   </mj-head>
</mjml>
a-company.mjml 
<mjml>
  <mj-head>
      <mj-attribute>
        <mj-class name="logo" src="./file-2.png" />
       </mj-attribute>
   </mj-head>
</mjml>

And then mj-include them, but that's not really the purpose of it.

[mlacorte]

I don't get why this would be wrong, when I'm building an APP I take some time to ensure that I'm using the right tool for the right job.

I think this is the bulk of my frustration surrounding templating in MJML. I thought I was choosing the right tool for the job. MJML advertises itself as "The only framework that makes responsive email easy", yet adding merge tags resulted in code generation issues, compiler errors, and the need to implement a custom pre/post-processor pipeline without any official support from the MJML beyond the README file of gulp-mjml.

I disagree that the introduction of an mj-var type necessitates additional templating features, again, I think you've already half-implemented them with mj-class, but if it's a bridge too far for the MJML team, I can think of some alternatives.

The primary pain points I experienced were:

1. Syntax and compiler errors during development
2. Template tags being munged by the compiler when placed in HTML attributes and CSS blocks
3. No turnkey solution to export templatized code while using preview data during development

Points 1 and 2 are partially supported with mj-raw, but that isn't usable in CSS and data attributes. Perhaps a compiler-supported string literal syntax such as backticks?

<mjml>
  <mj-head>
    <mj-style>
      a {
        color: `{{ this_is_passed_literally }}`;
      }
    </mj-style>
  </mj-head>
  <mj-body width="`*|THIS:IS:PASSED:LITERALLY|*`">
    <mj-section>
      <mj-column>
        <mj-text>
          `alternative_to_mj-raw?`
        </mj-text>
      </mj-column>
    </mj-section>
  </mj-body>
</mjml>

Point 3 doesn't need to be officially supported by the MJML markup itself (though I think that would be preferable), but an officially supported/documented project template or mjml init command that would provide users a basic development workflow would be a much appreciated addition.

I do want to highlight one point of contention:

I feel we do what we can while keeping our vision of MJML : a simple markup language.

A simple markup language for what?

I've never seen any polling data, but I suspect >95% of MJML users build templates that use merge tags of some sort. If your goal is to solve the 5% use case you've done it, MJML works great, but as a user who develops templates for ESPs, it feels a bit like a car where you have to build your own steering wheel.

I completely support your decision to limit the scope of MJML's markup to do one thing and do it well. I'm just not sure the "one thing" you've optimized for solves the "one thing" your users want, which is to build email templates, and I don't think you need to throw in custom data types and for-loops to solve the majority use case.

[iRyusa]

I think this is the bulk of my frustration surrounding templating in MJML. I thought I was choosing the right tool for the job.

I mean we never advertised MJML with templating feature at the beginning: this wasn't a selling point from the very start. You should take it easy on those "errors" it's more a warning at the end (and that can be totally removed if the templating is applied first because the XML input won't have any templating attribute).

Template tags being munged by the compiler when placed in HTML attributes and CSS blocks

As said this part is most likely the lib we use for beautify the HTML output interpretting those var as valid CSS, as mentioned the minified output seems to be fine maybe it's a simple issue and that can be solved quite easily (https://github.com/beautify-web/js-beautify#ignore-directive) (again we need to identify that if you will to open an issue)

A simple markup language for what?

I think popularity of it speak for itself ? If it has such an high demand, there's a good chance that a developer would work on it. And they did : #1630 (comment) look at the list mentioned here every big framework have a way to plug MJML, and all of those aren't really bothered by this approach. And those "feature" wouldn't be used by those implementation because every framework has already it's own templating language (and as pointed, ESP does too)

If your goal is to solve the 5% use case you've done it, MJML works great, but as a user who develops templates for ESPs, it feels a bit like a car where you have to build your own steering wheel.

Mailjet totally leverage MJML for their WYSIWYG editor, you can also use their templating language on top of that and you can also test your email with fake data).

You just pull up data from your imagination and use them as a fact it won't works like that.

[mlacorte]

A fix to prevent the 'beautification' of mustache-like syntaxes inside CSS blocks would work for the "theme_color" example I provided, but what about the "alt_text" example?

A reduced version of it:

<mj-text font-family="{{ sans_serif_theme | default: "Times New Roman, Times, serif" }}">
  Some text
</mj-text>

There's no way to tell MJML "don't process this text", and exporting this as HTML would fail because of the quotation marks inside the template.

In order build a pipeline for arbitrary templated code, one must have a pre-processing step that removes the template code and replaces it with MJML-safe substitutions, run the MJML compiler, and then re-substitute the template code in a post-processing step. This is the solution I have employed for my own projects, but it required a half-dozen Gulp plugins, a ~60 line hand-written Gulpfile (some of which was for hot-reload, local assets, etc), and the willingness to perpetually endure a lack of linting and broken syntax highlighting. "Just use Mailjet's proprietary online editor" is a potential solution, but it's very disappointing that this is the best solution the MJML team can can come up with beyond "turn off syntax highlighting and linting".

Neither you nor I know the exact breakdown of MJML users who develop HTML emails with merge tags, but I suspect it's a substantive majority of them, and I suspect that despite the inconclusive data, you likely agree.

Again, to summarize the official position of the MJML team: As a user who wants to develop an email template with merge tags and see dummy data while I'm developing it, the correct course of action is to:

1. Research and decide on a secondary templating language to use for variable interpolation.
2. Learn enough JavaScript, NPM, and Gulp (or non-JS based alternative) to implement a custom pipeline through whatever template pre-processor you picked in step 1.
3. Disable the convenience features of the default MJML toolset
4. Build your template

The pre-processing step may or may not need an additional post-processing step depending on if your templating language happens to be compatible with MJML's syntax, and if I want to enjoy development features like syntax highlighting and linting I should use Mailjet's proprietary editor or develop a custom editor plugin that supports my handcrafted MJML + templating language pipeline.

You could solve both of these problems with even a simple "ignore this text" capability like the 'backtick' proposal I made. That's not an absurd request, and it would greatly reduce the need to write custom pre and post-processors for the simple use case of compiling templated code.

I want to emphasize how useful MJML has been to me, and I have serious respect for the quality of work the MJML team has put into this project, but please don't act like I'm being unreasonable for being disappointed at the sheer number of hoops I've had to jump through to create an efficient development workflow for developing email templates. I expect and welcome criticism to my proposed solutions and will work with you to find the best path forward, but it's obstinate to act like this friction is rare, inconsequential, or the result of overly inflated expectations. Developing a MailChimp template is about as reasonable and common a use case as I can imagine.

[iRyusa]

There's no way we'll be able to parse this, even with some back tick or what ever you want.

<mj-text font-family="{{ sans_serif_theme | default: "Times New Roman, Times, serif" }}">
  Some text
</mj-text>

MJML markup is still at the end a HTML/XML-like file, so the input has to be valid.

As of today, custom attributes #200 is far more important than adding a pseudo var support for not so much added value as you can just grab any templating and plug it.

Neither you nor I know the exact breakdown of MJML users who develop HTML emails with merge tags, but I suspect it's a substantive majority of them, and I suspect that despite the inconclusive data, you likely agree.

I mean, i'm on the support slack 24/24 & 7/7 for 4 years now? So...

Most of templating support single & double quote. I mean you're complaining about 60 line of gulp file that does everything you want. If not everyone can do then, do a blog post about your workflow, open source it and share it to other people.

Disable the convenience features of the default MJML toolset

You clearly over-react here, so there's not so much to say, let's leave this discussion here.