Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance: use html parser in preprocessor #240

Closed
wants to merge 17 commits into from
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 1 addition & 8 deletions src/partialHydration/__tests__/inlineSvelteComponent.spec.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,4 @@
import { inlinePreprocessedSvelteComponent, escapeHtml, inlineSvelteComponent } from '../inlineSvelteComponent';

test('#escapeHtml', () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

escapeHtml and replaceSpecialCharacters are moved to htmlParser.ts.

expect(escapeHtml('')).toEqual('');
expect(escapeHtml(`<html>'Tom'&amp;"Jerry"</html>`)).toEqual(
'&lt;html&gt;&#039;Tom&#039;&amp;amp;&quot;Jerry&quot;&lt;/html&gt;',
);
});
import { inlinePreprocessedSvelteComponent, inlineSvelteComponent } from '../inlineSvelteComponent';

test('#inlinePreprocessedSvelteComponent', () => {
const options = '{"loading":"lazy"}';
Expand Down
10 changes: 0 additions & 10 deletions src/partialHydration/__tests__/mountComponentsInHtml.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,6 @@ describe('#mountComponentsInHtml', () => {
jest.mock('../../utils/svelteComponent.ts', () => mockHydrate);
beforeAll(() => {});

it('#replaceSpecialCharacters', () => {
// eslint-disable-next-line global-require
const { replaceSpecialCharacters } = require('../mountComponentsInHtml');
expect(replaceSpecialCharacters('{&quot;nh_count&quot;:15966,&quot;classes&quot;:&quot;mt-3&quot;}')).toEqual(
'{"nh_count":15966,"classes":"mt-3"}',
);
expect(replaceSpecialCharacters('&quot;&lt;&gt;&#39;&quot;\\n\\\\n\\"&amp;')).toEqual('"<>\'"\\n\\n"&');
expect(replaceSpecialCharacters('abcd 1234 <&""&>')).toEqual('abcd 1234 <&""&>');
});

it('mounts a single component in HTML correctly', () => {
hydrated = [];
// eslint-disable-next-line global-require
Expand Down
120 changes: 101 additions & 19 deletions src/partialHydration/__tests__/partialHydration.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ describe('#partialHydration', () => {
content: '<DatePicker hydrate-client={{ a: "b" }} />',
})
).code,
).toEqual(
`<div class="ejs-component" data-ejs-component="DatePicker" data-ejs-props={JSON.stringify({ a: "b" })} data-ejs-options={JSON.stringify({"loading":"lazy","element":"div"})} />`,
).toMatchInlineSnapshot(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switch to inline snapshot so it is easier to update.

`"<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props={JSON.stringify({ a: \\"b\\" })} data-ejs-options=\\"\\"><DatePicker {...({ a: \\"b\\" })}/></div>"`,
);
});

Expand All @@ -20,8 +20,8 @@ describe('#partialHydration', () => {
content: '<DatePicker hydrate-client={{ a: "c" }} hydrate-options={{ "loading": "lazy" }}/>',
})
).code,
).toEqual(
`<div class="ejs-component" data-ejs-component="DatePicker" data-ejs-props={JSON.stringify({ a: "c" })} data-ejs-options={JSON.stringify({"loading":"lazy","element":"div"})} />`,
).toMatchInlineSnapshot(
`"{#if ({ \\"loading\\": \\"lazy\\" }).loading === 'none'}<DatePicker {...({ a: \\"c\\" })}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props={JSON.stringify({ a: \\"c\\" })} data-ejs-options={JSON.stringify({ \\"loading\\": \\"lazy\\" })}><DatePicker {...({ a: \\"c\\" })}/></div>{/if}"`,
);
});

Expand All @@ -32,8 +32,8 @@ describe('#partialHydration', () => {
content: '<DatePicker hydrate-client={{ a: "c" }} hydrate-options={{ "timeout": 2000 }}/>',
})
).code,
).toEqual(
`<div class="ejs-component" data-ejs-component="DatePicker" data-ejs-props={JSON.stringify({ a: "c" })} data-ejs-options={JSON.stringify({"loading":"lazy","element":"div","timeout":2000})} />`,
).toMatchInlineSnapshot(
`"{#if ({ \\"timeout\\": 2000 }).loading === 'none'}<DatePicker {...({ a: \\"c\\" })}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props={JSON.stringify({ a: \\"c\\" })} data-ejs-options={JSON.stringify({ \\"timeout\\": 2000 })}><DatePicker {...({ a: \\"c\\" })}/></div>{/if}"`,
);
});

Expand All @@ -44,8 +44,8 @@ describe('#partialHydration', () => {
content: '<DatePicker hydrate-client={{ a: "b" }} hydrate-options={{ "loading": "eager" }} />',
})
).code,
).toEqual(
`<div class="ejs-component" data-ejs-component="DatePicker" data-ejs-props={JSON.stringify({ a: "b" })} data-ejs-options={JSON.stringify({"loading":"eager","element":"div"})} />`,
).toMatchInlineSnapshot(
`"{#if ({ \\"loading\\": \\"eager\\" }).loading === 'none'}<DatePicker {...({ a: \\"b\\" })}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props={JSON.stringify({ a: \\"b\\" })} data-ejs-options={JSON.stringify({ \\"loading\\": \\"eager\\" })}><DatePicker {...({ a: \\"b\\" })}/></div>{/if}"`,
);
});
it('eager, root margin, threshold', async () => {
Expand All @@ -56,18 +56,16 @@ describe('#partialHydration', () => {
'<DatePicker hydrate-client={{ a: "b" }} hydrate-options={{ "loading": "eager", "rootMargin": "500px", "threshold": 0 }} />',
})
).code,
).toEqual(
`<div class="ejs-component" data-ejs-component="DatePicker" data-ejs-props={JSON.stringify({ a: "b" })} data-ejs-options={JSON.stringify({"loading":"eager","element":"div","rootMargin":"500px","threshold":0})} />`,
).toMatchInlineSnapshot(
`"{#if ({ \\"loading\\": \\"eager\\", \\"rootMargin\\": \\"500px\\", \\"threshold\\": 0 }).loading === 'none'}<DatePicker {...({ a: \\"b\\" })}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props={JSON.stringify({ a: \\"b\\" })} data-ejs-options={JSON.stringify({ \\"loading\\": \\"eager\\", \\"rootMargin\\": \\"500px\\", \\"threshold\\": 0 })}><DatePicker {...({ a: \\"b\\" })}/></div>{/if}"`,
);
});
it('open string', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client="string />',
})
).code,
).toEqual(`<DatePicker hydrate-client="string />`);
await expect(async () => {
await partialHydration.markup({
content: '<DatePicker hydrate-client="string />',
});
}).rejects.toThrow();
Comment on lines +64 to +68
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not valid svelte syntax so I just let preprocessor throw.

});
it('text within component', async () => {
await expect(async () => {
Expand Down Expand Up @@ -105,8 +103,92 @@ describe('#partialHydration', () => {
content: `<Clock hydrate-client={{}} hydrate-options={{ "loading": "eager", "preload": true }} /><Block hydrate-client={{}} hydrate-options={{ "loading": "lazy" }} /><Alock hydrate-client={{}} hydrate-options={{ "loading": "lazy" }} />`,
})
).code,
).toEqual(
`<div class="ejs-component" data-ejs-component="Clock" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({"loading":"eager","element":"div","preload":true})} /><div class="ejs-component" data-ejs-component="Block" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({"loading":"lazy","element":"div"})} /><div class="ejs-component" data-ejs-component="Alock" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({"loading":"lazy","element":"div"})} />`,
).toMatchInlineSnapshot(
`"{#if ({ \\"loading\\": \\"eager\\", \\"preload\\": true }).loading === 'none'}<Clock {...({})}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"Clock\\" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({ \\"loading\\": \\"eager\\", \\"preload\\": true })}><Clock {...({})}/></div>{/if}{#if ({ \\"loading\\": \\"lazy\\" }).loading === 'none'}<Block {...({})}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"Block\\" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({ \\"loading\\": \\"lazy\\" })}><Block {...({})}/></div>{/if}{#if ({ \\"loading\\": \\"lazy\\" }).loading === 'none'}<Alock {...({})}/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"Alock\\" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({ \\"loading\\": \\"lazy\\" })}><Alock {...({})}/></div>{/if}"`,
);
});

it('options as identifier', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client hydrate-options={foo} />',
})
).code,
).toMatchInlineSnapshot(
`"{#if (foo).loading === 'none'}<DatePicker/>{:else}<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props=\\"\\" data-ejs-options={JSON.stringify(foo)}><DatePicker/></div>{/if}"`,
);
});

it('ssr props', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client foo={bar} />',
})
).code,
).toMatchInlineSnapshot(
`"<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props=\\"\\" data-ejs-options=\\"\\"><DatePicker foo={bar}/></div>"`,
);
});

it.skip('ssr props expression in string', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client foo="123/{"bar"}/456" />',
})
).code,
).toMatchInlineSnapshot(
`"{#if ({}).loading === 'none'}<DatePicker {...({})} foo={bar}/>{#else}<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props={JSON.stringify({})} data-ejs-options={JSON.stringify({})}><DatePicker foo={bar}/></div>{/if}"`,
);
});

it('ssr props no name', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client {foo} />',
})
).code,
).toMatchInlineSnapshot(
`"<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props=\\"\\" data-ejs-options=\\"\\"><DatePicker {foo}/></div>"`,
);
});

it('ssr props spread', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client {...foo} />',
})
).code,
).toMatchInlineSnapshot(
`"<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props=\\"\\" data-ejs-options=\\"\\"><DatePicker {...foo}/></div>"`,
);
});

it('style props', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client --foo="bar" />',
})
).code,
).toMatchInlineSnapshot(
`"<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props=\\"\\" data-ejs-options=\\"\\" style:--foo=\\"bar\\"><DatePicker/></div>"`,
);
});

it('style props with expression', async () => {
expect(
(
await partialHydration.markup({
content: '<DatePicker hydrate-client --foo={bar} />',
})
).code,
).toMatchInlineSnapshot(
`"<div class=\\"ejs-component\\" data-ejs-component=\\"DatePicker\\" data-ejs-props=\\"\\" data-ejs-options=\\"\\" style:--foo={bar}><DatePicker/></div>"`,
);
});
});
10 changes: 1 addition & 9 deletions src/partialHydration/inlineSvelteComponent.ts
Original file line number Diff line number Diff line change
@@ -1,19 +1,11 @@
import { HydrateOptions } from '../utils/types';
import { escapeHtml } from '../utils/htmlParser';

const defaultHydrationOptions: HydrateOptions = {
loading: 'lazy',
element: 'div',
};

export function escapeHtml(text: string): string {
return text
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#039;');
}

type InputParamsInlinePreprocessedSvelteComponent = {
name?: string;
props?: any;
Expand Down
19 changes: 6 additions & 13 deletions src/partialHydration/mountComponentsInHtml.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,11 @@
import svelteComponent from '../utils/svelteComponent';

export const replaceSpecialCharacters = (str) =>
str
.replace(/\\\\n/gim, '\\n')
.replace(/&quot;/gim, '"')
.replace(/&lt;/gim, '<')
.replace(/&gt;/gim, '>')
.replace(/&#39;/gim, "'")
.replace(/\\"/gim, '"')
.replace(/&amp;/gim, '&');
import { unescapeHtml as replaceSpecialCharacters } from '../utils/htmlParser';

export default function mountComponentsInHtml({ page, html, hydrateOptions }): string {
let outputHtml = html;
// sometimes svelte adds a class to our inlining.
const matches = outputHtml.matchAll(
/<([^<>\s]+) class="ejs-component[^"]*?" data-ejs-component="([A-Za-z]+)" data-ejs-props="({[^"]*?})" data-ejs-options="({[^"]*?})"><\/\1>/gim,
/<([^<>\s]+) class="ejs-component[^"]*?" data-ejs-component="([A-Za-z]+)" data-ejs-props="([^"]*)" data-ejs-options="([^"]*)"([^>]*)>(<\/\1>)?/gim,
);

for (const match of matches) {
Expand All @@ -23,12 +14,12 @@ export default function mountComponentsInHtml({ page, html, hydrateOptions }): s
let hydrateComponentOptions;

try {
hydrateComponentProps = JSON.parse(replaceSpecialCharacters(match[3]));
hydrateComponentProps = match[3] ? JSON.parse(replaceSpecialCharacters(match[3])) : {};
} catch (e) {
throw new Error(`Failed to JSON.parse props for ${hydrateComponentName} ${match[3]}`);
}
try {
hydrateComponentOptions = JSON.parse(replaceSpecialCharacters(match[4]));
hydrateComponentOptions = match[4] ? JSON.parse(replaceSpecialCharacters(match[4])) : {};
} catch (e) {
throw new Error(`Failed to JSON.parse props for ${hydrateComponentName} ${match[4]}`);
}
Expand All @@ -50,6 +41,8 @@ export default function mountComponentsInHtml({ page, html, hydrateOptions }): s
page,
props: hydrateComponentProps,
hydrateOptions: hydrateComponentOptions,
otherAttributes: match[5],
openTagOnly: !match[6],
});

outputHtml = outputHtml.replace(match[0], hydratedHtml);
Expand Down
88 changes: 55 additions & 33 deletions src/partialHydration/partialHydration.ts
Original file line number Diff line number Diff line change
@@ -1,45 +1,67 @@
import { inlinePreprocessedSvelteComponent } from './inlineSvelteComponent';
import MagicString from 'magic-string';
import { parseTag } from '../utils/htmlParser';

const extractHydrateOptions = (htmlString) => {
const hydrateOptionsPattern = /hydrate-options={([^]*?})}/gim;
const stringifyExpression = (s) => (s ? `{JSON.stringify(${s})}` : '""');

const optionsMatch = hydrateOptionsPattern.exec(htmlString);
if (optionsMatch) {
return optionsMatch[1];
const createReplacementString = (content, tag) => {
let options = '';
let clientProps = '';
let styleProps = '';
let stylePropsRaw = '';
let serverProps = '';
for (const attr of tag.attrs) {
if (/^hydrate-client$/i.test(attr.name)) {
if (attr.value) {
clientProps = content.slice(attr.value.exp.start, attr.value.exp.end);
}
} else if (/^hydrate-options$/i.test(attr.name)) {
options = content.slice(attr.value.exp.start, attr.value.exp.end);
} else if (/^--/i.test(attr.name)) {
stylePropsRaw += ` ${content.slice(attr.start, attr.end)}`;
styleProps += ` style:${attr.name}=${content.slice(attr.value.start, attr.value.end)}`;
} else {
serverProps += ` ${content.slice(attr.start, attr.end)}`;
}
}
return '';
};

const createReplacementString = ({ input, name, props }) => {
const options = extractHydrateOptions(input);
return inlinePreprocessedSvelteComponent({ name, props, options });
const spreadClientProps = clientProps ? ` {...(${clientProps})}` : '';
// FIXME: it should be possible to merge three attributes into one
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we should be able to generate simpler code in the preprocessor e.g.

`<div ejs-component=${stringifyExpression(`["${tag.name}",${clientProps},${options}]`)}>`

Then we just have to search for ejs-component to extract all information in mountComponentsInHtml.ts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely. This would be a huge simplification.

// FIXME: use hydrateOptions.element instead of 'div'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hydrateOptions.element no longer works in this PR. I didn't find the document for it so I'm not sure how to implement it correctly.

If this is needed, we can try to find the element via regex:

const rx = /\belement['"]?\s*:\s*['"]([^'"]+)/i;
const element = options.match(rx)?.[1];

This will work if the options is written in the source:

hydrate-options={{element: "span"}}
hydrate-options={{"element": "span"}}

But won't work in other cases:

<script>
const o = {element: 'span'};
</script>
<Component hydrate-client hydrate-options={o} />

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hydrate-options={{element: "span"}}
hydrate-options={{"element": "span"}}

This was the implementation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another solution is to use a special tag e.g.

<ejswrapper ... ></ejswrapper>

And replace it with the real tag name in mountComponentsInHtml

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eight04 Not strongly opinionated either way. Whatever you think is best.

const wrapper =
`<div class="ejs-component" data-ejs-component="${tag.name}"` +
` data-ejs-props=${stringifyExpression(clientProps)}` +
` data-ejs-options=${stringifyExpression(options)}` +
`${styleProps}>` +
`<${tag.name}${spreadClientProps}${serverProps}/></div>`;
if (!options) {
return wrapper;
}
return (
`{#if (${options}).loading === 'none'}<${tag.name}${spreadClientProps}${stylePropsRaw}${serverProps}/>` +
`{:else}${wrapper}{/if}`
);
};

export const preprocessSvelteContent = (content) => {
// Note: this regex only supports self closing components.
// Slots aren't supported for client hydration either.
const hydrateableComponentPattern = /<([a-zA-Z]+)[^>]+hydrate-client={([^]*?})}[^/>]*\/>/gim;
const matches = [...content.matchAll(hydrateableComponentPattern)];

const output = matches.reduce((out, match) => {
const [wholeMatch, name, props] = match;
const replacement = createReplacementString({ input: wholeMatch, name, props });
return out.replace(wholeMatch, replacement);
}, content);

const wrappingComponentPattern = /<([a-zA-Z]+)[^>]+hydrate-client={([^]*?})}[^/>]*>[^>]*<\/([a-zA-Z]+)>/gim;
// <Map hydrate-client={{}} ></Map>
// <Map hydrate-client={{}}></Map>
// <Map hydrate-client={{}}>Foo</Map>

const wrappedComponents = [...output.matchAll(wrappingComponentPattern)];

if (wrappedComponents && wrappedComponents.length > 0) {
throw new Error(
`Elder.js only supports self-closing syntax on hydrated components. This means <Foo /> not <Foo></Foo> or <Foo>Something</Foo>. Offending component: ${wrappedComponents[0][0]}. Slots and child components aren't supported during hydration as it would result in huge HTML payloads. If you need this functionality try wrapping the offending component in a parent component without slots or child components and hydrate the parent component.`,
);
let dirty = false;
const hydrateableComponentPattern = /<([a-zA-Z]+)[^>]+hydrate-client/gim;
const s = new MagicString(content);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work here. Huge simplification.

for (const match of content.matchAll(hydrateableComponentPattern)) {
const tag = parseTag(content, match.index);
if (!tag.selfClosed) {
throw new Error(
`Elder.js only supports self-closing syntax on hydrated components. This means <Foo /> not <Foo></Foo> or <Foo>Something</Foo>. Offending component: ${content.slice(
tag.start,
tag.end,
)}. Slots and child components aren't supported during hydration as it would result in huge HTML payloads. If you need this functionality try wrapping the offending component in a parent component without slots or child components and hydrate the parent component.`,
);
}
const repl = createReplacementString(content, tag);
s.overwrite(tag.start, tag.end, repl);
dirty = true;
}
return output;
return dirty ? s.toString() : content;
};

const partialHydration = {
Expand Down
Loading