[FR]: Publish canvas files

[rufuspollock]

Request - see #538 (comment) @yuri-karpovich

Can Obsidian Canvas drawings be displayed? Do you have it on your roadmap? It would be a game-changing feature that could replace Miro for me

Implementation

Noticed in this thread about svg export for canvas that someone has written a converter https://forum.obsidian.md/t/add-svg-export-feature-to-canvas/54379/7

He has written it in JS and documented it here https://janikvonrotz.ch/2023/06/07/convert-obsidian-canvas-to-svg/

This could definitely be run at build time ...

Acceptance

• Can render Obsidian canvas pages
• Can render inline canvas drawings
• Can render text, images, pages nodes, pdf, video, webpages, other canvases
• Can draw edges ( with edge color, text and position )
• Documented or at least announced

Notes

What features does Obsidian Canvas have? (see https://obsidian.md/canvas)

Card types (color)

1. Text card (MD-formatted text)
2. Markdown file embed
3. Image & video
4. PDF file
5. Webpage
6. Nested canvas
7. Edges (edge text, color)

[saviorand]

Definitely interested in this feature!

[rufuspollock]

@saviorand thanks for adding the encouragement 🙏

[rufuspollock]

@mohamedsalem401 can you look at this?

[mohamedsalem401]

@rufuspollock
The issue is not simple, but I managed to make some progress.

The canvas files in Obsidian have their unique structure, which requires studying for a while. (It consists of connected nodes, which can be of 3 types, one of which can be a whole embedded md page in the canvas itself!)

{
	"nodes":[
		{"id":"1ac775de1c4c5783","type":"text","text":"This's text","x":-220,"y":-380,"width":250,"height":60,"color":"6"},
		{"id":"e147cd87ab06b1b4","x":30,"y":-220,"width":400,"height":86,"type":"file","file":"table2.png"},
		{"id":"73f047cfedbff550","x":-560,"y":-240,"width":400,"height":400,"type":"file","file":"TEST2.md"}
	],
	"edges":[
		{"id":"01322ffb294beddf","fromNode":"1ac775de1c4c5783","fromSide":"bottom","toNode":"e147cd87ab06b1b4","toSide":"top"},
		{"id":"483466db653ca055","fromNode":"1ac775de1c4c5783","fromSide":"top","toNode":"73f047cfedbff550","toSide":"top"},
		{"id":"c8996c940037a136","fromNode":"1ac775de1c4c5783","fromSide":"right","toNode":"e147cd87ab06b1b4","toSide":"bottom"},
		{"id":"dc30c90058c68b5d","fromNode":"1ac775de1c4c5783","fromSide":"left","toNode":"73f047cfedbff550","toSide":"right"}
	]
}

This needs to be converted to an SVG, which requires researching SVG syntax. (The code in the article https://janikvonrotz.ch/2023/06/07/convert-obsidian-canvas-to-svg/ has some deficits, so I have to edit it.)

Also, adding the .canvas pages to FlowerShow is troublesome, as mddb doesn't generate the url_path field for anything other than md files. So, this also needs to be addressed.

[rufuspollock]

@mohamedsalem401 ok, good progress. It sounds like we want to update the description of the issue with the tasks as you now understand it. It sounds like it breaks into two parts:

• A renderer for canvas files that produces svgs ...
• changes to markdowndb to list canvas files (i'm not quite sure what the issue is atm in markdowndb - can you clarify that more)

[olayway]

It seems like a backbreaking task 😬 In the current apporach, i.e. converting Obsidian canvases to SVGs, how are we going to:

1. support all the Obsidian canvas features, like:

• other markdown notes embeds
• other canvas' embeds
• yt embeds
• rendering canvas cards supporting all Obsidian features (e.g. a card with callouts or even just headings)

2. make it look professional (doable but not so easy)

@mohamedsalem401 before continuing with the code, can you please write out how you would solve the above issues (all but the styling)?

Note 1:
I was trying to find a way to use Obsidian's "Export to image" command from the API instead so that maybe we could trigger it programmatically in our plugin. It seems you can do this with app.commands.executeCommandById('canvas:export-as-image') but it only triggers a popup and you can't (afaik) pass arguments to it. But maybe there is some other method?

[mohamedsalem401]

What features does Obsidian Canvas have? (see https://obsidian.md/canvas)

Card types (color)

1. Text card (MD-formatted text)
2. Markdown file embed
3. Image & video
4. PDF file
5. Webpage
6. Nested canvas
7. Edges (edge text, color)

Old solution

In the first approach, we used an SVG file, which had the following issues:

• Can't embed webpages
• Can't embed notes
• And many other issues

New approach

A better approach would be to use HTML canvas elements. We can use a library called Konva.js, which supports drawing rectangles (will be the card skeleton) and lines (will be the edges). To solve the issue with embedding other things (PDF, other notes, webpages, nested canvas), we can leave an empty space in the canvas, which will be filled by an iframe element by moving it using CSS position: fixed;.

Note

1. The coordinate system in Obsidian can be negative (e.g., x: -150). We can overcome this by adding the least negative value; in this example, it would be x: -150 + 150, i.e., adding 150 to all coordinates.
2. Edges only have IDs of the starting and ending nodes (i.e., we need to calculate the start and end coordinates for edges ourselves).
3. The note JSON would look like:

{
	"nodes":[
		{"id":"b3eedc7de503c3c7","type":"text","text":"","x":-677,"y":-905,"width":250,"height":60},
		{"id":"7af03c59336d7e50","type":"file","file":"Asset 21332.png","x":-650,"y":-174,"width":400,"height":378},
		{"id":"407e8252f3b3037d","type":"text","text":"","x":-561,"y":-597,"width":250,"height":60},
		{"id":"1a414e25a4598008","type":"file","file":"hello world.md","x":-160,"y":80,"width":400,"height":400},
		{"id":"0eaf1351bf30e4d9","type":"text","text":"Text Box ","x":-85,"y":-65,"width":250,"height":80,"color":"5"},
		{"id":"21212ab79f0be1a6","type":"text","text":"","x":-120,"y":-280,"width":250,"height":106},
		{"id":"c56951957feba5c0","type":"text","text":"Hello there welcome to my Blog ","x":-575,"y":-380,"width":250,"height":100}
	],
	"edges":[
		{"id":"c9a78200b99497ea","fromNode":"21212ab79f0be1a6","fromSide":"left","toNode":"c56951957feba5c0","toSide":"right","fromEnd":"arrow"},
		{"id":"b28a187d6fae9315","fromNode":"21212ab79f0be1a6","fromSide":"bottom","toNode":"0eaf1351bf30e4d9","toSide":"top"},
		{"id":"3dfab590c6fb36e6","fromNode":"c56951957feba5c0","fromSide":"bottom","toNode":"7af03c59336d7e50","toSide":"top"},
		{"id":"b67215fa4fb833eb","fromNode":"0eaf1351bf30e4d9","fromSide":"bottom","toNode":"1a414e25a4598008","toSide":"top"}
	]
}

[olayway]

@mohamedsalem401 I thought it through and I think the best (if not the only) solution that would allow us to support all or at least most of Obsidian canvas features is to create a react Canvas component that would receive the canvas JSON (after a few modifications to it; see below for explanation) and render it as HTML - just like it's done in Obsidian.

Here is the outline of the approach for rendering canvas pages (not canvas embeds):

In [[...slug]] file:

• Adjust getStaticPaths to return paths to canvas files as well (not only to md or mdx)
• Adjust getStaticProps so that for .canvas pages it loads the canvas JSON file and modify it like so:
  • for any node of type text use our parse function to parse its content and save the resulting mdxSource in an additional property on the node, e.g. called mdxSource
  • for any node of type file with either md or mdx extension load the file and do the same thing as in the point above
• Adjust Page function so that based on the current page type - regular md/mdx or canvas - it either uses MdxPage component or a new Canvas component.

For zoom and pinch behavior you could probably use this package https://github.com/BetterTyped/react-zoom-pan-pinch?tab=readme-ov-file

As for negative coordinate values, you're correct, we need to normalize them, and the easiest way to do this is probably the way you do it, i.e. add |min x or y value| to all x or y coordinates respectively.

I wanted to test this approach and here is the quick and dirty implementation: https://github.com/datopian/flowershow/tree/canvas-2

Here is how it renders atm (without any styles at all):

As you can see, it does render both cards and note embeds with all the syntax elements supported, and not just rendered as plain text.

Note: If you want to try it out, remember to manually add the url_path to the Example.canvas file in the db, as atm mddb only adds it for md and mdx files.

[mohamedsalem401]

@olayway

This is a similar approach to mine but the main difference is that I am using a canvas tag to draw the edges and then the rest will be HTML...

[rufuspollock]

@mohamedsalem401 @olayway this originally started as a quick investigation - to see if the code i had linked worked for a "quick and dirty" solution.

If there is a decent 80/20 solution we can quickly ship then let's do it ...

But if is a big hassle i would strongly recommend leaving this and focusing on other better value/cost things.

[olayway]

@olayway

This is a similar approach to mine but the main difference is that I am using a canvas tag to draw the edges and then the rest will be HTML...

Maybe it is (although I don't think they have much in common), but I have also managed to get embeds and cards with Obsidian syntax elements to render correctly...
Also, why would you need to use canvas for edges vs just using svg?

A better approach would be to use HTML canvas elements. We can use a library called Konva.js, which supports drawing rectangles (will be the card skeleton) and lines (will be the edges). To solve the issue with embedding other things (PDF, other notes, webpages, nested canvas), we can leave an empty space in the canvas, which will be filled by an iframe element by moving it using CSS position: fixed.

It doesn't sound like my approach to me 🤔 I'm not using canvas at all, no additional library is needed, and I'm not using iframes for embeds.

[olayway]

@mohamedsalem401 @olayway this originally started as a quick investigation - to see if the code i had linked worked for a "quick and dirty" solution.

If there is a decent 80/20 solution we can quickly ship then let's do it ...

But if is a big hassle i would strongly recommend leaving this and focusing on other better value/cost things.

@rufuspollock as you can see in my previous comment, we already have a working version, so let's maybe not ditch it at this point. We'll put a few more hours into it and ship it even if not perfect, and we'll iterate in the future.

[saviorand]

Very impressive, how fast y'all did it! 👏👏