-
Notifications
You must be signed in to change notification settings - Fork 0
/
normalheight.html
1 lines (1 loc) · 4.47 KB
/
normalheight.html
1
<!DOCTYPE html> <html lang=en > <link rel=stylesheet href="theme/css/base.css" type="text/css" /> <meta name=viewport content="width=device-width,initial-scale=1"> <title>HugoTini - Normals to height</title> <meta charset=utf-8 /> <meta name=generator content=Pelican /> <link href="https://hugotini.github.io/feeds/all.atom.xml" type="application/atom+xml" rel=alternate title="HugoTini Blog Atom Feed" /> <!-- --> <link rel=stylesheet href="theme/css/article.css" type="text/css" /> <body id=index class=home > <div id=main_container > <!--/ header menu --> <header id=header > <div class=menu-item-active > <a href="https://hugotini.github.io/">Blog</a> <div class=menu-underline-active ></div> </div> <div class=menu-item > <a href="https://hugotini.github.io/about">About</a> <div class=menu-underline ></div> </div> </header> <section id=content > <div style="background-image: url('assets/normalheight/banner.jpg');" id=banner ></div> <div id='title_container'> <h1>Normals to height</h1> <p>Computing depth from gradients</p> </div> <time id=item_date datetime="2020-05-25T00:00:00+02:00"> May 2020 </time> <div id=article_content > <p>Previously, I wrote about <a href="https://hugotini.github.io/deepbump.html">DeepBump</a>, a machine learning experiment to generate normal maps from single pictures. However, when creating 3D (or 2D) content, the actual height ("displacement") of the surface might also be useful. To complement DeepBump, <a href="https://github.com/HugoTini/NormalHeight">NormalHeight</a> is small extra Blender add-on that computes the height from normals.</p> <h1>Depth-from-gradient</h1> <p>Going from a gradient (normal map) to the depth (height map) is often called "shape-from-gradient" or "depth-from-gradient". This has typically been used in computer vision, for instance after having found the normal map of an object through <a href="https://en.wikipedia.org/wiki/Photometric_stereo">photometric stereo</a>. In our case, DeepBump already takes care of generating the normal map from a single photo. NormalHeight then uses the <a href="https://webdocs.cs.ualberta.ca/~vis/courses/CompVis/readings/photometric/FrankotIntegrpami88.pdf">Frankot-Chellappa algorithm</a> (Eq. 22) which uses the Fourier Domain to compute a height based on normals.</p> <h1>Usage</h1> <p>By combining DeepBump with NormalHeight, we can now get a displacement from a single photo :</p> <p class='img_container'> <a href='https://hugotini.github.io/assets/normalheight/compares.jpg'> <img class='article_img' src='https://hugotini.github.io/assets/normalheight/compares.jpg'> </a> </p> <p class='img_caption'>original photos : <a href='https://unsplash.com/photos/wtjrpjZABcQ'>a</a>, <a href='https://unsplash.com/photos/lXVbqVIImZY'>b</a></p> <p>Displacement can also be used for material mixing by thresholding the generated height (3rd picture) and using it as a mask :</p> <p class='img_container'> <a href='https://hugotini.github.io/assets/normalheight/mixing.jpg'> <img class='article_img' src='https://hugotini.github.io/assets/normalheight/mixing.jpg'> </a> </p> <p class='img_caption'>original textures : <a href='https://opengameart.org/content/2048-digitally-painted-tileable-desert-sand-texture'>a</a>, <a href='https://opengameart.org/content/handpainted-stone-wall-textures'>b</a></p> <p class='img_container'> <a href='https://hugotini.github.io/assets/normalheight/mixing2.jpg'> <img class='article_img' src='https://hugotini.github.io/assets/normalheight/mixing2.jpg'> </a> </p> <p class='img_caption'><a href='https://unsplash.com/photos/KCq781yFwWM'>original photo</a></p> <h1>Limitations</h1> <p>DeepBump learned to create locally probable normals, however, as the texture is processed tile by tile, on large pictures the normals might lack general structure (and thus the height map too).</p> <p>To go further, we could get better results by giving the neural net more context about the whole picture rather than processing tiles independently. Moreover, we might get better height maps by teaching the neural net to predict directly the depth rather than the normals.</p> </div> </section> </div> <div class='footer'> <a href="https://hugotini.github.io/feeds/all.atom.xml"> <img class='svg_icon' src="theme/icons/rss-alt.svg"> </a> <a href="https://github.com/HugoTini"> <img class='svg_icon' src="theme/icons/github.svg"> </a> <a href="https://twitter.com/Hugo_Tini"> <img class='svg_icon' src="theme/icons/twitter.svg"> </a> <a href="https://hugotini.github.io/about"> <img class='svg_icon' src="theme/icons/email.svg"> </a> </div>