2023 Recap and 2024 Plans for OpenSceneGraph and VulkanSceneGraph projects

[robertosfield]

Happy New Year Pixel Pushers :-)

I'm writing this post for both the OpenSceneGraph and VulkanSceneGraph communities, so posting it in both forums. The work I've done over the past years and my plans for the next year affect both communities, and many developers either already straddle both communities or may be looking to do in the future so hopefully will appreciate knowing what is happening on both sides.

2023 Recap

Personally I had an extremely busy year with VulkanSceneGraph development and support, and working with clients consuming all the hours I had available. As an open source developer a great deal of what I do is logged by GitHub - with 2616 commits I averaged over 7 commits a day!

All this work didn't go unnoticed - I was included in OpenUK's 2024 New Year's honours list. I guess that makes me the Honourable Graphics Nerd.

VulkanSceneGraph 2023 releases and major features

I tagged 6 VulkanSceneGraph minor point releases between March and August taking us from VulkanSceneGraph-1.0.3 to 1.0.9. These releases included lots of bug fixes, refinements and new features. Rather than trying to enumerate it all here you can checked out the VulkanSceneGraph/releases page for all the changes.

Between August and December I was focused on developing major new VulkanSceneGraph features - introducing a flexible scheme for pre and post rendering and a visually pleasing Cascaded Shadow Maps implementation. We never got the shadow map support right in the OpenSceneGraph so it's great to come back to this topic afresh and achieve results that are far more robust and easy to use.

The shadowing support is fully integrated into the core scene graph and the vsg::ShaderSet scheme so it's easy to render Physics Based Rendering models with shadows from multiple directional lights:

It also seamlessly handles whole earth models:

All this new functionality and much more was wrapped in the VulkanSceneGraph-1.1.0 release that I tagged last week.

The family of VulkanSceneGraph projects also continues to advance with new releases for vsgXchange, vsgExamples, vsgImGui, vsgQt and a new member of the vsg-dev family - vsgPoints](https://github.com/vsg-dev/) that adds support for paged and quantized point clouds capable of rendering billions of point datasets at 60fps.

In the Spring of 2023 I also begun work on an free online tutorial book - vsgTutorial. I have completed the first 2 chapters and begun the 3rd, but since the Summer 2023 I've been so overrun with other VSG work that I haven't had time to return to the effort to complete it. It takes a lot of time to write detailed documentation so will have to get back to it once I have the opportunity.

@timoore's also continued work on vsgCs that integrates VulkanSceneGraph with Ceisum-ion to provide 3D Tiles datasets, which is now very close to it's first stable release. Cesium were so impressed that offered Tim a job!

@pelicanmapping have also been developing their new Rocky - 3D Geospatial Application Engine, which builds up the VulkanSceneGraph.

Sundog Software closed out 2023 with announcing availability of SilverLining for Vulkan & VulkanSceneGraph.

OpenSceneGraph in 2022/2023

After the first initial VulkanSceneGraph-1..0.0 made in November 2022 I jumped back to the OpenSceneGraph project to review and merge community PR's and make steps towards making another 3.6.6 maintenance release from the OpenSceneGraph-3.6 stable branch. The most notable public facing change was to move from mailing list/broken forum to using GitHub Discussions forum for support.

I also worked on creating new master branch which stripped out my experimental work on shader composition that in hindsight I shouldn't have merged with OSG master right after the 3.6.5.

Unfortunately once we were in 2023 I became so busy with VulkanSceneGraph client and open source work that I had no time to dive properly into OpenSceneGraph support work, so had to fallback to just doing a minimum of answering end user support questions that even now trickle in. Watching the Issue and PRs sit there unanswered sits heavily on me, after years of being so proactive I simply don't have the time and mind space to handle them.

I have to admit that I wince each time I encounter a new developer wanting to learn OpenSceneGraph/OpenGL and use it in their projects, I can understand existing OpenSceneGraph users needing support, but OpenSceneGraph/OpenGL are both legacy APIs that are now just in maintenance mode. OpenGL hasn't had an new release since 4.6 was made in July 2017, and the OSG hasn't had a the 3.6.5 release in January 2020, while neither is ideal it is the state of play. New developers really should be basing their work on modern and thriving APIs, not APIs that are over 20 to 30 years old and a decade beyond their Best Before Date.

Plans for VulkanSceneGraph in 2024

The first half of this year is rapidly filling with client work and open source work, which dispels any romantic notion that I might have time and brain cells left over to tackle other work. Like the rest of the last 5 years I've been working on the VulkanSceneGraph my brain cells are going to being going arrrrhhhhhhhh between episodes of learning new tricks and coming up with hopefully useful ways to fit a scene graph to the needs of modern application developers.

Features I have penciled include:

1. Tone Mapping
2. Percentage Closer Soft Shadows
3. Support for shadows for Spot and Point lights
4. A cascaded view dependent render to texture for rendering high quality overlays
5. vsgIG Framework that will build upon family of VSG projects for vis-sim applications
6. Rewrite the osg2vsg library was written early in the VSG project's life and doesn't leverage all the latest features

Plans for OpenSceneGraph in 2024.

For me the supporting the OpenSceneGraph is all about what we can do to support existing users that have built their applications on top of the OpenSceneGraph. Maintaining the existing API and updating the software to build with new OS's, build tools and 3rd party dependencies needs to be the focus of efforts.

We also need to rebuild elements of the project infrastructure like the website so it is all centrally managed on GitHub in a similar fashion to how the VulkanSceneGraph project works. As the OpenSceneGraph project is now in maintenance mode we need to keep time and financial overhead to a minimum so we can concentrate our scare time resources on maintaining the software not the systems that surround it.

To help make this happen back in November 2023 I hired Chris Djali (@AnyOldName3) on a part-time basis to look at archiving the present OpenSceneGraph website, that is run on Joomla and hosted on Dreamhost, using Github project, and to create a new GitHub pages based OpenSceneGraph website that is more modest and more in keeping with the VulkanSceneGraph project in style and layouts. Once these websites are working well enough I'll redirect openscenegraph.com/.org to new sites and I can close my Dreamhost account. Chris and I will follow up with more details of this work with dedicate thread on openscenegraph forum.

I would love to be able to make a OpenScenegraph-3.6.6 release this year, and then a OpenSceneGraph-3.8 based off the new master. I can't promise anything at this point as I just don't have any room in my schedule for even a weeks work just on the OSG side, let alone the 6 weeks that it typically took to get a stable release tested and debugged and ready for release.

Vulkan is the future and VulkanSceneGraph is my best attempt at making that future user/developer friendly

Khronos has left OpenGL in a coma and on life support for 6 and half years. It's not going to suddenly spring back to life and become relevant for future applications. Khronos is fully behind Vulkan and it's developing fast, support latest hardware features with good developer support. For Khronos OpenGL is legacy, Vulkan is the present and future for low level cross platform graphics APIs.

I see my work on VulkanSceneGraph as crucial for the existing OpenSceneGraph user base - the quicker the VulkanSceneGraph project and it's associated libraries/framework are feature rich enough to tackle all the tasks that the OpenSceneGraph currently provides the sooner the members of the OpenSceneGraph have a safe and friendly harbour in which to dock their projects long term.

I also see examples where developers use middle-ware, like Game Engines,, that are poorly suited to their needs, the world needs scene graphs more than ever - in many ways Vulkan is lower level than OpenGL so the need for middle-ware to encapsulate functionality is crucial for lowering the barrier to entry and improving end results.

Developing Vulkan application needn't be complicated, many tasks are now actually easier to do with the VulkanSceneGraph than it ever was with OpenSceneGraph - for instance Physics Based Rendering and high quality shadows are all available out of the box. For those in the OpenSceneGraph skeptical of Vulkan and this new fanged VulkanSceneGraph please try it out. If there areas that aren't yet up to par with OpenSceneGraph functionality lets us know - together we build a modern scene graph that fits your needs fully.

Dearly departed

GitHub now has a mechanism for passing on responsibility of what to do with accounts when an account holder dies, a write up can be found on the github-deceased-user-policy page.

This isn't critical for my own personal GitHub account but for the openscenegraph and vsg-dev accounts I think it makes sense for us to start discussing seriously how we want to tackle it and how we should involve as backup for if/when the worst happens.

GitHub Sponsorship?

For the past 23 years I have run, OpenSceneGraph Professional Services, my little one man free software business, where I use income from my client work - bespoke development work, consulting, training and support to pay the bills and enable me to dedicate so much time to unpaid Open Source work. Historically I have averaged about 1/3 of my time to client work and 2/3 of my time to unpaid open source development and support work. I have made it work because I'm a mug/love coding/helping folks, but might there be a better way?

Something I get asked about occasionally is about donations and have previously not followed this up as setting things up takes time and the level of income needs to be far more few small $ donations, for it to really make a big different we'll wanting to raise 10's of thousands of dollars each year to cover months of developer time. As GitHub now has Sponsorship support so is it time to look at this seriously?

If you have experience with Sponsoring projects on GitHub or using Sponsorship for your own projects let me know your thoughts/experiences.

Wrapping up

I've covered lots of topics in this post, though it's really just a taster of what's been happening and what will likely unfold, no doubt lots of unexpected things will happen this year like they have in previous years. Feel free to chip in with this thread on the topics covered here, if you want to discuss other topics please create a separate Issue or Discussion thread for them.

All the best for 2024!
Robert Osfield.
VulkanSceneGraph & OpenSceneGraph Project Lead.

[XenonofArcticus]

Excellent news, Robert, on all fronts. Exciting times indeed. We are very happy with the direction of VSG, and all the new possibilities. If you set up some kind of recurring sponsorship system, I'd like to be the first to start donating to it. I know a stable income stream makes it much easier to plan future work.

[robertosfield]

Do know any other GitHub projects that are using Sponsorship in a way that would form a good template for what we could do for the VulkanSceneGraph/OpenSceneGraph?

[robertosfield]

I forgot to include mention of my work in December - creating an extensible instrumentation scheme for the VulkanSceneGraph which also provides support for RenderDoc and Tracy Profiler. I merged this functionality with VSG master on Monday and wrote up the work on VSG forum yesterday. Integrating with RenderDoc and Tracy provides access to two very powerful and free tools, life for scene graphs users has never been so plush ;-)

On the VulkanSceneGraph features planned for 2024 I also neglected to mention work on animations:

1. Rigid body animation updating transform matrices
2. Morphing geometry between key frames
3. Character skinning

Further out I'd also like to see an environmental effects library for stars/planets/atmosphere/clouds and oceans/lakes/rivers.

[cvilas]

Great update, and amazing progress. Thank you @robertosfield. I really do hope that you will have the time and energy (or perhaps get support from others) in 2024 to continue working on the tutorials and possibly a book. Given the nature of the topic, I feel good documentation is critical, especially for adoption from newcomers.

Wish you a fantastic and productive 2024.

[robertosfield]

The vsgTutorial is essentially my book, it's just online and will be updated when I have time. I expect it'll take several months to finish the vsgTutorial. This is an area where Sponsorship might be useful - providing an income to pay the bills so I don't have to take on client work during that period of writing the book.

Once vsgTutorial is written we could possibly look at editing for the purposes of publishing in a physical book, but personally I think having an online resource provides more value as your can search it, link to it and provide links within it that are convenient to follow. Personally I think it'd take more time and money to get a physical book than it's worth but if there's solid demand for it when vsgTutorial is finished then perhaps we can sponsor someone to undertake this work.

[cvilas]

I know it's not much, but if you were to open up your projects for github sponsorship, I would donate as an individual. Hopefully this will motivate others to donate to it as well.

(I don't use VSG in a professional setting anymore, but I am still interested in it and would like to consider using it professionally in the future again).

[xarray]

Hi Robert, I've been planning for years to implement an independent .osgb format parsing library that doesn't depend on OSG (which is really huge and complicated). It can then be easily merged with game engines and even web applications. After all, the .osgb format is still widely used in oblique photography modeling.

I think this would be an opportunity for me to get involved in the osg2vsg project or something else. I also maintain the osgverse library from 2022, which mainly focuses on modern rendering techniques and WASM pipeline based on OSG. Hopefully I could port/share some of these feature codes to VSG, like character skeleton animation and blendshapes.

I'm still beginners of Vulkan and VSG myself. Hope I could get the gist of it soon and write some articles to help more people. :)

[robertosfield]

I did briefly considering writing a .osgb/.osgt loader as part of osg2vsg but the serializes are pretty huge in themselves and it'd need struct front ends for OSG class equivalents. Perhaps a cut down version of the OSG would be the easiest way to do something like that.

If there is a number of .osgb/.osgt files that users want to convert then it may just be best to have a convert run as batch then load them in the new format, be that something like .vsgb or .gltf etc.

For my own plans for this year I would like to see a rewrite of osg2vsg to user modern VSG features, I needn't be the one to do such a rewrite.

I have also toyed with the idea of having native gltf read/write support in the VSG.

[xarray]

I could try reimplementing the .osgb reader with an automatic batch tool to parse source code of existing OSG serializers and generate new independent files. It would save a lot of manual work.
I've also implemented native gltf and fbx reader in osgverse with tinygltf and openfbx, both of which are lightweight, easy to use and with friendly licenses. The skeleton and skinning supports are based on the ozz-animation library (https://github.com/guillaumeblanc/ozz-animation). I think some code and resources there could be reused in the VSG 2024 plan.
Maybe some initial work could be done in this and the following month. I will first have a deep look at current VSG related projects to find the best place to add these new functionalities and examples.

[robertosfield]

Perhaps rather than reusing the OSG serializes directly one could use the serialize interface to write a format specification that can be read in a generic way.

Thanks for the link to ozz-animation. Just had a quick look, perhaps this has ideas that would be worthwhile following but code wise it I don't think it makes sense to use for the VSG project. My expectation is that animation will be relatively small extension to the existing VSG code base and loaders.