GSOC23: Renderer Docs

docs/index.rst

@@ -17,6 +17,7 @@ Welcome to Synfig developers documentation!
    community/contribution guidelines
    ide/Setting up your preferred IDE
    common/structure
+   renderer
    building/Building Synfig
    packaging/packaging
    tutorials

docs/renderer.rst

@@ -0,0 +1,16 @@
+.. _renderer:
+
+Renderer Docs
+=====================
+
+This section explains the different parts of Cobra Engine and the algorithm which uses them to render images.
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   renderer_docs/introduction
+   renderer_docs/target_surface
+   renderer_docs/tasks
+   renderer_docs/render_queue
+   renderer_docs/optimizers

docs/renderer_docs/introduction.rst

@@ -0,0 +1,49 @@
+.. _renderer_intro:
+
+Introduction
+============
+
+There are two ways to render a Synfig document, by using the **synfig** CLI tool or via **Synfig Studio**. The rendering code is the same, but some concepts would be easier explained using the CLI tool. A basic command to render a Synfig document using the CLI looks like this
+
+.. code-block:: bash
+
+   synfig $FILE -o out.png --time=0 --width=1920 --height=1080
+
+This will render only the first frame(``--time=0``) of ``$FILE`` with dimensions 1920x1080 to target *out.png*. Synfig supports rendering to multiple file formats. These file formats are represented as ``Target`` in Synfig's code base. The CLI performs the following tasks to render an image:
+
+* Boot Synfig by using ``synfig::Main``, which initializes different modules and systems.
+* Read the document file and create the internal representation of the document.
+* Extract and execute a ``Job``.
+
+The first two steps are not part of the Renderer. Therefore this section only covers the last step.
+
+Job
+~~~
+
+This class is present in CLI only, and it is simple. This class stores information used by other functions to start rendering the file. The most important fields it stores are ``synfig::RendDesc desc`` and a handle to the ``synfig::Canvas``. After all the initialization and reading of the document is done, the first step taken by the CLI is to create and fill a Job.
+
+Usually, only one Job is created, but two jobs are created if the user wants to extract Alpha to another file. These jobs are first set up and run. The setup step attempts to find the ``Target`` specified by the user or by the file extension, Render Engine to use, permissions, etc.
+
+To start rendering the file, ``job.target->render(..)`` is called. This function actually starts the rendering process.
+
+Target
+~~~~~~
+
+``Target`` represents the output (file or memory) and handles the frame-by-frame rendering process. The base class ``Target`` has a few virtual methods overridden by derived classes like ``Target_Scanline``. Targets for output files are modules that derive mostly from ``Target_Scanline``. Details about how the correct ``Target`` class is acquired and the actual working of ``Target::render()`` can be found in :ref:`renderer_target_surface`.
+
+The Cobra engine is multithreaded, executing independent Tasks on different threads. The function ``Canvas::build_rendering_task`` creates a Task for rendering a frame. This function is called by ``Target::render()``.
+
+Tasks
+~~~~~
+
+Tasks are the main objects of the Cobra engine, which writes and transforms pixels. There are tasks for blending, rendering shapes, transformation, etc. Tasks can have dependencies stored in a ``sub_tasks`` list inside the class ``Task``. ``Canvas::build_rendering_task`` builds this graph of Tasks, which is then sent to the Render Engine for execution. Details on how this Task list is build can be found in :ref:`renderer_tasks`.
+
+Renderer 
+~~~~~~~~
+
+A Renderer in Synfig receives a Task list, processes it, and runs it. Synfig has multiple renderers, like Draft SW, Preview SW, etc. Currently, there are only Software Renderers in Synfig. ``Tasks`` are specialized based on the chosen renderer. This is because Software Tasks can not be directly run using GPU. The base class ``Renderer`` does most of the work. It is responsible for Optimizing the Task List, constructing the Tasks, and then sending them to the Render Queue. More details in :ref:`renderer_queue`.
+
+Render Queue
+~~~~~~~~~~~~
+
+There is a singleton Render Queue always waiting for new Tasks to run. It creates threads that are always waiting for new Tasks. More details in :ref:`renderer_queue`.

docs/renderer_docs/optimizers.rst

@@ -0,0 +1,93 @@
+.. _renderer_optimizers:
+
+Optimizers
+==========
+
+Optimizers can be found in ``synfig-core/src/synfig/rendering/common/optimizer``. They derive from the base class ``Optimizer``.
+
+Renderer::optimize
+~~~~~~~~~~~~~~~~~~
+
+This function is responsible for running all the optimizations. It needs to take care of many things like handling changes to the ``Task::List`` by optimizers. The overall functions pseudocode looks like this:
+
+.. code-block:: cpp
+
+
+    Task::List list; // task list to optimize
+
+    while(categories_to_process & ALL_CATEGORIES) // each category has an ID, which is used to create its bitmask. 1 << CATEGORY_ID and all ones is ALL_CATEGORIES
+    {
+        // this doesnt have to be while loop, since prepared_category >= current_category - 1 always
+        // but it is while in code so I kept it like this
+        while(prepared_category_id < current_category_id)
+        {
+            switch(++prepared_category_id) {
+                // if theres some step required before running the categories optimizers do it here
+                // example:
+                case SPECIALIZED:
+                    specialize(list); break; // specialize tasks before running optimizers which work on specialized tasks
+            }
+        }
+
+        // check if we need to process this category, if not then skip
+        if(!((1 << current_category_id) & categories_to_process))
+        {
+            // reset indexes
+            optimizer_index = 0;
+            current_category_id++;
+        }
+
+        Optimizer::List optimizers; // list of optimizers to run, depending on whether this category allows simultaneous run or not, if is a list of multiple optimizers or just one
+
+        // run all for_list optimizers
+        for(auto opt in optimizers)
+        {
+            if(opt->for_list)
+            {
+                opt->run(params);
+            }
+        }
+
+        // for_task are recursive unless specefied by the task after running once
+        bool nonrecursive = false;
+
+        // run all for_task/for_root_task optimizers
+        for(auto task in list)
+        {
+            Optimizer::RunParams params; // create params from list 
+            Renderer::optimize_recursive(optimizers, params, for_root_task ? 0 : nonrecursive ? 1 : INT_MAX); // only let it run recursively if optimizer wants
+            nonrecursive = false;
+
+            task = params.ref_task;
+            if((task.ref_mode & Optimizer::MODE_REPEAT_LAST) == Optimizer::MODE_REPEAT_LAST)
+            {
+                // dont go next
+                if(!(task.ref_mode & Optimizer::MODE_RECURSIVE)) nonrecursive = true;
+            }
+
+            if(!params.ref_task) remove_from_list(task); // and dont go next
+
+            categories_to_process |= params.ref_affects_to; // optimizer can ask to re run a category, it does that by setting ref_affects_to
+            // only go next if the optimizer does not want to repeat optimization
+        }
+
+        optimizer_index += optimizers.size();
+    }
+
+    remove_dummy(list);
+
+Renderer::optimize_recursive
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This function is responsible for running the optimizers on the task and its subtasks. It executes 4 main steps,
+
+* Call non-deep-first optimizers
+* Create a ``jump`` array, where each index stores index to next non-null sub task
+* While there is a sub task to optimize
+    * for each sub task in ``jump``
+        * Call ``optimize_recursive`` on each subtask in ``jump``
+        * Merge the result to ``params``, like ``ref_affects_to``
+        * Remove sub task from ``jump``, unless optimizer tells to repeat
+* Call deep-first optimizers
+
+It uses a ``ThreadPool::Group`` to run ``optimize_recursive`` on subtasks in parallel.

docs/renderer_docs/render_queue.rst

@@ -0,0 +1,69 @@
+.. _renderer_queue:
+
+Renderer and Render Queue
+=========================
+
+A renderer in Synfig is responsible to take a ``Task::List``, optimize it, specialize the tasks and then run them. Renderers apply different optimizations and settings on the tasks. For example, the LowRes SW renderer changes various settings like resolution, blur type, etc. to make the render faster. The Safe SW Renderer does no optimizations, so its slower than other renderers.
+
+The renderer then sends the optimized and specialized task list to the Render Queue.
+
+Renderer
+~~~~~~~~
+
+The renderer is selected by the user from Synfig Studio UI or by passing a CLI argument.
+
+Each renderer has some modes registers, these modes are used for specializing tasks. For example, a software renderer will register the ``TaskSW::mode_token`` like so,
+
+.. code-block:: cpp
+
+    register_mode(TaskSW::mode_token.handle()); // function in Renderer class
+
+Renderers derive from the ``Renderer`` class, which has most of the functionality already built in. So, creating a new renderer is as simple as,
+
+* derive from ``Renderer`` class,
+* override ``get_name()``,
+* register optimizers and mode in the constructor,
+* register renderer in ``Renderer::initialize_renderers``.
+
+Renderer::enqueue
+-----------------
+
+This function takes a ``Task::List list`` and a ``TaskEvent finish_event``. It then makes ``finish_event`` dependent on every task in the ``list``, so once all the tasks in the ``list`` are finished executing, the ``finish_event`` task is ran and it signals the rendering as finished.
+
+Before inserting the ``finish_event`` into the list, this function calls two very important functions ``optimize`` and ``find_deps``.
+
+Renderer::Optimize
+------------------
+
+This function changes the ``list`` by adding/removing/modifying tasks to improve the render times. It also calls ``linearize`` and ``specialize``. More details in :ref:`render_optimizers`.
+
+Renderer::find_deps
+-------------------
+
+This function fills ``deps`` and ``back_deps`` members of ``Task::RenderData`` for each task in the passed linearized task list. It first finds dependencies based on same target surface. Tasks are also depended on other tasks if their sub tasks share the target with the other task. Dependency direction is based on position in the linear list. Tasks that come later are dependend on the tasks that come earlier if they share taret.
+
+It also removes dependencies between tasks if their target rect is non-overlapping.
+
+Now sub tasks don't have the same target surface as their parent task. So by the logic above the parent task is not depended on the sub task, but in reality it should be. This is handled by ``Renderer::linearize``.
+
+Renderer::linearize
+-------------------
+
+This function turns the tree of tasks into a linear list. Sub tasks are inserted before the parent task in the list, and are converted into ``TaskSurface`` in the parents ``sub_task`` list. Since sub tasks come before parent tasks in the list and the ``TaskSurface`` has the same target surface as the sub task, ``find_deps`` is able to find the dependency.
+
+Render Queue
+~~~~~~~~~~~~
+
+This class is responsible for running tasks that support multithreading and those that don't. A separate thread is for running just the tasks which don't support multithreading. ``Renderer`` initializes a static ``RenderQueue``. It creates a set number of threads and calls ``process(thread_index)`` on each thread.
+
+This class uses ``std::condition_variable`` for notifying other threads when new tasks are available. And it uses ``std::mutex`` when any changes are being made to the queue.
+
+process
+-------
+
+This function picks up any task in the ``ready_tasks`` or ``single_ready_tasks`` (for tasks that don't allow multithreading). It does so by calling ``get()`` which waits on ``cond``. It then runs the task and calls ``done()`` after completion.
+
+done
+----
+
+This function goes through all the tasks that depend on the completed task and then removes the completed task from their dependency. If no more dependencies exist, it inserts the task to ``ready_tasks`` or ``single_ready_tasks``. After that, it calls ``cond.notify_one()`` some times, which depends on the number of tasks added to ``ready_tasks``. A similar thing is done for ``single_cond``.

docs/renderer_docs/target_surface.rst

@@ -0,0 +1,83 @@
+.. _renderer_target_surface:
+
+Target and Surface
+==================
+
+Synfig supports rendering to different file formats and uses different modules for writing to those file formats. These modules are called Targets. They inherit from the ``Target`` class and can be selected by the user or automatically (depending on the file extension).
+
+Selecting Target
+~~~~~~~~~~~~~~~~
+
+Synfig stores a dictionary(key-value pair) of all available targets with their name as the key and a factory function as value(``book``). It keeps another dictionary where the file extension to which this target can write is used as the key and the target's name as value(``ext_book``).
+
+Macros are used to fill these dictionaries; check ``synfig-core/src/modules/mod_imagemagick/main.cpp`` and ``synfig-core/src/synfig/module.h``.
+
+Rendering to a Target
+~~~~~~~~~~~~~~~~~~~~~
+
+Synfig calls the ``Target::render(...)`` function to start the rendering process. The function is responsible for rendering each frame and then writing the output files. Progress is reported using ``ProgressCallback`` passed as the function parameter.
+
+Target_Scanline
+---------------
+
+``Target_Scanline`` is the base class for most targets. It writes the output of Tasks to files line by line. The frame-by-frame render loop looks like this:
+
+.. code-block:: cpp
+
+    do
+    {
+        frame = next_frame(t); // increase frame and time, returns 0 when no frames left
+
+        start_frame();
+
+        // create a new surface
+        surface = new SurfaceResource();
+
+        // build and execute tasks
+        call_renderer();
+
+        for(int y = 0; y < height; y++)
+        {
+            Color* colorData = start_scanline(y);
+            // fill values from surface to colorData
+            end_scanline(); // finally write scanline(colorData) to file
+        }
+
+        end_frame();
+    } while(frame);
+
+The functions ``start_scanline`` and ``end_scaline`` are overridden by modules. The actual data is written to file in these functions only.
+
+Surface
+~~~~~~~
+
+See file ``synfig-core/src/synfig/surface.cpp``.
+
+Tasks exchange pixels using Surfaces. Tasks do not write to Targets directly. They write on Surfaces given to them by the Targets. Surfaces store actual pixel data. For OpenGL, a surface is like a Framebuffer.
+
+The ``Surface`` base class only declares essential virtual functions like ``create_vfunc`` for creating a new Surface of this type, ``assign_vfunc`` for assigning data from another surface to this surface, etc.
+
+Since the Cobra engine is multi-threaded and supports different render engines(ex. software and hardware), there are a few requirements that Surfaces must meet:
+
+* Reading and writing from multiple threads with proper locking mechanisms must be possible.
+* There should be an easy way to convert Surfaces from one type to another.
+
+Thread-Safety
+-------------
+
+Synfig ensures thread-safety of Surfaces using ``std::mutex`` and ``Glib::Threads::RWLock``. (We use ``Glib::Threads::RWLock`` because we still support C++11 and unfortunately it doesn't have the same primitive). To keep locking Surfaces simple, these are not used directly but by ``SurfaceResource::LockBase``. To safely read from a Surface, all you need to do is:
+
+.. code-block:: cpp
+
+   SurfaceResource::LockRead<SurfaceSW> lock(resource); // read locks the surface, unlocks on going out of scope(desctructor called)
+
+   const Surface surface = lock->get_surface(); // calls get_surface() of SurfaceSW
+
+Conversion
+----------
+
+``SurfaceResource`` can store more than one surface. But only one of each type, i.e., when ``SurfaceResource::get_surface`` and ``SurfaceResource(surface)``  is called, it stores the surface in a map where ``surface->token`` is the key. ``surface->token`` is like a string used to distinguish/name surfaces of different types. Token is static for each surface.
+
+Conversion is mainly done by ``SurfaceResource::get_surface``. It takes multiple arguments, but its main job is to attempt to convert any available surfaces from the map into the requested surface type. It stores the conversion in the same map. When a lock is created, it converts the passed resource to the type argument and stores it.
+
+This pattern of using tokens to distinguish between types and convert from one to another can be seen multiple times in Synfig. See :ref:`renderer_tasks`; tasks use a similar pattern.