GSOC23: Renderer Docs

docs/index.rst

@@ -17,6 +17,7 @@ Welcome to Synfig developers documentation!
    community/contribution guidelines
    ide/Setting up your preferred IDE
    common/structure
+   renderer
    building/Building Synfig
    packaging/packaging
    tutorials

docs/renderer.rst

@@ -0,0 +1,29 @@
+.. _renderer:
+
+
+.. Start with intro to synfig cli and how it starts rendering
+   Introduce targets, null, tile and scanline
+   Difference between target and surface
+   Use Target_Scanline as base
+   Explain Target_Scanline::render
+   Explain Target_Scaline::call_renderer
+   Explain Canvas::build_rendering_task, reference to Task page
+   Now start with Renderer::run
+   Introduce optimizers and render queue
+   Explain render queue
+   Explain how render engines are choosen
+
+Renderer Docs
+=====================
+
+This section explains the different parts of Cobra Engine and the Algorithm which uses them to render images.
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   renderer_docs/introduction
+   renderer_docs/target_surface
+   renderer_docs/tasks
+   renderer_docs/render_queue
+   renderer_docs/optimizers

docs/renderer_docs/introduction.rst

@@ -0,0 +1,49 @@
+.. _renderer_intro:
+
+Introduction
+============
+
+There are two ways to render a Synfig document, using the **synfig** CLI tool or via **Synfig Studio**. The rendering code is the same, but some concepts would be easier explained using the CLI tool. A basic command to render a Synfig document using the CLI looks like this
+
+.. code-block:: bash
+
+   synfig $FILE -o out.png --time=0 --width=1080 --height=1920
+
+This will render only the first frame of ``$FILE`` with dimensions 1920x1080 to target *out.png*. Now Synfig supports rendering to multiple file formats. These file formats are represented as ``Target`` in Synfig's code base. The CLI does the following tasks to render a file:
+
+* Boot Synfig using ``synfig::Main``, which initializes different modules and systems.
+* Read the document file and create the internal representation of the document.
+* Extract and execute a ``Job``.
+
+The first two steps are not part of the Renderer. Therefore this section only covers the last step.
+
+Job
+~~~
+
+This class is present in CLI only, and it is simple. This class stores information used by other functions to start rendering the file. The most important fields it stores are ``synfig::RendDesc desc`` and a handle to the ``synfig::Canvas``. After all the initialization and reading of the document is done, the first step taken by the CLI is to create and fill a Job.
+
+Usually, only one Job is created, but two jobs are created if the user wants to extract Alpha to another file. These jobs are first set up and run. The setup step attempts to find the ``Target``, Render Engine to use, permissions, etc.
+
+To start rendering the file, ``job.target->render(..)`` is called. This function actually starts the rendering process.
+
+Target
+~~~~~~
+
+``Target`` represents the output file and handles the frame-by-frame rendering process. The base class ``Target`` has a few virtual methods overridden by derived classes like ``Target_Scanline``. Targets for output files are modules that derive mostly from ``Target_Scanline``. Details about how the correct ``Target`` class is acquired and the actual working of ``Target::render()`` can be found in :ref:`renderer_target_surface`.
+
+The Cobra engine is multithreaded, executing independent Tasks on different threads. The function ``Canvas::build_rendering_task`` creates a Task for rendering a frame. This function is called by ``Target::render()``.
+
+Tasks
+~~~~~
+
+Tasks are the main objects of the Cobra engine, which writes and transforms pixels. There are tasks for blending, rendering shapes, transformation, etc. Tasks can have dependencies stored in a ``sub_tasks`` list inside the class ``Task``. ``Canvas::build_rendering_task`` builds this graph of Tasks, which is then sent to the Render Engine for execution. Details on how this Task list is build can be found in :ref:`renderer_tasks`.
+
+Render Engine
+~~~~~~~~~~~~~
+
+A Render Engine in Synfig receives a Task list, processes it, and runs it. Synfig has multiple render engines, like Draft SW, Preview SW, etc. Currently, there are only Software Renderers in Synfig. ``Tasks`` are specialized based on the chosen engine. This is because Software Tasks can not be directly run using GPU. The base class ``Renderer`` does most of the work. It is responsible for Optimizing the Task List, constructing the Tasks, and then sending them to the Render Queue. More details in :ref:`renderer_queue`.
+
+Render Queue
+~~~~~~~~~~~~
+
+There is a singleton Render Queue always waiting for new Tasks to run. It creates threads that are always waiting for new Tasks. More details in :ref:`renderer_queue`.

docs/renderer_docs/optimizers.rst

@@ -0,0 +1,4 @@
+.. _renderer_optimizers:
+
+Optimizers
+==========

docs/renderer_docs/render_queue.rst

@@ -0,0 +1,4 @@
+.. _renderer_queue:
+
+Render Engine and Queue
+=======================

docs/renderer_docs/target_surface.rst

@@ -0,0 +1,83 @@
+.. _renderer_target_surface:
+
+Target and Surface
+==================
+
+Synfig supports rendering to different file formats and uses different modules for writing to those file formats. These modules are called Targets. They inherit from the ``Target`` class and can be selected by the user or automatically.
+
+Selecting Target
+~~~~~~~~~~~~~~~~
+
+Synfig stores a dictionary(key-value pair) of all available targets with their name as the key and a factory function as value(``book``). It keeps another dictionary where the file extension to which this target can write is used as the key and the target's name as value(``ext_book``).
+
+Macros are used to fill these dictionaries; check ``synfig-core/src/modules/mod_imagemagick/main.cpp`` and ``synfig-core/src/synfig/module.h``.
+
+Rendering to a Target
+~~~~~~~~~~~~~~~~~~~~~
+
+Synfig calls the ``Target::render(...)`` function to start the rendering process. The function is responsible for rendering each frame and then writing the output files. Progress is reported using ``ProgressCallback`` passed as the function parameter.
+
+Target_Scanline
+---------------
+
+``Target_Scanline`` is the base class for most targets. It writes the output of Tasks to files line by line. The frame-by-frame render loop looks like this:
+
+.. code-block:: cpp
+
+    do
+    {
+        frame = next_frame(t); // increase frame and time, returns 0 when no frames left
+
+        start_frame();
+
+        // create a new surface
+        surface = new SurfaceResource();
+
+        // build and execute tasks
+        call_renderer();
+
+        for(int y = 0; y < height; y++)
+        {
+            Color* colorData = start_scanline(y);
+            // fill values from surface to colorData
+            end_scanline(); // finally write scanline(colorData) to file
+        }
+
+        end_frame();
+    } while(frame);
+
+The functions ``start_scanline`` and ``end_scaline`` are overridden by modules. The actual data is written to file in these functions only.
+
+Surface
+~~~~~~~
+
+See file ``synfig-core/src/synfig/surface.cpp``.
+
+Tasks exchange pixels using Surfaces. Tasks do not write to Targets directly. They write on Surfaces given to them by the Targets. Surfaces store actual pixel data. For OpenGL, a surface is like a Framebuffer.
+
+The ``Surface`` base class only declares essential virtual functions like ``create_vfunc`` for creating a new Surface of this type, ``assign_vfunc`` for assigning data from another surface to this surface, etc.
+
+Since the Cobra engine is multi-threaded and supports different render engines(ex. software and hardware), there are a few requirements that Surfaces must meet:
+
+* Reading and writing from multiple threads with proper locking mechanisms must be possible.
+* There should be an easy way to convert Surfaces from one type to another.
+
+Thread-Safety
+-------------
+
+Synfig ensures thread-safety of Surfaces using ``std::mutex`` and ``Glib::Threads::RWLock``. To keep locking Surfaces simple, these are not used directly but by ``SurfaceResource::LockBase``. To safely read from a Surface, all you need to do is:
+
+.. code-block:: cpp
+
+   SurfaceResource::LockRead<SurfaceSW> lock(resource); // read locks the surface, unlocks on going out of scope(desctructor called)
+
+   const Surface surface = lock->get_surface(); // calls get_surface() of SurfaceSW
+
+Conversion
+----------
+
+``SurfaceResource`` can store more than one surface. But only one of each type, i.e., when ``SurfaceResource::get_surface`` and ``SurfaceResource(surface)``  is called, it stores the surface in a map where ``surface->token`` is the key. ``surface->token`` is like a string used to distinguish/name surfaces of different types. Token is static for each surface.
+
+Conversion is mainly done by ``SurfaceResource::get_surface``. It takes multiple arguments, but its main job is to attempt to convert any available surfaces from the map into the requested surface type. It stores the conversion in the same map. When a lock is created, it converts the passed resource to the type argument and stores it.
+
+This pattern of using tokens to distinguish between types and convert from one to another can be seen multiple times in Synfig. See :ref:`renderer_tasks`; tasks use a similar pattern.

docs/renderer_docs/tasks.rst

@@ -0,0 +1,156 @@
+.. _renderer_tasks:
+
+Layers and Tasks
+================
+
+Synfig documents are made up of layers. Synfig supports many different types of layers. Layers also have properties like opacity and z-depth. These are important when rendering. The layers are first sorted based on their depth and then rendered.
+
+All the visual information under a layer is called its ``Context``.
+
+Context
+~~~~~~~
+
+See file ``synfig-core/src/synfig/context.cpp``.
+
+In code, ``Context`` is a const iterator over a list of layers. So it supports operators like ``++`` and ``--``. When ``Canvas::build_rendering_task`` is called, it first creates a sorted list of layers: the Context. Then it calls ``Context::build_rendering_task`` on the ``Context``. ``Context::build_rendering_task`` finds the first active layer, and calls ``Layer::build_rendering_task``, and sends ``context.get_next()`` as the Context for the layer.
+
+``Layer::build_rendering_task`` calls ``Layer::build_rendering_task_vfunc``, this function is overridden by layers. It returns an **abstract** task for rendering this layer and everything below it(Context).
+
+Tasks
+~~~~~
+
+See file ``synfig-core/src/synfig/rendering/task.cpp``.
+
+Synfig has multiple engines, and not all tasks are compatible with every engine. For example, a blur task would differ for OpenGL and software rendering. Therefore, tasks need to be specialized based on the engine used. 
+
+Abstract tasks are converted to Real tasks before running. Abstract tasks are used just for defining the overall task list. In contrast, Real tasks are the actual implementation of Abstract tasks. For example, the clamp task is implemented using an Abstract task ``TaskClamp`` and a Real task ``TaskClampSW``.
+
+.. code-block:: cpp
+
+    rendering::Task::Token TaskClamp::token(
+        DescAbstract<TaskClamp>("Clamp") );
+    rendering::Task::Token TaskClampSW::token(
+        DescReal<TaskClampSW, TaskClamp>("ClampSW") );
+
+
+``DescAbstract`` tells the token that it is a token for an Abstract task. Whereas ``DescReal`` tells it that it is a token for a Real task ``TaskClampSW``, which implements the Abstract task ``TaskClamp``.
+
+Token
+~~~~~
+
+Before we understand the conversion, we need to understand Tokens in Synfig. Class ``Token`` in Synfig is used mainly for identification. It is used to create sort of an internal language for storing the different types used in Synfig.
+
+The ``Token`` class is a doubly linked list. It has ``static`` members for the first and last token. The linked list operations are handled in the Constructor and Destructor. It also stores information like parents and all parents. Tokens also have a state ``prepared``, valid only after ``prepare_vfunc`` is called. ``prepare_vfunc`` is used for extra initialization done by any derived class.
+For example, ``Task::Token`` has a map called ``alternatives_``. This map is filled when ``Task::Token.prepare_vfunc`` is called.
+
+This initialization is started by ``Token::rebuild``, called by ``Synfig::Main``.
+
+In most cases, classes have a synfig static member variable called ``token``, which is redeclared by every derived class. You will also see that these classes have a virtual function ``get_token``, which is implemented by derived classes and returns the redeclared token variable.
+
+The base class ``Token`` doesn't store any information. The derived classes usually store more information; for example, they can store information for converting between types.
+
+Task::Token
+-----------
+
+``Task::Token`` derieves from both ``Synfig::Token`` and ``Task::DescBase``. ``Task::DescBase`` is a class that stores some function pointers and other information useful during specialization. There are function pointers for creating, cloning, and converting. ``DescAbstract``, ``DescReal``, etc., are derived classes from ``DescBase``, and they assign the function pointers using the type arguments.
+
+Looking at the ``TaskClamp`` example again, the code above initializes ``TaskClamp::token`` using a ``DescAbstract<TaskClamp>``. An abstract can be created and cloned but not converted from another task. So, the ``convert`` function pointer is null, whereas the ``create`` function pointer is set using ``DescBase::create_func`` and the ``clone`` function pointer is set using ``DescBase::convert_func`` which copies in a case like this. ``mode`` is assigned an empty handle for abstract tasks.
+
+Then it initializes ``TaskClampSW::token`` using a ``DescReal<TaskClampSW, TaskClamp>``, which sets the convert function pointer using ``DescBase::convert_func``. It stores ``TaskClamp::token`` in ``abstract_task`` member variable. ``mode`` is assigned value of ``TaskClampSW::mode_token.handle()``. Then the most important step is done, in ``prepare_vfunc`` of ``Task::token`` if the task is a Real task, then its abstract task's ``alternative_`` map is filled, i.e., ``abstract_task.alternatives()[mode] = _Handle(*this)``. The abstract task is ``TaskClamp`` in this case. ``mode`` is explained in the next section.
+
+Then an abstract task can be easily converted to a Real task given a mode by using ``alternatives_[mode]->convert(*this)``. ``alternatives_[mode]`` is storing the token of ``TaskClampSW`` in this example.
+
+Specialization
+~~~~~~~~~~~~~~
+
+Tasks in Synfig are specialized based on ``mode``. Currently, in Synfig, there is only one mode ``TaskSW`` because there is only a software renderer. Each rendered has some modes associated with it. This gives users the ability to run only software or hardware tasks. It's not like hardware tasks cannot work with software tasks. They can work, which is possible due to automatic surface conversion. But it's faster if dependent tasks are of the same mode.
+
+So, how does a renderer know which task runs on which mode? Real tasks derive from the ``Mode`` class, which stores a static token for its type and some additional functions which help the renderer. Now instead of deriving directly from the ``Mode`` class, software tasks derive from the ``TaskSW`` class, which derives from the ``Mode`` class. This is so that the mode token is the same for every software task.
+
+Renderers register the mode they work on using the ``register_mode`` function. A software renderers calls ``register_mode(TaskSW::mode_token.handle())``. A renderer uses the registered modes to specialize tasks before sending them to the render queue.
+
+Now that we know, what is required to implement a task, lets learn how to do it.
+
+Implementation
+~~~~~~~~~~~~~~
+
+First, we need to create an Abstract task class. This will store all the properties necessary for executing the task.
+
+.. code-block:: cpp
+
+    class MyTask : public Task
+    {
+    public:
+        typedef etl::handle<MyTask> Handle;
+        static Token token;
+        virtual Token::Handle get_token() const { return token.handle(); }
+
+        // properties/settings
+        float mul;
+
+        // virtual functions as required or redeclare as required
+
+        MyTask() : mul(0) {}
+    }
+
+Then we need to create its software implementation.
+
+.. code-block:: cpp
+
+    class MyTaskSW : public MyTask, public TaskSW
+    {
+    public:
+        typedef etl::handle<MyTaskSW> Handle;
+        static Token token;
+        virtual Token::Handle get_token() const { return token.handle(); }
+
+        virtual bool run(RunParams &params) const;
+    }
+
+Then we need to initialize ``MyTask::token`` and ``MyTaskSW::token`` in a cpp file.
+
+.. code-block:: cpp
+
+    rendering::Task::Token MyTask::token(
+        DescAbstract<MyTask>("MyTask") );
+    rendering::Task::Token MyTaskSW::token(
+        DescReal<MyTaskSW, MyTask>("MyTaskSW") );
+
+Implementation of ``run`` for ``ClampSW`` looks like,
+
+.. code-block:: cpp
+
+    bool
+    TaskClampSW::run(RunParams&) const
+    {
+        RectInt r = target_rect;
+        if (r.valid())
+        {
+            VectorInt offset = get_offset();
+            RectInt ra = sub_task()->target_rect + r.get_min() + get_offset();
+            if (ra.valid())
+            {
+                rect_set_intersect(ra, ra, r);
+                if (ra.valid())
+                {
+                    LockWrite ldst(this); // lock target surface of this task
+                    if (!ldst) return false;
+                    LockRead lsrc(sub_task()); // lock target surface of sub_task, assumes only 1 sub task
+                    if (!lsrc) return false;
+
+                    const synfig::Surface &a = lsrc->get_surface();
+                    synfig::Surface &c = ldst->get_surface();
+
+                    for(int y = ra.miny; y < ra.maxy; ++y)
+                    {
+                        const Color *ca = &a[y - r.miny + offset[1]][ra.minx - r.minx + offset[0]];
+                        Color *cc = &c[y][ra.minx];
+                        for(int x = ra.minx; x < ra.maxx; ++x, ++ca, ++cc)
+                            clamp_pixel(*cc, *ca);
+                    }
+                }
+            }
+        }
+
+        return true;
+    }