Native Component

This page contains the documentation of the native component.

CAFF Parser

The task of the native component is to parse CAFF files. Our take on this task was that the native component should be able to parse a CAFF file and generate a preview image from it. The format of the preview is BMP for the simplicity of the file format.

The native component is implemented in C, in the native folder of the root of the repository. The implementation has the following public functions that are provided by the library:

• CIFF_RES ciff_parse(const unsigned char *buffer, unsigned long long size, CIFF **ciff): Parses a CIFF file that is loaded in the size long buffer and moves the parsed CIFF to ciff. The function returns an error code that indicates whether the parsing was successful, or the parsing failed. Note: This function allocates memory, and it is the responsibility of the caller to free it with calling free_ciff. The possible return values:
  • CIFF_OK: The parsing process was successful, the parsed ciff is available in ciff
  • CIFF_FORMAT_ERROR: The CIFF file contains errors.
  • CIFF_SIZE_ERROR: The size of the CIFF image is out of the supported range.
  • CIFF_MEMORY: The implementation was not able to reserve enough memory.
• void ciff_free(CIFF *ciff): Releases the resources of ciff.
• void ciff_to_bmp(const CIFF *ciff, unsigned char **bmp, unsigned long long *file_size): Converts the CIFF image ciff to a BMP file, and outputs the BMP in the buffer bmp of size file_size. Note: This function allocates memory, and it is the responsibility of the caller to free the bmp buffer.
• CAFF_RES caff_parse(const unsigned char *buffer, unsigned long long size, CAFF **caff): Parses a CAFF file that is loaded in the size long buffer and moves the parsed CAFF to caff. The function returns an error code that indicates whether the parsing was successful, or the parsing failed. Note: This function allocates memory, and it is the responsibility of the caller to free it with calling free_caff. The possible return values:
  • CAFF_OK: The parsing process was successful, the parsed caff is available in caff
  • CAFF_FORMAT_ERROR: The CAFF file contains errors.
  • CAFF_SIZE_ERROR: The size of the CAFF image is out of the supported range.
  • CAFF_MEMORY: The implementation was not able to reserve enough memory.
• void caff_free(CAFF *caff): Releases the resources of caff.
• void caff_preview(const CAFF *caff, unsigned char **bmp, unsigned long long *file_size): Converts the CAFF image caff to a BMP file, and outputs the BMP in the buffer bmp of size file_size. Note: This function allocates memory, and it is the responsibility of the caller to free the bmp buffer.

Integrating the CAFF parser to Java

The server side component is written in Java, so the parser needs to communicate with it. The standard way of communicating with C code from Java is to use the JNI (Java Native Interface).

The JNI wrap for the parser is implemented in the jni folder. The implementation exposes the CAFF java class which calls the needed native functions. The public interface of the class:

• public static CAFF from(final ByteBuffer buffer): Parses the CAFF in buffer.
• public ByteBuffer generatePreview(): Generates the BMP preview of the CAFF and returns it as a buffer.
• public void close(): Closes the resource and frees the native resources.

As a further security measure, the native component should be run in a different process (preferably on a different host) from the Spring server. To communicate between the processes, we decided to use GRPC with the following schema:

package caff_parser;

service CaffParser {
  rpc PreviewCaff (PreviewRequest) returns (PreviewResponse) {}
}

message PreviewRequest {
  required bytes caff = 1;
}

message PreviewResponse {
  optional bytes bmp = 1;
  optional string errorMessage = 2;
}

The GRPC server that exposes this service and calls the native component is implemented in the native-component folder of the root of the repository.

Limitations of the implementation

The CIFF format encodes both the width and the height of the picture on 8 bytes. However, representing an image with both width and heigth of 2^64 would require more than 36 exabytes of RAM, which is unrealistic assumption.

Due to this observation, we decided to limit the supported image sizes to maximum 8K resolution (7680 x 4320 pixels). Moreover, to limit the data flowing through our networks, and to provide acceptable response times we also decided to limit the processed image sizes to be maximum 10 MB.