TFLite: Stable Delegate: how to use stable_delegate with python API

[suyash-narain]

I am using tflite 2.16.1

My target is an aarch64 linux system

I have a tflite stable_delegate .so shared library, and a stable_delegate settings json file called 'stable_delegate_settings.json which I can execute using TFLite benchmark model using the commands

./benchmark_model --graph=model.tflite --stable_delegate_settings_file=stable_delegate_settings.json

this executes the model using stable delegate through the json stable_delegate settings file.

I want to know how can I use this stable_delegate_settings.json file to execute the model via stable delegate using python API.

I am not sure whats the format to execute stable delegate using tflite.load_delegate, since it searches for the .so shared library, not the json settings file.

thanks

[suyash-narain]

Hi @sushreebarsa
any updates on this?
thanks

[sushreebarsa]

@suyash-narain
tflite.load_delegate directly searches for the shared library (.so file) and doesn't handle settings files like stable_delegate_settings.json. There's a workaround to achieve your goal using the Python API.
Could you try to Parse the stable_delegate_settings.json file;
Firstly Import the json library:

import json

then Open the stable_delegate_settings.json file:

with open('stable_delegate_settings.json', 'r') as f:
  settings = json.load(f)

Thank you!

[suyash-narain]

Hi @sushreebarsa
thank you for your reply.

So do I need to specify this 'settings' somewhere in my python code, or will the delegate automatically get loaded and execute the tflite model?

since interpreter api loads external shared delegate library .so files using tfltie experimental load_delegate function...
so does it need to be provided this settings variable with loaded json contents?

i mean, in the below format, how will i add this settings file after its parsed?

tf.lite.Interpreter(
    model_path=None,
    model_content=None,
    experimental_delegates=None,
    num_threads=None,
    experimental_op_resolver_type=[tf.lite.experimental.OpResolverType.AUTO]
)

secondly, stable delegate does not make use of tflite_plugin_create_delegate or tflite_plugin_destroy_delegate since it has its own stable delegate settings... but if i pass it through python interpreter api, it forces it to search for tflite_plugin_create_delegate or tflite_plugin_destroy_delegate.

do you know of a workaround to make stable delegate work with python?

thanks

[sushreebarsa]

@suyash-narain If you're using a custom delegate library with its own settings, follow the TFLite delegate loading documentation to incorporate it. There isn't a well-established workaround to directly use the stable delegate with the Python API for TFLite.
Thank you!