[DISCUSSION] MobileNetV2 train and quantize

[Jakub-Bielawski]

Hello!
I'm wondering how to train a MobileNetV2 and fully quantize the tflite model. I know that I can do it in EdgeImpulse but I want to change the architecture a bit.

I'm using the tf2.3 version and mobile net from tf.keras.applications.MobileNetV2. While converting compiling model I got an error

Didn't find op for builtin opcode 'SHAPE' version '1'

Failed to get registration from op code SHAPE

Failed starting model allocation.

AllocateTensors() failed
Could not set up compiler
I'm wondering how did you do that. I'll be very thankfull for any advice

[rafzi]

This is an error thrown by the TFLite Micro Interpreter, which is also used in this tool.

Are you able to use your model with the normal TFLite Micro Flow? If no, then your model does not seem to be compatible with your version of tensorflow. If yes, please make sure to use the same version of tensorflow in the testing flow and the tflite_micro_compiler.

[Jakub-Bielawski]

I've checked it and normal TFLite Micro interpreter can not inference successfuly. I saw in commit 538aafd that it's way to work with MobileNetV2, I have read all stuff I found about quantizing MobileNetV2 and still don't know wich tf version should I use. So mayby You can help me with create flow for MobileNetV2 quantizing and compiling. I'll be very thankfull for any advice :) It will be awesome if you can link or add some template workflow or source code for correct implementation, training and quantization.

[cpetig]

Hi Jakub,
given the easy implementation of SHAPE (returns the shape of another tensor) this will most likely not be the most difficult issue. But I suspect that a network which contains SHAPE has other unsupported tflite operators as well (and probably dynamic shapes which aren't supported by tflite, IIRC).
Adding an operator is quite easy, you look into the tensorflow lite reference operator and add the missing implementation file (tensorflow/lite/kernels/shape.cc) to the tflite library (or directly your program).
Also when it gets to quantization, the best results are available via QAT (quantization aware training) which is not straightforward.
Good luck, Christof

[Jakub-Bielawski]

Hi Christof!,
I'll try with implementing the SHAPE layer, we'll see what error I'll get next :D As I understand you haven't successfully quantize MobileNetV2, just V1 ? Because I'm a bit confused about names of files like (mobilenet2.cc). Have you got some useful repositories or examples of workflow with QAT? I'll try with the Tensorflow examples but I know that sometimes they didn't work well.

[cpetig]

Hi Jakub, I fear that the 2 in my mobilenet example file name is just to indicate the second variant (compiled vs interpreted) and the model in use is MobileNet_v1_0.25_160_quant from this page https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md .
I see the google also provides pretrained 8bit quantized V2 and V3 models at https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet
I hope these links help you, Christof

[Jakub-Bielawski]

Thank you a lot, I'll definitly check it. I don't know how I can miss this obvious about 2, sorry for this :)