ONNX conversion?

[tripathiarpan20]

Greetings!

I happened to notice you mentioning ONNX conversion in the original basic-pitch repo (spotify/basic-pitch#64 (comment)), I was hoping you could provide some insights on the same, and if possible provide some relevant code.

Was hoping to deploy it fully on the web using ONNXruntime with WebGPU and benchmark the speedup.

References:
https://onnxruntime.ai/docs/tutorials/web/
https://github.com/webonnx/wonnx

Thanks in advance

[achimmihca]

Hi,

onnx conversion is not a big deal.

• The original model can be found in TensorFlow saved_model format here: https://github.com/spotify/basic-pitch/tree/main/basic_pitch/saved_models/icassp_2022/nmp
• The tf2onnx tool can convert the model to onnx: https://github.com/onnx/tensorflow-onnx
  • install via python package manager: pip install -U tf2onnx
  • run via python: python -m tf2onnx.convert --saved-model tensorflow-model-path --output model.onnx

However, this is only the neural network of the machine learning model.
What is not part of the model itself is

• load samples of an audio file
• convert the samples of the audio file to the input format for the machine learning model (typically mono channel, specific sample rate, etc.)

These steps are somewhere in the basic-pitch python code.
But I did not have enough time to understand and rewrite this for my own (C#) code. As a result, I took the shortcut of compiling Basic Pitch to an exe and just call this from my code.

But it would be great if you could write code to load audio file and create the correct inputs for the model. Good luck!