[Build] Incompatible binaries error for multiple ORT and QNN version combinations on AArch64-Linux (including ORT 1.18.1 and QNN 2.22.10)

[deivard]

Describe the issue

I am trying to use QNN execution provider on a QCS8550 based dev kit but cannot get it to work. Cross compiling from a BitBake environment to arm works fine, but during runtime I get stuck on "QNN SetupBackend failed Failed to create device. Error: 1008", which indicates incompatible binaries. I have tried multiple combinations of ORT and QNN, but cannot get anything to work. The combination I really thought would work is ORT 1.18.1 and Qnn 2.22.10, but no.

Runtime config I used is QNN:soc_model: 66 QNN:htp_arch: 73 QNN:backend_path: libQnnHtp.so for a QCS8550 chip, which should be correct.

With some combination of versions, it wont build, and for some I get "hardware unavailable error". But it seems that the furthest I have gotten is the incompatible binaries error.

Thankful for any advice or proven to work combination of ORT and QNN on Linux ARM.

Urgency

Blocker for our project.

Target platform

AArch64-Linux

Build script

Custom bitbake recipe using CMake, might not be useful to share.

Error / output

Logs that might be of interest:

[I:onnxruntime:, inference_session.cc:533 TraceSessionOptions] Session Options {  execution_mode:0 execution_order:DEFAULT enable_profiling:0 optimized_model_filepath: enable_mem_pattern:1 enable_mem_reuse:1 enable_cpu_mem_arena:1 profile_file_prefix:onnxruntime_profile_ session_logid: session_log_severity_level:-1 session_log_verbosity_level:0 max_num_graph_transformation_steps:10 graph_optimization_level:2 intra_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } inter_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } use_per_session_threads:0 thread_pool_allow_spinning:1 use_deterministic_compute:0 config_options: {   QNN:htp_graph_finalization_optimization_mode: 3  QNN:htp_performance_mode: high_performance  QNN:enable_htp_fp16_precision: 1  QNN:soc_model: 66  QNN:htp_arch: 73  QNN:backend_path: libQnnHtp.so  kOrtSessionOptionsConfigAllowInterOpSpinning: 0  kOrtSessionOptionsConfigAllowIntraOpSpinning: 0 } } 
...
...
...
[I:onnxruntime:AI, qnn_backend_manager.cc:133 LoadBackend] Found valid interface, version: 2.15.0 backend provider name: HTP_QTI_AISW backend id: 6
[I:onnxruntime:AI, qnn_backend_manager.cc:361 CreateDevice] Create device.                                                                                                                         
[E:onnxruntime:, qnn_execution_provider.cc:513 GetCapability] QNN SetupBackend failed Failed to create device. Error: 1008

Visual Studio Version

No response

GCC / Compiler Version

No response