Getting libfabric to work with multiple providers

[matthewjmc]

Hi,

We are trying to use libfabric for our application, but we have issues using non TCP providers, like verbs. We are using InfiniBand, and we have already configured both IPoIB and the SM. We can get normal TCP working over the ib interface. Additionally RDMA CM by itself connects fine but our libfabric implementation does not. I was wondering if there were any common mistakes with libfabric usage that would make tcp;ofi_rxm work but verbs;ofi_rxm not work? Any information would be greatly appreciated.

Also when we used FI_SOCKADDR instead of FI_SOCKADDR_IN the data we insert into the AV becomes corrupted in both cases, but I am not sure if this is related or not.

For reference this is an excerpt from our codebase that demonstrates this behavior.
Link to gist

Thank you in advance.