Move __CLANG_OFFLOAD_BUNDLE__ around in final shared library

[CaoZhongZ]

What would happen if I moved __CLANG_OFFLOAD_BUNDLE__ up or down inside shared library with linker script? Provided that the contents inside __CLANG_OFFLOAD_BUNDLE__ are AOT compiled into gen12 native instructions, for example.

[bader]

I think the exact location of symbols is not important. @sndmitriev, what do you think?

@CaoZhongZ, do you have any problems with your linker script?

[sndmitriev]

Yes, I agree, it should not be important.

[CaoZhongZ]

Can't believe I have dug up this again, Hi @bader I know that we have huge device binary support, how could we do it in Windows? Any suggestions?

[bader]

@CaoZhongZ, let me check if I got your question correctly. Are you asking if we can add Windows support for -flink-huge-device-code option?
If so, could you please file a feature request.
CC-ing @AlexeySachkov, @mdtoguchi and @asudarsa for awareness.

[CaoZhongZ]

Yes, thanks @bader

[AlexeySachkov]

We did an investigation about what can we do to support huge binaries on Windows and I think that we ended up with conclusion that it would be way harder than on Linux and it never went through our prioritization.

If you look at Windows PE format you can see that all section sizes are 4-byte integers, meaning that the maximum section (and whole binary!) size is 4 GB.

Well, there is even a phrase that limit is just 2 GB in that document:

PE32+ images allow for a 64-bit address space while limiting the image size to 2 gigabytes.

We could do some magic and append device code after the executable without touching its main headers, but I suppose that would require a custom tool to do such packing. Another approach would be to add a compiler flag to outline device code into a separate file instead of embedding it into the binary. That file doesn't have to be in PE format, i.e. we can use some custom format and custom loading for it, which should allow to support bigger binaries.

I agree with @bader that if you are interested, a feature request should be opened, but it won't be implemented quick.

[CaoZhongZ]

Thanks @AlexeySachkov! Yes, the problem is similar to old one on ELF while offload bundle stuck between GOT and data sections, PE32+ requires that loadable image must be in 2GB range, however there is no linker script on windows. wondering if lld support linker script on windows 🤔

Anyway, take time, now we are at the border of 1.5GB and sometimes problem emerge when debug or some options were enabled. Feel there might be problem in near future.