runtime/jit-rt: revive and migrates the whole thing to OrcJIT v2 #4774

liushuyu · 2024-11-07T17:19:13Z

This pull request migrates the dynamic compile system to LLVM OrcJIT v2 (the newer "v2" version from post-LLVM 16 with opaque pointers, not the older "v2" version from the LLVM 12 ~ 15 era).

The implementation is currently in very rough shape, and I am not entirely sure what I am doing. However, the "Hello World" example in the documentation has started to work now.

liushuyu · 2024-11-07T17:20:10Z

runtime/jit-rt/cpp-so/optimizer.cpp

-#include "llvm/IR/LegacyPassManager.h"
-#include "llvm/IR/Module.h"
-#include "llvm/IR/Verifier.h"
+//===-- optimizer.cpp -----------------------------------------------------===//


This is copied almost verbatim from gen/optimizer.cpp.

Maybe we can use a single file for both projects then, with a little predefine if needed.

kinke · 2024-11-07T22:34:29Z

Hehe, looks like one's motivated, nice! - In case you haven't seem them, there are quite a few lit-tests for this feature: https://github.com/ldc-developers/ldc/tree/master/tests/dynamiccompile. No idea if anything can still be salvaged from #3184, Ivan's WIP from 5 (!) years ago by now.

liushuyu · 2024-11-07T22:47:58Z

Hehe, looks like one's motivated, nice! - In case you haven't seem them, there are quite a few lit-tests for this feature: https://github.com/ldc-developers/ldc/tree/master/tests/dynamiccompile. No idea if anything can still be salvaged from #3184, Ivan's WIP from 5 (!) years ago by now.

I see. I did not know there was another pull request doing the same thing from 5 years ago. Now that I see they made very similar changes before, I realize I duplicated a fair amount of changes (but this seems like a form of cross-validation? Two independent implementations look very similar).

liushuyu · 2024-11-09T00:49:09Z

there are quite a few lit-tests for this feature: https://github.com/ldc-developers/ldc/tree/master/tests/dynamiccompile.

Now, 100% of those LIT tests are passing (with LLVM 18).

kinke · 2024-11-10T14:49:34Z

Very nice progress! - I've pushed 2 commits to reduce the remaining failures on aarch64.

kinke · 2024-11-11T19:26:50Z

tests/dynamiccompile/bind_bool.d

@@ -28,7 +28,7 @@ void main(string[] args)
    settings.optLevel = 3;
    settings.dumpHandler = (DumpStage stage, in char[] str)
    {
-      if (DumpStage.FinalAsm == stage)
+      if (DumpStage.FinalAsm == stage || DumpStage.OptimizedModule == stage)


Note that this doesn't just output the optimized IR now too in case the test fails, but also makes the indexOf substring searches now also search in the IR.

Note that this doesn't just output the optimized IR now too in case the test fails, but also makes the indexOf substring searches now also search in the IR.

Yes, LLVM on arm64 uses very clever optimizations, and the constants do not appear in the assembly (because LLVM loads the constant into the register using bit shifts).

kinke · 2024-11-11T21:31:56Z

In my Windows VM, the tests don't hang as they do for CI (with a pushed Win32 fix wrt. __chkstk, now on 32-bit Windows too). Instead, some dynamic-compile tests pass (on both 32 and 64-bit Windows), others fail with errors like:

dynamiccompile/simd_simple_opt.d:

64-bit:

Dynamic compiler fatal: Symbol not found in jitted code: "�.jit_bind" (".jit_bind")

32-bit:

Incorrect number of arguments passed to called function!
  call x86_stdcallcc void @"\01__D3ldc15dynamic_compile__T4bindTPFG16hZG32hTxG16hZQyFQvxQlZ7wrapperFSQCpQCo__TQCaTQByTxQBqZQCnFQClxQCcZ7ContextQCzZQCy"(ptr inreg noalias align 1 %3, ptr byval(%"ldc.dynamic_compile.bind!(ubyte[32] function(ubyte[16]), const(ubyte[16])).bind.Context") align 4 %4) #0
LLVM ERROR: Broken module found, compilation aborted!

dynamiccompile/simple.d:

64-bit:

Dynamic compiler fatal: Symbol not found in jitted code: "�_D6simple3bazFZv" ("_D6simple3bazFZv")

32-bit:

Dynamic compiler fatal: Symbol not found in jitted code: "�__D6simple3barFZi" ("__D6simple3barFZi")

Looks like there's some trouble with the \1 (�) mangle prefix on Windows, which is used to make LLVM not apply any extra mangling (e.g., extra leading underscore on Win32, and vectorcall/stdcall suffix on Win64).

Edit: We do use \1.jit_bind in general for all targets (https://github.com/liushuyu/ldc/blob/9531ab3828e6b4d1fe675cfcc15085d4ce948155/runtime/jit-rt/cpp-so/bind.cpp#L47-L48), but it's only a problem on Windows. So maybe not name/mangle-related after all, but some other Windows-specific issue.

liushuyu · 2024-11-11T22:26:33Z

dynamiccompile/simple.d:

64-bit:

Dynamic compiler fatal: Symbol not found in jitted code: "�_D6simple3bazFZv" ("_D6simple3bazFZv")

I have tried to debug this problem on a Windows 11 installation with Visual Studio 2022 (Community Edition).
It seems like either our dynamic compile mechanism setup LLVM in the wrong way, or there is an LLVM bug on Windows: the symbols were not materialized when they should be (LLVM on Unix platforms will materialize the JIT symbols during the lookup, i.e. lazily compile/resolve the symbols).

tests/dynamiccompile/simd_simple_opt.d

liushuyu · 2024-11-12T02:25:47Z

On Ubuntu 20.04 aarch64:

********************
Unexpectedly Passed Tests (1):
  LDC :: dynamiccompile/throw.d

But it's still broken on macOS aarch64 unfortunately

... currently only supports LLVM 18+ Co-Authored-By: Ivan <[email protected]>

To avoid Apple linker errors on arm64 wrt. misaligned pointers. AFAICT, there's no need to pack the struct, just use natural alignment. Bump the LDC_DYNAMIC_COMPILE_API_VERSION accordingly.

... LLVM can be very clever about loading values into registers on aarch64

Fix 'unresolved external symbol ___chkstk' by NOT letting LLVM add an implicit extra leading underscore - the final mangle is `__chkstk` on both 32/64-bit Windows.

... the current JIT linker

liushuyu · 2024-11-17T04:57:57Z

It seems like I may have discovered several bugs in LLVM that are exclusive to Windows. I have currently worked around them because I am not entirely sure if they are "bugs" or "intended behaviors".

... also adds a batched lookup function

... there seems to be some LLVM behavior issues on Windows, where looking up a symbol on the second time will crash either LLVM itself or corrupts the symbol address obtained from LLVM

... JITLink does not work very well for the code patterns generated by LDC

liushuyu commented Nov 7, 2024

View reviewed changes

liushuyu force-pushed the orcjit-v2 branch from 5a88686 to b0a0da9 Compare November 9, 2024 00:47

liushuyu force-pushed the orcjit-v2 branch 4 times, most recently from 2c90767 to bd2c4b5 Compare November 9, 2024 19:28

liushuyu force-pushed the orcjit-v2 branch 2 times, most recently from 465a543 to 8cbe0d1 Compare November 11, 2024 18:15

kinke reviewed Nov 11, 2024

View reviewed changes

liushuyu force-pushed the orcjit-v2 branch 2 times, most recently from 60fedcc to 36bd753 Compare November 12, 2024 02:05

liushuyu commented Nov 12, 2024

View reviewed changes

tests/dynamiccompile/simd_simple_opt.d Show resolved Hide resolved

liushuyu and others added 11 commits November 16, 2024 21:43

runtime/jit-rt: migrate the whole thing to OrcJIT v2 ...

417cb01

... currently only supports LLVM 18+ Co-Authored-By: Ivan <[email protected]>

cmake: enable dynamic compile for LLVM 18 ~ 19

aa7d633

dynamic_compile.d: fix several minor warnings wrt in storage class

e05245e

jit-rt: handle stack return arguments

2a2932f

tests/dynamiccompile: add a new test case simd_simple_opt.d

ea961cf

dynamic-compile: Don't use *packed* RtCompileModuleList IR struct

97b8d43

To avoid Apple linker errors on arm64 wrt. misaligned pointers. AFAICT, there's no need to pack the struct, just use natural alignment. Bump the LDC_DYNAMIC_COMPILE_API_VERSION accordingly.

tests/dynamiccompile/asm_output.d: Add ARM support

adf33e1

dynamic-compile: Fix registerBindPayload() signature on C++ side

72cabda

jit-rt: quick hack around function type ABI rewrites on aarch64

2969694

tests: check LLVM IR for canary values in several tests ...

9485989

... LLVM can be very clever about loading values into registers on aarch64

dynamic-compile: Fix __chkstk dummy on 32-bit Windows

5fc48f2

Fix 'unresolved external symbol ___chkstk' by NOT letting LLVM add an implicit extra leading underscore - the final mangle is `__chkstk` on both 32/64-bit Windows.

liushuyu added 3 commits November 16, 2024 21:43

jit-rt: add support for experimental LLVM JITLink linker

b790b22

jit-rt: add aarch64 eh_personality thunk if LLVM JITLink is not ...

3c9cb7e

... the current JIT linker

cmake: use LLVM JITLink by default on well-supported platforms

7f6f8ed

liushuyu force-pushed the orcjit-v2 branch from 36bd753 to 20f2121 Compare November 17, 2024 04:44

liushuyu added 3 commits November 16, 2024 22:42

jit-rt: integrate mangler with lookup function ...

36d9749

... also adds a batched lookup function

jit-rt: workaround symbol lookup problems on Windows ...

d435b34

... there seems to be some LLVM behavior issues on Windows, where looking up a symbol on the second time will crash either LLVM itself or corrupts the symbol address obtained from LLVM

cmake: disable LLVM JITLink on Windows by default ...

849c1ca

... JITLink does not work very well for the code patterns generated by LDC

liushuyu force-pushed the orcjit-v2 branch 2 times, most recently from 297af71 to 6662b46 Compare November 17, 2024 06:13

lit.site.cfg.in: set a 60s timeout for each LIT test

e41d3b4

liushuyu force-pushed the orcjit-v2 branch 2 times, most recently from f2266e0 to b0606f7 Compare November 17, 2024 19:51

jit-rt: batched symbol lookup

045bf6a

liushuyu force-pushed the orcjit-v2 branch from b0606f7 to 045bf6a Compare November 17, 2024 19:51

jit-rt: do not lookup cached symbols on macOS

ac21071

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime/jit-rt: revive and migrates the whole thing to OrcJIT v2 #4774

runtime/jit-rt: revive and migrates the whole thing to OrcJIT v2 #4774

liushuyu commented Nov 7, 2024 •

edited

Loading

liushuyu Nov 7, 2024

kinke Nov 7, 2024

kinke commented Nov 7, 2024

liushuyu commented Nov 7, 2024

liushuyu commented Nov 9, 2024

kinke commented Nov 10, 2024

kinke Nov 11, 2024 •

edited

Loading

liushuyu Nov 11, 2024

kinke commented Nov 11, 2024 •

edited

Loading

liushuyu commented Nov 11, 2024 •

edited

Loading

liushuyu commented Nov 12, 2024

liushuyu commented Nov 17, 2024

runtime/jit-rt: revive and migrates the whole thing to OrcJIT v2 #4774

Are you sure you want to change the base?

runtime/jit-rt: revive and migrates the whole thing to OrcJIT v2 #4774

Conversation

liushuyu commented Nov 7, 2024 • edited Loading

liushuyu Nov 7, 2024

Choose a reason for hiding this comment

kinke Nov 7, 2024

Choose a reason for hiding this comment

kinke commented Nov 7, 2024

liushuyu commented Nov 7, 2024

liushuyu commented Nov 9, 2024

kinke commented Nov 10, 2024

kinke Nov 11, 2024 • edited Loading

Choose a reason for hiding this comment

liushuyu Nov 11, 2024

Choose a reason for hiding this comment

kinke commented Nov 11, 2024 • edited Loading

liushuyu commented Nov 11, 2024 • edited Loading

liushuyu commented Nov 12, 2024

liushuyu commented Nov 17, 2024

liushuyu commented Nov 7, 2024 •

edited

Loading

kinke Nov 11, 2024 •

edited

Loading

kinke commented Nov 11, 2024 •

edited

Loading

liushuyu commented Nov 11, 2024 •

edited

Loading