Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for GraalVM Native Image #787

Draft
wants to merge 111 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
30ae2d8
Provide Support for GraalVM native images
DRSchlaubi Mar 19, 2023
8920cfe
Also generate reflect-config.json for other modules
DRSchlaubi Mar 19, 2023
22cd84d
[ci skip] Cleanup
DRSchlaubi Mar 19, 2023
a5e099e
Add kspProcessor to common again
DRSchlaubi Mar 19, 2023
8acc1e0
[ci skip] Remove common/test reflect-config.json
DRSchlaubi Mar 19, 2023
94a8105
FIx generation for Serializable types with uncommon names
DRSchlaubi Mar 19, 2023
9b5e1ce
Make cache data classes visible via reflection
DRSchlaubi Mar 19, 2023
5489305
Make gradlew executable
DRSchlaubi Mar 19, 2023
5ba9eb9
Initial steps for running tests
DRSchlaubi Mar 19, 2023
7dd91e0
Better name and error messages for binaryName
lukellmann Mar 20, 2023
0d7113c
Update ksp-processors/src/main/kotlin/graal/ReflectConfig.kt
DRSchlaubi Mar 20, 2023
b73a3a1
Update gradle/libs.versions.toml
DRSchlaubi Mar 20, 2023
63e1069
Improve ReflectConfig
DRSchlaubi Mar 20, 2023
ffc4d5c
Cleanup gradle files
DRSchlaubi Mar 20, 2023
08afaf1
Enable color test again
DRSchlaubi Mar 20, 2023
2e464ea
Cleanup reflect-config.json
DRSchlaubi Mar 20, 2023
f8c6e01
Remove @GraalVisible from StickerData
DRSchlaubi Mar 20, 2023
da19d34
Use new loaction for reflect-config.json
DRSchlaubi Mar 20, 2023
2f7d985
Pretty-print reflect-config.json
DRSchlaubi Mar 20, 2023
8088424
Move reflect-config.json to correct location
DRSchlaubi Mar 20, 2023
923d18f
Disable incompatible tests on Graal
DRSchlaubi Mar 20, 2023
e1660ee
Fix failing test
DRSchlaubi Mar 20, 2023
246fa0a
Add CI for GraalVM
DRSchlaubi Mar 20, 2023
e960926
Use official action
DRSchlaubi Mar 20, 2023
9399ae4
[CI skip] Remove no longer needed matrix
DRSchlaubi Mar 20, 2023
8323bcc
Fix wrong location of reflect-config.json
DRSchlaubi Mar 20, 2023
d6635a5
Bump kotlin-logging and slf4j-simple
DRSchlaubi Mar 20, 2023
133dcf9
Add missing generated files
DRSchlaubi Mar 20, 2023
b1495d3
please work
lukellmann Mar 20, 2023
39746f1
Maybe fix native tests
DRSchlaubi Mar 20, 2023
19c843b
Fix native tests
DRSchlaubi Mar 20, 2023
18cf514
Fix typo in file name
DRSchlaubi Mar 20, 2023
6b010e6
Fix wrongly configured matrix
DRSchlaubi Mar 20, 2023
965d823
Setup windows compiler toolchain
DRSchlaubi Mar 20, 2023
4b82516
Fix ci file
DRSchlaubi Mar 20, 2023
20d7c31
Trigger ci build
DRSchlaubi Mar 20, 2023
3e029f2
Use DSL instead of resource-config.json
lukellmann Mar 21, 2023
e9d0e20
Merge remote-tracking branch 'kordlib/feature/graal' into feature/graal
lukellmann Mar 21, 2023
ec0130f
More compact reflect-config.json
lukellmann Mar 21, 2023
745d108
Rename KordGraalProcessor to GraalProcessor
lukellmann Mar 21, 2023
3b57e30
Make prettyPrintIndent smaller
lukellmann Mar 21, 2023
ac8e4a6
Update GraalVisible KDoc
lukellmann Mar 21, 2023
aea9586
Update ReflectConfigEntry
lukellmann Mar 21, 2023
bcb99dc
Check if CI build passes if we skip online tests
DRSchlaubi Mar 21, 2023
b71c0b2
Run Gradle with -i
DRSchlaubi Mar 21, 2023
74dd138
Only use ubuntu for CI
DRSchlaubi Mar 21, 2023
0c49c5d
Improve logging and move entries into GraalProcessor
lukellmann Mar 21, 2023
9d6abb6
Merge remote-tracking branch 'kordlib/feature/graal' into feature/graal
lukellmann Mar 21, 2023
cf9af3d
Use only macos
DRSchlaubi Mar 21, 2023
36eb55f
Merge remote-tracking branch 'kordlib/feature/graal' into feature/graal
lukellmann Mar 21, 2023
e67685a
Use GraalVM Native Image terminology consistently
lukellmann Mar 21, 2023
92d6633
Proper KSP Dependencies
lukellmann Mar 21, 2023
d4511bd
Move companionObject to KSPUtils.kt
lukellmann Mar 21, 2023
314a71a
Deterministic order for reflect-config.json
lukellmann Mar 21, 2023
dec0101
Clean graalvm-native-image-ci.yml
lukellmann Mar 21, 2023
3f3568c
Run again on ubuntu
lukellmann Mar 21, 2023
0062325
Merge branch '0.8.x' into feature/graal
lukellmann Mar 21, 2023
0773937
Regenerate reflect-config.json files
lukellmann Mar 21, 2023
310b979
Remove import
lukellmann Mar 21, 2023
1022f52
Try to run builds sequentially
lukellmann Mar 21, 2023
10b2dcc
Also setup JDK
lukellmann Mar 21, 2023
ca80c77
Revert "Also setup JDK"
lukellmann Mar 21, 2023
ca4d99c
Exclude :voice:nativeTest
lukellmann Mar 21, 2023
0d12b94
Debug logs
lukellmann Mar 21, 2023
5470e71
Build without daemon
lukellmann Mar 21, 2023
1634c24
Explicitly specify GraalVM version and bump to Java 19
DRSchlaubi Mar 22, 2023
94e2c74
Fox test reflect-config.json
DRSchlaubi Mar 22, 2023
be5ae9e
Do not run native tests in parallel
DRSchlaubi Mar 22, 2023
586f1c3
Fix typo
DRSchlaubi Mar 22, 2023
f8df5fa
[skip ci] Use @DisabledInNativeImage instead of system property
DRSchlaubi Mar 22, 2023
21ace0d
Disable Windows for now
DRSchlaubi Mar 22, 2023
8b28408
Run matrix in parallel again and use info logging
lukellmann Mar 22, 2023
9364240
Let's switch to bleeding edge Gradle
DRSchlaubi Mar 22, 2023
9d6893b
Fix workflow file
lukellmann Mar 22, 2023
3251062
Let's switch to bleeding edge Gradle for real
DRSchlaubi Mar 22, 2023
37b2deb
Specify proper download URL
DRSchlaubi Mar 22, 2023
31870ef
TeamCity change in 'Kord' project: Versioned settings configuration u…
Mar 23, 2023
9c5c0e9
TeamCity change in 'Kord' project: runners of 'Run Native Tests on Li…
Mar 23, 2023
c84a5f1
TeamCity change in 'Kord' project: runners of 'Run Native Tests on Li…
Mar 23, 2023
b390891
Use JDK 17 Graal
DRSchlaubi Mar 23, 2023
8aa0142
Use Gradle to auto-provision GraalVm JDK 17
DRSchlaubi Mar 23, 2023
397edc3
Run Gradle with --stacktrace
DRSchlaubi Mar 23, 2023
dde30c8
Enable debug logs
DRSchlaubi Mar 23, 2023
04f73d3
Remove Graal CI
DRSchlaubi Mar 23, 2023
b3d92dd
Initial migration to TeamCity CI
DRSchlaubi Mar 23, 2023
4998050
Add mandatory ID
DRSchlaubi Mar 23, 2023
1b5cd63
TeamCity change in 'Kord' project: 'Run Native Tests on Linux' build …
Mar 23, 2023
bdb1d89
TeamCity change in 'Kord' project: 'Validation' build configuration w…
Mar 23, 2023
921b4ff
TeamCity change in 'Kord' project: VCS roots of 'Validation' build co…
Mar 23, 2023
2da76e2
Add mandatory ID
DRSchlaubi Mar 23, 2023
e4ef8b6
TeamCity change in 'Kord' project: 'https://github.com/kordlib/kord#r…
Mar 23, 2023
a70f57c
TeamCity change in 'Kord' project: 'Validation' build configuration w…
Mar 23, 2023
0918ae0
TeamCity change in 'Kord' project: commit current project settings
Mar 23, 2023
b077b16
Fix CI
DRSchlaubi Mar 23, 2023
cc0deee
Remove TeamCity patches
DRSchlaubi Mar 23, 2023
fa570a2
Fail if test count changes significantly
DRSchlaubi Mar 23, 2023
f9b2bae
Run Validation CI on pull requests as well
DRSchlaubi Mar 23, 2023
a500f50
Fix Pull request configuration
DRSchlaubi Mar 23, 2023
ae339e1
TeamCity change in 'Kord' project: Versioned settings configuration u…
Mar 23, 2023
967d52b
Merge remote-tracking branch 'origin/0.8.x' into feature/graal
DRSchlaubi Mar 23, 2023
5889bfe
Add GITHU_BRANCH environment variable
DRSchlaubi Mar 23, 2023
22d688d
Convert more CI
DRSchlaubi Mar 24, 2023
3967334
Convert Maven deployment to TeamCity
DRSchlaubi Mar 24, 2023
af3267f
Fix typo
DRSchlaubi Mar 24, 2023
5b838e5
Run live tests in TeamCity
DRSchlaubi Mar 24, 2023
1438531
Fix wrong environment variable name
DRSchlaubi Mar 24, 2023
49d0ec7
Mark environmen variable as secure
DRSchlaubi Mar 24, 2023
2ac7eaa
Fix CI secret
DRSchlaubi Mar 24, 2023
91c85d7
Stop running CI on GH-pages
DRSchlaubi Mar 25, 2023
a21bc35
Merge remote-tracking branch 'origin/0.9.x' into feature/graal
DRSchlaubi Apr 11, 2023
bc4ce7b
MPP migrations
DRSchlaubi Apr 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 0 additions & 69 deletions .github/workflows/deployment-ci.yml

This file was deleted.

39 changes: 0 additions & 39 deletions .github/workflows/docs-ci.yml

This file was deleted.

2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,7 @@ out/

**/build/*
!**/build/generated/
**/build/generated/**/test
.teamcity/target/
# re-exclude BuildConfigGenerated.kt
common/build/generated/source/buildConfig/main/main/dev/kord/common/BuildConfigGenerated.kt
27 changes: 27 additions & 0 deletions .teamcity/DebuggableGradle.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import jetbrains.buildServer.configs.kotlin.BuildSteps
import jetbrains.buildServer.configs.kotlin.buildSteps.GradleBuildStep
import jetbrains.buildServer.configs.kotlin.buildSteps.gradle

const val debugParamName = "debug"

/**
* Adds a new [Gradle build step][GradleBuildStep] that supports the debug parameter.
*/
fun BuildSteps.debuggableGradle(name: String, configure: GradleBuildStep.() -> Unit) {
gradle {
this.name = "$name (Debug)"
conditions {
equals(debugParamName, true.toString())
}
configure()
gradleParams = (gradleParams?.plus(' ') ?: "") + "-d"
}

gradle {
this.name = name
conditions {
equals(debugParamName, false.toString())
}
configure()
}
}
18 changes: 18 additions & 0 deletions .teamcity/DocsCI.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import jetbrains.buildServer.configs.kotlin.RelativeId
import jetbrains.buildServer.configs.kotlin.triggers.vcs

val DocsCI = KordBuild("Deploy documentation") {
id = RelativeId("Deploy_docs")

triggers {
vcs {
branchFilter = "+:0.8.x"
}
}

steps {
debuggableGradle("Deploy documentation") {
tasks = "gitPublishPush"
}
}
}
50 changes: 50 additions & 0 deletions .teamcity/GitHub.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import jetbrains.buildServer.configs.kotlin.BuildFeatures
import jetbrains.buildServer.configs.kotlin.DslContext
import jetbrains.buildServer.configs.kotlin.ProjectFeatures
import jetbrains.buildServer.configs.kotlin.buildFeatures.commitStatusPublisher
import jetbrains.buildServer.configs.kotlin.buildFeatures.pullRequests
import jetbrains.buildServer.configs.kotlin.projectFeatures.githubIssues
import jetbrains.buildServer.configs.kotlin.vcs.GitVcsRoot

object GitHub : GitVcsRoot({
name = "https://github.com/kordlib/kord#refs/heads/0.8.x"
url = "https://github.com/kordlib/kord"
branch = "refs/heads/0.8.x"
branchSpec = """
+:refs/heads/*
-:refs/heads/gh-pages
""".trimIndent()
authMethod = password {
userName = "DRSchlaubi"
password = "credentialsJSON:6c4cf2b5-741b-420a-bd80-587c4746b827"
}
param("oauthProviderId", "tc-cloud-github-connection")
})

fun BuildFeatures.installGitHubPublisher() = commitStatusPublisher {
vcsRootExtId = "${DslContext.settingsRoot.id}"
publisher = github {
githubUrl = "https://api.github.com"
authType = personalToken {
token = "credentialsJSON:6c4cf2b5-741b-420a-bd80-587c4746b827"
}
}
param("github_oauth_user", "DRSchlaubi")
}

fun BuildFeatures.installGitHubPullRequest() = pullRequests {
provider = github {
authType = token {
token = "credentialsJSON:8a0ab174-fa52-45d4-950a-05a3c36d0e63"
}
}
}

fun ProjectFeatures.installGitHubIssueTracker() = githubIssues {
id = "PROJECT_EXT_2"
displayName = "GitHub"
repositoryURL = "https://github.com/kordlib/kord"
authType = accessToken {
accessToken = "credentialsJSON:8a0ab174-fa52-45d4-950a-05a3c36d0e63"
}
}
9 changes: 9 additions & 0 deletions .teamcity/GraalVMNativeImageCI.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import jetbrains.buildServer.configs.kotlin.RelativeId

val GraalVMNativeImageCI = MultiOSKordBuild("Run GraalVM Native image Tests", RelativeId("Run_Native_Image_tests")) {
steps {
debuggableGradle("Run Native Image Tests") {
tasks = "nativeTest"
}
}
}
32 changes: 32 additions & 0 deletions .teamcity/OSMatrix.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import jetbrains.buildServer.configs.kotlin.AbsoluteId
import jetbrains.buildServer.configs.kotlin.BuildType
import jetbrains.buildServer.configs.kotlin.Id

/**
* Creates a matrix for [operatingSystems] and runs the build everywhere.
*
* @param name a base name for all [types][BuildType]
* (Is converted to `$name ($operatingSystem)`
* @param id a base [Id] for all [types][BuildType]
* (Is converted to `$name_$operatingSystem)
* @param operatingSystems [List] of operating systems
* @param configure configuration for all types
*/
@Suppress("FunctionName")
fun MultiOSKordBuild(
name: String,
id: Id,
operatingSystems: List<String> = listOf("Linux", "Windows", "Mac OS"),
configure: BuildType.() -> Unit
): List<BuildType> {
return operatingSystems.map { operatingSystem ->
KordBuild("$name ($operatingSystem)") {
this.id = AbsoluteId("${id.value}_${operatingSystem.replace(' ', '_').lowercase()}")
requirements {
matches("teamcity.agent.jvm.os.family", "Linux")
}

configure()
}
}
}
29 changes: 29 additions & 0 deletions .teamcity/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# .teamcity

This folder houses the [TeamCity](https://jetbrains.com/teamcity) configurations for the Kord build process

# Documentation

Documentation can be
found [here](https://www.jetbrains.com/help/teamcity/kotlin-dsl.html) ([KDoc](https://kord.teamcity.com/app/dsl-documentation/index.html))

# Importing in IntelliJ

File -> Project Structure -> Modules -> Add -> Import ->
Select [pom.xml](pom.xml) -> Select Maven

# Validating

To Validate the config format run the following Maven goal

```shell
mvnDebug org.jetbrains.teamcity:teamcity-configs-maven-plugin:generate
```

# Builds

| Name | Description | Definition |
|--------------------|----------------------------------------------------------------------|----------------------------------------------------|
| Validation | Runs checks (Test, binary compatibility) and publish Maven artifacts | [ValidationCI.kt](ValidationCI.kt) |
| Docs | Deploys documentation to GitHub Pages | [DocsCI.kt](DocsCI.kt) |
| GraalVM Validation | Runs GraalVM Native image Tests | [GraalVMNativeImageCI.kt](GraalVMNativeImageCI.kt) |
23 changes: 23 additions & 0 deletions .teamcity/Shared.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import jetbrains.buildServer.configs.kotlin.*

@Suppress("FunctionName")
fun KordBuild(name: String, configure: BuildType.() -> Unit) = object : BuildType({
this.name = name
configure()

vcs {
root(DslContext.settingsRoot)
}

params {
checkbox(debugParamName, false.toString(), "Debug Mode", "Run build with debug logging enabled")

param("env.GITHUB_BRANCH", "%teamcity.build.branch%")

password("env.KORD_TEST_TOKEN", "credentialsJSON:cbd8be1d-4808-42ff-90cf-551e9f60e83d")
}

features {
installGitHubPublisher()
}
}) {}
16 changes: 16 additions & 0 deletions .teamcity/TeamCityExtensions.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import jetbrains.buildServer.configs.kotlin.BuildType
import jetbrains.buildServer.configs.kotlin.Project
import jetbrains.buildServer.configs.kotlin.Triggers
import jetbrains.buildServer.configs.kotlin.triggers.vcs

/**
* Short-hand for only VCS triggers.
*/
fun Triggers.vcs() = vcs { }

/**
* Adds all [types] to this [Project].
*
* @see MultiOSKordBuild
*/
fun Project.buildType(types: Iterable<BuildType>) = types.forEach(::buildType)
55 changes: 55 additions & 0 deletions .teamcity/ValidationCI.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import jetbrains.buildServer.configs.kotlin.FailureConditions
import jetbrains.buildServer.configs.kotlin.RelativeId
import jetbrains.buildServer.configs.kotlin.failureConditions.BuildFailureOnMetric
import jetbrains.buildServer.configs.kotlin.failureConditions.failOnMetricChange

val ValidationCI = KordBuild("Validate Code") {
id = RelativeId("Validation")
triggers {
vcs()
}

steps {
debuggableGradle("Run checks") {
tasks = "check"
}

debuggableGradle("Publish Artifacts") {
// Secrets need to be specified
enabled = false
param("env.NEXUS_USER", "TODO")
param("env.NEXUS_PASSWORD", "TODO")
param("system.org.gradle.project.signingKey", "TODO")
param("system.org.gradle.project.signingPassword", "TODO")

conditions {
// Meaning: Do not run on Pull Requests
doesNotExist("teamcity.pullRequest.number")
}

tasks = "publish"
gradleParams = "-x test"
}
}

failureConditions {
failOnSignificantDecreaseOf(BuildFailureOnMetric.MetricType.TEST_COUNT)
failOnSignificantDecreaseOf(BuildFailureOnMetric.MetricType.TEST_IGNORED_COUNT)
}

features {
installGitHubPullRequest()
}
}

private fun FailureConditions.failOnSignificantDecreaseOf(metricType: BuildFailureOnMetric.MetricType) {
failOnMetricChange {
metric = metricType
threshold = 20
units = BuildFailureOnMetric.MetricUnit.PERCENTS
comparison = BuildFailureOnMetric.MetricComparison.LESS
compareTo = build {
buildRule = lastSuccessful()
}
}
}
Loading