Skip to content

Latest commit

 

History

History
347 lines (250 loc) · 12.1 KB

DEV.md

File metadata and controls

347 lines (250 loc) · 12.1 KB
Raw

Developer docs

Requirements

Building Scala CLI requires JVM 17 to work properly.

In theory, our build is able to download and install for its own needs JVM 17 on some OSes however it may not work in Intellij / Metals out of the box.

The Scala CLI sources ship with Mill launchers, so that Mill itself doesn't need to be installed on your system.

Common commands

Running the CLI from sources

Run the scala target with Mill:

./mill -i scala …arguments…

This is the equivalent of running the cli task with the default Scala version:

./mill -i 'cli[]'.run …arguments…

Debugging the CLI from sources

./mill -i debug debug-port …arguments…

which is short for:

./mill -i 'cli[]'.debug debug-port …arguments…

E.g:

./mill -i 'cli[]'.debug 5050 ~/Main.scala -S 3.3.0

Run unit tests

This command runs the unit tests from the build-module module.

./mill 'build-module.test'

If you want to run unit tests for another module, set module_name to the name of the module from which you want to run the unit tests:

./mill 'module_name.test'

To can filter unit test suites:

./mill 'build-module.test' 'scala.build.tests.BuildTestsScalac.*'
./mill 'build-module.test' 'scala.build.tests.BuildTestsScalac.simple'

Run integration tests with the JVM launcher

./mill integration.test.jvm

Filter test suites with

./mill integration.test.jvm 'scala.cli.integration.RunTestsDefault.*'
./mill integration.test.jvm 'scala.cli.integration.RunTestsDefault.Multiple scripts'

You can pass the --debug option to debug Scala CLI when running integration tests. Note that this allows to debug the Scala CLI launcher (the app) and not the integration test code itself. The debugger is being run in the attach mode.

./mill integration.test.jvm 'scala.cli.integration.RunTestsDefault.*' --debug

The debug option uses 5005 port by default. It is possible to change it as follows:

./mill integration.test.jvm 'scala.cli.integration.RunTestsDefault.*' --debug:5006

Run integration tests with the native launcher

(generating the launcher can take several minutes)

./mill integration.test.native
./mill integration.test.native 'scala.cli.integration.RunTestsDefault.*'

Generate JUnit test reports

As running tests with mill generates output in a non-standard JSON format, we have a script for converting it to the more well known JUnit XML test report format which we can then process and view on the CI. In case you want to generate a test report locally, you can run the following command:

.github/scripts/generate-junit-reports.sc <test suite title> <test report title> <output-path out/

The test should fail when no test reports were found or if no tests were actually run.

Generate native packages

Build native packagers:

  • deb for linux
  • msi for windows
  • dmg and pkg for macOS

(generating native packager for specified format)

./mill -i scala package ..arguments... --deb --output 'path.deb'
./mill -i scala package ..arguments... --dmg --output 'path.dmg'
./mill -i scala package ..arguments... --pkg --output 'path.pkg'

IDE Import

The Scala CLI repository should work when imported automatically from Mill to IDEA IntelliJ and Metals. Please raise an issue if you run into any problems.

When working with IntelliJ make sure that the project's Java is set correctly. To confirm, check under File -> Project Structure that:

  • in Project Settings/Project SDK and Language level is set to 17
  • in Project Settings/Modules all the modules have Language level set to 17
  • in Platform Settings/SDKs only Java 17 is visible

Otherwise, some IDE features may not work correctly, i.e. the debugger might crash upon connection.

Generate a native launcher

./mill -i show 'cli[]'.nativeImage

This prints the path to the generated native image. The file named scala at the root of the project should also be a link to it. (Note that the link is committed and is always there, whether the files it points at exists or not.)

Generate a JVM launcher

./mill -i show 'cli[]'.launcher

This prints the path to the generated launcher. This launcher is a JAR, that directly re-uses the class directories of the modules of the project (so that cleaning up those classes will break the launcher). If this is a problem (if you wish to run the launcher on another machine or from a Docker image for example), use a native launcher (see above) or a standalone JVM one (see below).

Generate a standalone JVM launcher

./mill -i show 'cli[]'.standaloneLauncher

This prints the path to the generated launcher. This launcher is a JAR, that embeds JARs of the scala-cli modules, and downloads their dependencies from Maven Central upon first launch (using the coursier cache, just like a coursier bootstrap).

Helper projects

A number of features of Scala CLI are managed from external projects, living under the scala-cli and VirtusLab organizations on GitHub. These projects can be used by Scala CLI as libraries pulled before it's compiled, but also as binaries. In the latter case, Scala CLI downloads on-the-fly binaries from these repositories' GitHub release assets, and runs them as external processes.

Here's some of the more important external projects used by Scala CLI:

The use of external binaries allows to make the Scala CLI binary slimmer and faster to generate, but also allow to lower memory requirements to generate it (allowing to generate these binaries on the GitHub-provided GitHub actions hosts).

Website

The Scala CLI website is built with Docusaurus and uses Infima for styling.

Ensure you are using Node >= 16.14.2.

Generate the website once

cd website
yarn
yarn build
npm run serve

Generate the website continuously

cd website
yarn
yarn run start

Verifying the documentation

We have a built-in tool to validate .md files called Sclicheck. All Sclicheck tests can be run with Mill + munit: (and this is what we run on the CI, too)

./mill -i 'docs-tests[]'.test

The former also includes testing gifs and Sclicheck itself. To just check the documents, run:

./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.*'

You can also check all root docs, commands, reference docs, guides or cookbooks:

./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.root*'
./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.guide*'
./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.command*'
./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.cookbook*'
./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.reference*'

Similarly, you can check single files:

./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.<category> <doc-name>'

For example, to run the check on compile.md

./mill -i 'docs-tests[]'.test 'sclicheck.DocTests.command compile'

Scala CLI logos

Package with various logos for scala-cli can be found on google drive

Launcher script

There is a script scala-cli-src in the repository root that is intended to work exactly like released scala-cli, but using a binary compiled the worktree. Just add it to your PATH to get the already-released-scala-cli experience.

Releases

Instructions on how to release - Release Procedure

Debugging BSP server

The easiest way to debug BSP sever is using scala-cli-src script with --bsp-debug-port 5050 flag (the port should be unique to the workspace where BSP will be debugged). In such case BSP will be launched using local source and will run on JVM. It will also expects a debugger running in the listen mode using provided port (so the initialization of the connection can be debugged). In such case we recommend to have option to auto rerun debugging session off (so there is always a debugger instance ready to be used).

GraalVM reflection configuration

As Scala CLI is using GraalVM native image, it requires a configuration file for reflection. The configuration for the cli module is located in the reflect-config.json file.

When adding new functionalities or updating dependencies, it might turn out the reflection configuration for some class may be missing. The relevant error message when running integration.test.native may be misleading, usually with a ClassNotFoundException or even with a functionality seemingly being skipped. This is because logic referring to classes with missing reflection configuration may be skipped for the used native image.

To generate the relevant configuration automatically, you can run:

./mill -i 'cli[]'.runWithAssistedConfig <scala-cli-sub-command> <args> <options>

Just make sure to run it exactly the same as the native image would have been run, as the configuration is generated for a particular invocation path. The run has to succeed as well, as the configuration will only be fully generated after an exit code 0.

Config generated in out/cli/<scalaVersion>/runWithAssistedConfig.dest/config

As a result, you should get the path to the generated configuration file. It might contain some unnecessary entries, so make sure to only copy what you truly need. As the formatting of the reflect-config.json is verified on the CI, make sure to run the following command to adjust it accordingly before committing:

./mill -i __.formatNativeImageConf

For more info about reflection configuration in GraalVM, check the relevant GraalVM Reflection docs.

Overriding Scala versions in Scala CLI builds

It's possible to override the internal Scala version used to build Scala CLI, as well as the default version used by the CLI itself with Java props.

  • scala.version.internal - overrides the internal Scala version used to build Scala CLI
  • scala.version.user - overrides the default Scala version used by the CLI itself

NOTE: remember to run ./mill clean to make sure the Scala versions aren't being cached anywhere.

./mill -i clean
./mill -i --define scala.version.internal=3.4.0-RC1-bin-20231012-242ba21-NIGHTLY --define scala.version.user=3.4.0-RC1-bin-20231012-242ba21-NIGHTLY scala version --offline
# Scala CLI version: 1.x.x-SNAPSHOT
# Scala version (default): 3.4.0-RC1-bin-20231012-242ba21-NIGHTLY