Document how to import an absolute path with import.

[amano-kenji]

I recently learned that I can import /path/to/module.janet with

(import @/path/to/module)

This is not documented on the website, yet.

[sogaiu]

May be we can figure out a good place to put some helpful text -- though I wonder what sorts of circumstances [1] are a good fit for using absolute paths for imports.

For reference, I found the following in Janet's changelog:

Allow importing modules from custom directories more easily with the @ prefix to module paths. For example, if there is a dynamic binding :custom-modules that is a file system path to a directory of modules, import from that directory with (import @custom-modules/mymod).

This looks like a relevant commit.

The docstring for module/expand-path was changed in that commit to have some info as well:

* :@all: -- Same as :all:, but if path starts with the @
  character, the first path segment is replaced with a
  dynamic binding `(dyn <first path segment as keyword>)`.

The above is perhaps not the same, but if there isn't any documentation for it on the website, may be it could go in a similar location as for absolute path imports...

---

[1] I was informed via another channel that it's good for testing and that seems reasonable.

[sogaiu]

Perhaps after the Working Directory Imports section, is a reasonable location for a section about imports that involve the @all "replacement" (to use the language of the docstring).

I think it makes sense to point out specifically that @all can be used for absolute path imports AND that a typical use case for those types of imports is for testing. Further, may be it's worth saying that it seems unlikely that one will want to use absolute path imports in production.

[amano-kenji]

👏

[sogaiu]

I'm going to try to collect some bits here to inform the to-be-constructed section text. Feel free to skip the next bit to the summary of what the details revealed.

---

Details that most people might want to skip...

I've found this code in the C implementation of module/expand-path which I think handles :@all::

            } else if (strncmp(template + i, ":@all:", 6) == 0) {
                if (input[0] == '@') {
                    const char *p = input;
                    while (*p && !is_path_sep(*p)) p++;
                    size_t len = p - input - 1;
                    char *str = janet_smalloc(len + 1);
                    memcpy(str, input + 1, len);
                    str[len] = '\0';
                    janet_formatb(out, "%V", janet_dyn(str));
                    janet_sfree(str);
                    janet_buffer_push_cstring(out, p);

Using gdb / rr, I observed the execution of the code above when janet ran a simple .janet script :

# 0. directory preparation
(os/mkdir "/tmp/")
(os/mkdir "/tmp/import-test")


# 1. @-prefixed path import demo
(spit "/tmp/import-test/module-1.janet" "(def a 1)")

(setdyn :fun "/tmp/import-test")

# after the @, the first path segment is the string "fun"
# so @fun is replaced with "/tmp/import-test", to produce
# the path "/tmp/import-test/module-1"
(import @fun/module-1)

(print module-1/a)


# 2. absolute path import demo
(spit "/tmp/import-test/module-2.janet" "(def b 2)")

# after the @, the first path segment is seen as an empty
# string, and is not replaced with anything
(import @/tmp/import-test/module-2)

(print module-2/b)

It was indeed the case that the (import @...) portions for both 1. and 2. were handled by the C code quoted above.

For 2., len turns out to 0 and str[len] = '\0'; (i.e. str ends up as a zero-terminated empty string), so:

janet_formatb(out, "%V", janet_dyn(str));

doesn't really affect out (a Janet buffer which starts out empty).

Thus in this case, because p had ended up as /tmp/import-test/module-2, calling:

janet_buffer_push_cstring(out, p);

resulted in out having the content /tmp/import-test/module-2.

---

Summary

So what does this mean?

The above investigation made the manner in which absolute path imports work for @-prefixed paths clearer.

Specifically, although the quote from the changelog above has the text:

For example, if there is a dynamic binding :custom-modules that is a file system path to a directory of modules, import from that directory with (import @custom-modules/mymod).

that mentions a dynamic binding, this isn't apparent in the section of code starting with 2. in the above sample .janet script because there is no dynamic binding visible in the code.

The docstring for module/expand-path has this text:

but if path starts with the @ character, the first path segment is replaced with a dynamic binding (dyn <first path segment as keyword>)

The investigation also revealed that "the first path segment" for a path like @/tmp/import-test/module-1 is an empty string. It refers to what is between @ and the first subsequent path separator (since my test was on a Linux machine, the path separator was /).

---

To clarify a bit, "path separator" can behave differently depending on whether one is using Windows:

static int is_path_sep(char c) {
#ifdef JANET_WINDOWS
    if (c == '\\') return 1;
#endif
    return c == '/';
}

This is not to say that / is not treated as a path separator on Windows, but rather on Windows, \ is also treated as a path separator.

In any case, if interested in writing code that can work on things other than Windows, it's probably best to avoid using \ in paths related to Janet's import machinery.

[sogaiu]

Ok, below is an initial draft that incorporates some earlier text from the changelog along with results from what we learned about absolute path imports.

@-prefixed Imports

Starting in 1.26.0, Janet allows importing modules from 
custom directories more easily with the `@` prefix to 
module paths.

For example, if there is a dynamic binding `:custom-modules` 
that is a file system path to a directory of modules, 
import from that directory with:

`(import @custom-modules/mymod)`

As a special case, it is possible to import from absolute 
paths by prefixing an absolute path with `@`.  For 
example, to import from `/tmp/custom-modules/mymod.janet`,
express this as:

`(import @/tmp/custom-modules/mymod)`

Note that although using absolute paths is possible and 
useful for testing, it is not recommended for most 
production use cases.

[amano-kenji]

Excellent.

[sogaiu]

Thanks for taking a look.

PR #213 (with some minor edits) has been submitted.

[sogaiu]

As #213 has been merged, I think this can be closed now.