Support for R

[edavidaja]

I'm interested in adding support for R scripts, RMarkdown documents, and R packages. I'm happy to do as much of the implementation myself as is necessary, but I'm looking for a little guidance on how best to do so within the project structure.

examples

scripts and RMarkdown:

• https://github.com/edavidaja/urbanindex/blob/master/data-raw/tracts.R#L1 should resolve to https://github.com/walkerke/tidycensus

packages are typically 'imported' via the library() function, though there are some alternative, less frequently used ways to do this, shown below:

# @octoLinkerResolve(http://rlang.r-lib.org)
library(rlang)

# @octoLinkerResolve(http://rlang.r-lib.org)
library("rlang")

# @octoLinkerResolve(http://rlang.r-lib.org)
require(rlang, quietly = TRUE)

# @octoLinkerResolve(http://rlang.r-lib.org)
require("rlang")

# @octoLinkerResolve(http://rlang.r-lib.org)
requireNamespace("rlang", quietly = TRUE)

# @octoLinkerResolve(http://rlang.r-lib.org)
loadNamespace("rlang")

The second capture group of the regex below contains the unquoted package name:

export const RSTATS_LIBRARY = regex`
  (library|require|requireNamespace|loadNamespace)\(
    ['"]? # optional open quote
    ([\w\.]+) # package name
    ['"]? # optional close quote
    .+ # irrelevant function arguments
  \)
`;

R Packages:
R package manifests (DESCRIPTION files) are Debian Control Format files. This is a more complex one. The fields Imports Suggests, Depends, and Enhances enumerate dependencies, each separated by a comma, with optional version numbering. Remotes usually resolves to GitHub or some other hosted git service:

https://github.com/r-lib/usethis/blob/master/DESCRIPTION#L27 should resolve to https://github.com/r-lib/cli#readme
https://github.com/r-lib/usethis/blob/master/DESCRIPTION#L58 should resolve to https://github.com/r-lib/gert

Additional information

R packages are hosted by Comprehensive R Archive Network (CRAN). A web version of each package manifest is available at the link: https://cran.r-project.org/package={package} (e.g. https://cran.r-project.org/package=flextable)

Packages with web pages list them in the URL field of the DESCRIPTION file; however, not all packages have a webpage. There is a GitHub Mirror of CRAN packages at https://github.com/cran, which has the source code, including the manifest, for any package on CRAN.

Thoughts on where I ought to begin? Tests in e2e? A DESCRIPTION resolver?

[stefanbuck]

Thanks for this very detailed explanation. Looks like you figured out the internals how OctoLinker works yourself already. I more than happy to guide you to this part in more detail, but first we need to make a change in another service. The OctoLinker API is a service responsible to resolve external packages. Adding support for a new package registry is super easy if we can call an API endpoint that returns a JSON document. For example, npm returns such a JSON document for react. We use XPath to find urls in the specified fields. The configuration for npm and other registries can be found in the config.json file. If CRAN doesn't have such a JSON endpoint it's a little bit more work, but overall still simple.

I opened an issue OctoLinker/api#252 so we can continue in more detail there.

Once this is done, I'll walk you through the other changes required to land this feature.

[stefanbuck]

As promised here are the steps to get support for R packages working.

1. Define regular expression

OctoLinker is using regular expressions to find code snippets in the DOM. All regular expressions are defined in helper-grammar-regex-collection. The file should be self-describing and it's just a matter of adding a new export.

To ensure the regular expression is working as expected, test cases can be added in the helper-grammar-regex-collection test-file. This file contains many test cases already.

2. Create a new plugin

This is best done by looking at an existing plugin for example the rust plugin. The public API of a plugin consists of three methods resolve, getPattern and getLinkRegexes.

The regular expression you defined earlier, needs to be returned by getLinkRegexes. This can be an array if you have more than one regular expression.

We use the return value from getPattern to figure out when to invoke a plugin. The pathRegexes is a regular expression which either matches file suffixes or entire file names.
Feel free to set githubClasses to an empty array for now. This is being used to invoke plugins in Markdown files or issue comments if a code snippet is added.

The resolve method is called with the extracted value. We have a bunch of different resolvers, but in your case all you need is the liveResolverQuery. This is just a wrapper to expose a simple API when using OctoLinker API.

resolve(path, [target]) {
  return liveResolverQuery({ type: 'cran', target });
},

3. Register plugin

Add the new plugin to https://github.com/OctoLinker/OctoLinker/blob/3692335dbc3e348b5a2a1f59e271f0817a4ca859/packages/core/load-plugins.js

4. Add E2E tests

Please follow the E2E test guide to get an understanding how we use E2E test.

5. Run extension locally

Your plugin won't work, until I added cran support to the API, but maybe worth going through the setup steps already. Follow the developer guide

That's it. I hope this all make sense. Please let me know if you have any further questions.

[xt0rted]

Feel free to set githubClasses to an empty array for now. This is being used to invoke plugins in Markdown files or issue comments if a code snippet is added.

I've been wondering what those classes are for.

[stefanbuck]

I've been wondering what those classes are for.

You never asked 😉 I'm glad that this issue is spreading some knowledge. @xt0rted do you see value in having a better documentation of OctoLinker's core?

[xt0rted]

@stefanbuck having plugins documented would be really helpful. There's been a number of things recently that I wasn't sure what they were used for or that they were available. Maybe this is something TypeScript could help with?

[stefanbuck]

@edavidaja are you still interested in contributing this feature? I added cran support to the API already OctoLinker/api#255 Once this is merged, you should be able to use the liveResolver as explained to resolve cran packages. Please let me know if you need help.

[edavidaja]

Yep, still interested! Life is a bit hectic at the moment but I'll be able to give this more serious attention next weekend.

[stefanbuck]

All good, there is no need to rush this, I was just curious. By next week the API change should be merged