Enabling absolute checkpoint dir

[lade-odoo]

Sometimes, we don't want the checkpoints to be saved into our working directory. Instead, we may want to recreate the project structure inside a separate folder (e.g.: /home/{user}/.cache) like some software are doing.

[welcome]

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.

You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[kevin-bates]

Hi @lade-odoo - thanks for opening this pull request. I have a couple of questions that come to mind...

1. Won't this approach encounter duplicate checkpoint files otherwise protected by directory/file semantics? For example, if I have two notebooks named Untitled.ipynb residing in separate directories (dir1 and dir2), won't this lead to the same checkpoint path for each file (e.g., Untitled-checkpoint.ipynb)?
2. Should there be any validation for where this single checkpoint directory can reside? I.e., should it reside within the root_dir (aka notebook_dir)?

[lade-odoo]

@kevin-bates Thank you for you questions:

1. No, that's the point, there won't be any duplicate. The directory structure will be reconstructed inside the checkpoint_dir folder. So, following your scenario, there will be two files created: dir1/Untitled-checkpoint.ipynb and dir2/Untitled-checkpoint.ipynb.
2. From my point of view, I like the idea of being able to put it anywhere we want (we leave this freedom to the user).

[kevin-bates]

No, that's the point, there won't be any duplicate. The directory structure will be reconstructed inside the checkpoint_dir folder.

Shoot - I'm sorry, I missed that piece.

I like the idea of being able to put it anywhere we want (we leave this freedom to the user).

I tend to agree. One caveat with an absolute checkpoint-dir would occur if users change their root_dir from one invocation to the next. In those instances, the checkpoint file's location will vary within the single checkpoint directory structure. I wouldn't expect that to commonly occur and could probably be addressed via the help string stating that an absolute checkpoint dir essentially requires consistent root_dir values between invocations.

Hmm - I suppose for users that change their root_dir locations entirely (i.e., disjoint hierarchies), you could have similar sub-directory structures in place that could lead to the duplicates issue. But, as you point out, perhaps absolute checkpoint directories are not for them.

[lade-odoo]

@kevin-bates Is there any-news here ?

[kevin-bates]

Hi @lade-odoo - thank you for the ping and I apologize for my inactivity (new job outside of Jupyter OSS).

I'm going to approve this PR, but I'm hoping someone closer to the Contents Service will take look prior to its merge. I'm just not sure if this might side-affect things in any non-obvious ways.

[kevin-bates]

This looks good to me - thank you @lade-odoo. I'm hoping someone closer to the Contents Service can take a look at this prior to its merge.

[blink1073]

From a security standpoint, we only ever write to files that are contained withing the startup directory. This would have to be an opt-in feature, as a new traitlet on FileCheckpoints.