Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update intermediate bucket related doc #4521

Merged
merged 9 commits into from
Jan 3, 2025
Merged

Update intermediate bucket related doc #4521

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6085556
doc
zpoint Jan 2, 2025
7dc615c
Update docs/source/examples/managed-jobs.rst
zpoint Jan 2, 2025
78d4600
Update docs/source/examples/managed-jobs.rst
zpoint Jan 2, 2025
fb28ffb
Update docs/source/examples/managed-jobs.rst
zpoint Jan 2, 2025
e6eca83
Update docs/source/examples/managed-jobs.rst
zpoint Jan 2, 2025
ea1c189
Update docs/source/examples/managed-jobs.rst
zpoint Jan 2, 2025
81253ca
Update docs/source/examples/managed-jobs.rst
zpoint Jan 2, 2025
e9eee76
add tip
zpoint Jan 2, 2025
d14e1a5
minor changes
romilbhardwaj Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion docs/source/examples/managed-jobs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,39 @@ The :code:`MOUNT` mode in :ref:`SkyPilot bucket mounting <sky-storage>` ensures
Note that the application code should save program checkpoints periodically and reload those states when the job is restarted.
This is typically achieved by reloading the latest checkpoint at the beginning of your program.


.. _intermediate-bucket:
zpoint marked this conversation as resolved.
Show resolved Hide resolved

Intermediate Bucket
zpoint marked this conversation as resolved.
Show resolved Hide resolved
~~~~~~~~~~~~~~~~~~~~

For managed jobs, SkyPilot requires a bucket to store intermediate files, such as local file mounts, temporary files, and the workdir.
zpoint marked this conversation as resolved.
Show resolved Hide resolved
If you do not configure a bucket, SkyPilot will automatically create a default temporary bucket named :code:`skypilot-filemounts-{username}-{run_id}` for each job launch.
zpoint marked this conversation as resolved.
Show resolved Hide resolved
Alternatively, you can specify a persistent bucket using :code:`jobs.bucket` in :code:`~/.sky/config.yaml`. If you choose to specify a bucket, ensure that the bucket already exists and that you have the necessary permissions.
zpoint marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: yaml

# ~/.sky/config.yaml
jobs:
bucket: s3://my-bucket # Supports s3://, gs://, https://<azure_storage_account>.blob.core.windows.net/<container>, r2://, cos://<region>/<bucket>

When using :code:`jobs.bucket`, SkyPilot organizes files in the following structure:
zpoint marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: text

# cloud bucket, s3://my-bucket/ for example
my-bucket/
├── job-15891b25/ # Job-specific directory
│ ├── local-file-mounts/ # Files from local file mounts
│ ├── tmp-files/ # Temporary files
│ └── workdir/ # Files from workdir
└── job-cae228be/ # Another job's directory
├── local-file-mounts/
├── tmp-files/
└── workdir/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a tip mentioning that multiple users can use the same intermediate bucket?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

SkyPilot automatically deletes the entire bucket after the job completes if no bucket is specified. When using a custom bucket (:code:`jobs.bucket`), only the job-specific directories (e.g., :code:`job-15891b25/`) are cleaned up after job completion, while the bucket itself persists.
zpoint marked this conversation as resolved.
Show resolved Hide resolved

.. _spot-jobs-end-to-end:

An End-to-End Example
Expand Down Expand Up @@ -505,4 +538,3 @@ The :code:`resources` field has the same spec as a normal SkyPilot job; see `her
These settings will not take effect if you have an existing controller (either
stopped or live). For them to take effect, tear down the existing controller
first, which requires all in-progress jobs to finish or be canceled.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: undo?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is stripped by trim trailing whitespace of pre-commit. Can we keep it as it is, or do we prefer a new line at the end of the file? If so, I will update the pre-commit config.

trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing docs/source/examples/managed-jobs.rst

Loading