Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature : pull only parts of a repository #13

Open
apraga opened this issue Feb 9, 2024 · 3 comments · May be fixed by #22
Open

Feature : pull only parts of a repository #13

apraga opened this issue Feb 9, 2024 · 3 comments · May be fixed by #22
Labels
enhancement New feature or request

Comments

@apraga
Copy link

apraga commented Feb 9, 2024

Hi,

In my experiments, data can be quite large. It would be awesome to have a way to pull only some files or subfolders to avoid taking too much space. For example, working on the output files of an experiments would be improved if one did not have to download the raw data to be processed.

Thanks,
@vsbuffalo
Copy link
Owner

Sure, I think this a great idea and something other users have requested. I think the main thing is to figure out how best particular subsets of the data should be downloaded. So I can best design this upcoming feature, what did you envision? E.g. would you use sdf get file_1 file_2 etc? or have some sort of "tag" that you could use sdf <tag>?

@apraga
Copy link
Author

apraga commented Feb 9, 2024 •
edited
Loading

Thanks for the quick feedback ! My first idea was to follow the model of git add where a list of files or folders is supplied. If a folder is given, all files and subdirectories should be included.

This could be passed as optional arguments to sdf pull (that feels the most natural at the moment). What do you think ?

@vsbuffalo vsbuffalo added the enhancement New feature or request label Feb 22, 2024
@apraga
Copy link
Author

apraga commented May 29, 2024

@vsbuffalo If that's okay with you, I would like to take a stab at it.

@apraga apraga mentioned this issue Jun 1, 2024
Open
@apraga apraga linked a pull request Jun 16, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Pull only a single file or a whole folder. apraga/scidataflow
2 participants
@vsbuffalo @apraga