Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xnatdownload repeated crashes on large downloads #430

Open
baxpr opened this issue Jun 30, 2023 · 1 comment
Open

Xnatdownload repeated crashes on large downloads #430

baxpr opened this issue Jun 30, 2023 · 1 comment

Comments

@baxpr
Copy link
Member

baxpr commented Jun 30, 2023

Been having a lot of problems with extended (hours-long) Xnatdownload requests failing entirely due to connection timeout or connection failure. Here's a sketch of a way to download an assessor resource much more reliably at the cost of more XNAT sessions being created:

import os

from dax import XnatUtils


with XnatUtils.get_interface() as xnat:
    assrs = xnat.list_project_assessors('ABCD')
assrs = [a for a in assrs if a['proctype']=='cersuit_v3']

for a in assrs:

    print(a['label'])

    outdir = (
        'cersuit_v3/' + 
        a['project_id'] + '-x-' + 
        a['subject_label'] + '-x-' + 
        a['session_label'] + '-x-' + 
        a['label']  
        )
    if os.path.exists(outdir):
        print('    WARNING: Outdir exists - skipping')
        continue
    else:
        os.mkdir(outdir)

    try:
        with XnatUtils.get_interface() as xnat:
            rsrc = xnat.select_assessor_resource(
                a['project_id'],
                a['subject_label'],
                a['session_label'],
                a['label'],
                'VOLS_NATIVE',
                )
            rsrc.get(outdir, extract=True)
    except:
        print('    WARNING: Download failed')
@baxpr
Copy link
Member Author

baxpr commented Aug 25, 2023

I think the underlying issue is really that Xnatdownload doesn't handle or recover from intermittent connection failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants