Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add NWBHDF5IO.read_nwb() method #1979

Merged
merged 12 commits into from
Nov 12, 2024

Conversation

h-mayorquin
Copy link
Contributor

@h-mayorquin h-mayorquin commented Nov 7, 2024

This was discussed in #1974

This is the first part of the program described by @oruebel here. In a future PR and once hdmf-dev/hdmf-zarr#225 is merged I will implement pynwb.read_nwb in another PR.

Meanwhile, I have two questions:

  1. What streaming method should be used when an s3 path is passed?
  2. For testing this, the ros3 test have some files that I think I could use, is that OK?

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Have you checked our Contributing document?
  • Have you ensured the PR clearly describes the problem and the solution?
  • Is your contribution compliant with our coding style? This can be checked running ruff check . && codespell from the source directory.
  • Have you checked to ensure that there aren't other open Pull Requests for the same change?
  • Have you included the relevant issue number using "Fix #XXX" notation where XXX is the issue number? By including "Fix #XXX" you allow GitHub to close issue #XXX when the PR is merged.

Copy link

codecov bot commented Nov 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.92%. Comparing base (ad04661) to head (2ed4aa5).
Report is 1 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1979      +/-   ##
==========================================
+ Coverage   91.87%   91.92%   +0.04%     
==========================================
  Files          27       27              
  Lines        2697     2713      +16     
  Branches      704      708       +4     
==========================================
+ Hits         2478     2494      +16     
  Misses        145      145              
  Partials       74       74              
Flag Coverage Δ
integration 72.87% <100.00%> (+0.16%) ⬆️
unit 82.89% <18.75%> (-0.39%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@oruebel
Copy link
Contributor

oruebel commented Nov 7, 2024

2. For testing this, the ros3 test have some files that I think I could use, is that OK?

Yes, that should be fine

  1. What streaming method should be used when an s3 path is passed

I think we should probably use fsspec as the default https://pynwb.readthedocs.io/en/latest/tutorials/advanced_io/streaming.html#streaming-method-1-fsspec

If installed, we could also use remfile https://pynwb.readthedocs.io/en/latest/tutorials/advanced_io/streaming.html#method-3-remfile and use fsspec as the fall-back. remfile tends to be a bit faster than fsspec, but I think fsspec is more widely used.

@bendichter what would you suggest for this?

@oruebel
Copy link
Contributor

oruebel commented Nov 7, 2024

Aside from adding s3 and testing, the outline you started here looks good to me.

@h-mayorquin h-mayorquin marked this pull request as ready for review November 9, 2024 19:04
@h-mayorquin
Copy link
Contributor Author

OK, I added s3 testing.

@h-mayorquin
Copy link
Contributor Author

Is the test failing a permission issue?
image

@rly
Copy link
Contributor

rly commented Nov 12, 2024

Seems like it was a codecov issue with detecting forks, perhaps when commits happen in quick succession: codecov/codecov-action#1580 . I re-ran the failing runs.

@rly rly requested a review from oruebel November 12, 2024 02:30
Copy link
Contributor

@oruebel oruebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from the codecov question, this looks good to me. I approve the PR, because I assume the codecov question can probably be resolved in the comment, rather than requiring additional code changes, and it seems easier for me to re-approve if there are changes, rather than holding up the PR.

@rly rly merged commit 9de4d4c into NeurodataWithoutBorders:dev Nov 12, 2024
24 of 25 checks passed
@h-mayorquin h-mayorquin deleted the add_io_can_read branch November 12, 2024 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants