Process ICESat-2 ATL11 data up to 20210421 #295
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
weiji14
added
enhancement ✨
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
DeepCode's analysis on #120676 found:
Top issues
👉 View analysis in DeepCode’s Dashboard | Configure the bot👉 The DeepCode service and API will be deprecated in August, 2021. Here is the information how to migrate. Thank you for using DeepCode 🙏 ❤️ !If you are using our plugins, you might be interested in their successors: Snyk's JetBrains plugin and Snyk's VS Code plugin. |
weiji14
changed the title
Updating the ATLAS catalog to include ATL11 data up to and including 20210421, about five months of extra data over 2c755d5. Had to temporarily downgrade (locally, again) to fsspec=0.7.4 and intake-xarray=0.3.2 for the ATL11 intake download code in atl06_to_atl11.ipynb to work. Yes, taking a shortcut this time by pulling the official ATL11.003 data straight from NSIDC instead of wasting a week processing it ourselves. Glad that they have cycle 03 to cycle 09 in a single HDF5 file!! Doing more preprocessing in the atl06_to_atl11.ipynb script by adding a set_xy_and_mask function which does lon/lat to x/y coordinate conversion and masking out of low quality data points. Should save some hassle in atl11_play.ipynb. Using 2 cores instead of 8 because of memory restrictions on the HPC cluster. So yeah, basically a quick refresh over d9ad9db.
weiji14
force-pushed
the
update/atl11_20210421
branch
from
31abdb0 to
120676a
Compare
Sourcery Code Quality Report✅ Merging this PR will increase code quality in the affected files by 2.11%.
Here are some functions in these files that still need a tune-up:
Legend and ExplanationThe emojis denote the absolute quality of the code:
The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request. Please see our documentation here for details on how these metrics are calculated. We are actively working on this report - lots more documentation and extra metrics to come! Help us improve this quality report! |
4 tasks
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
From 3️⃣ to 9️⃣ ICESat-2 repeat cycles to analyze. See also previous update at #231.
ATL11.003 release is out at https://doi.org/10.5067/ATLAS/ATL11.003 which goes up to April 2021. This time, no more week-long ATL06 to ATL11 processing on our end (woohoo)! Just downloading straight from NSIDC 😁 Doing the HDF5 to Zarr conversion though, until I can be bothered with resolving #100.