Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export OD results in Parquet #147

Merged

Conversation

ChristopherRabotin
Copy link
Member

@ChristopherRabotin ChristopherRabotin commented May 17, 2023

  • Fix semi minor axis computation (was NaN for non hyperbolic orbits)
  • Add a timestamped initializer for ExportCfg (and related TODOs)
  • Remove heterogeneous rms_pre/postfit_residual function, use rms_residual_ratios instead
  • Remove old code in trigger.rs
  • Merge Residual<T> and Estimate Compiler got very unhappy with all permutations I tried of this. This also complicates the fact that Estimate would need to know what kind of measurement is used: that isn't a deal-breaker but it isn't necessarily the most evident idea. For example, this would make it impossible to share a KfEstimate between a filter that runs on Range and one that ingests RangeDoppler.
  • Add Kalman gain to Estimate (as an option since it isn't set for a time update) Requires the above.
  • Remove predicted: bool from Estimate and change to a pub const function depending on whether there is a residual or not Now irrelevant
  • Export the full estimate to parquet (on a separate thread)
  • Add ExportCfg to Python interface to control how to export data in parquet

Effects

Closes #134

If this is a new feature or a bug fix ...

  • Yes, the branch I'm proposing to merge is called issue-xyz where xyz is the number of the issue.

If this change adds or modifies a validation case

  • No.

+ Fix semi minor axis computation (was NaN for non hyperbolic orbits)
+ Add a `timestamped` initializer for ExportCfg (and related TODOs)
+ Remove heterogeneous `rms_pre/postfit_residual` function, use `rms_residual_ratios` instead
+ Remove old code in `trigger.rs`

Signed-off-by: Christopher Rabotin <[email protected]>
@codecov
Copy link

codecov bot commented May 20, 2023

Codecov Report

Patch coverage: 81.81% and project coverage change: +1.48 🎉

Comparison is base (90e9926) 64.92% compared to head (d8f4c69) 66.40%.

❗ Current head d8f4c69 differs from pull request most recent head 87ae42b. Consider uploading reports for the commit 87ae42b to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #147      +/-   ##
==========================================
+ Coverage   64.92%   66.40%   +1.48%     
==========================================
  Files         126      127       +1     
  Lines       33087    32368     -719     
==========================================
+ Hits        21482    21495      +13     
+ Misses      11605    10873     -732     
Impacted Files Coverage Δ
src/cosmic/eclipse.rs 67.16% <0.00%> (-1.44%) ⬇️
src/io/trajectory_data.rs 0.00% <ø> (ø)
src/md/mod.rs 0.00% <ø> (ø)
src/md/trajectory/mod.rs 0.00% <ø> (-60.61%) ⬇️
src/md/trajectory/traj_it.rs 30.00% <0.00%> (+2.72%) ⬆️
src/od/filter/mod.rs 30.00% <ø> (ø)
src/od/mod.rs 0.00% <ø> (ø)
src/od/process/rejectcrit.rs 37.50% <ø> (ø)
src/od/process/trigger.rs 88.00% <ø> (+22.32%) ⬆️
tests/orbit_determination/mod.rs 87.09% <ø> (-2.38%) ⬇️
... and 21 more

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ChristopherRabotin
Copy link
Member Author

ChristopherRabotin commented May 20, 2023

Cleaning up tasks:

  • Change ODP in Python to return the path to the stored parquet file
  • Ensure ExportCfg is available in Python
  • Update OD plotting scripts for new format, include the ability to scale to covariance and plot as an uncertainty (should be the default)
  • Remove NavFormatter (finally replaced by parquet output)
  • Remove StateFormatter (replaced entirely by parquet output)

Remove StateFormatter and NavSolutionFormatter, use `to_parquet` instead.

Signed-off-by: Christopher Rabotin <[email protected]>
@ChristopherRabotin
Copy link
Member Author

ChristopherRabotin commented May 24, 2023

One last thing to make this immediately usable:

  • Parse OEM file into a Spacecraft or Orbit trajectory

These are no longer relevant given the switch to parquet export

Signed-off-by: Christopher Rabotin <[email protected]>
the csv crate is still needed for the Cosmo export

Signed-off-by: Christopher Rabotin <[email protected]>
Signed-off-by: Christopher Rabotin <[email protected]>
Signed-off-by: Christopher Rabotin <[email protected]>
cargo +nightly udeps

Signed-off-by: Christopher Rabotin <[email protected]>
Signed-off-by: Christopher Rabotin <[email protected]>
@ChristopherRabotin ChristopherRabotin merged commit a53dfef into master May 24, 2023
@ChristopherRabotin ChristopherRabotin deleted the 134-store-orbit-determination-results-in-parquet branch May 24, 2023 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
i-rust Relative to the Rust interface P-high T-orbit-determination
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store orbit determination results in parquet
1 participant