Preliminary: generate new planetary datasets

[gwbres]

No description provided.

[gwbres]

@ChristopherRabotin,

the idea is to create new planetary datasets that define ECEFrames with Ellipsoid shapes that are used in GNSS nav.
I create an application that we may run either from time to time, or only once, it does not have to be integrated to CI/CD.
It actually only needs to run anytime new frames need to be introduced.

major: I don't understand whether we should create a single planetary constant PCA: one per frame, or it can be grouped for all of them.
I presume it should be grouped, for example in a gnss.PCA, because it is possible to build any frame that is defined within a single almanac. I presume this is what happens if I push two definitions in a row

minor: i'm experiencing issues with the pseudo code you gave in the other PR:

1. constants::celestial_objects::EARTH is not a Uid so it is not compatible with almanac.frame_from_uid
2. MetaAlmanac::latest() seems very convenient, especially in this scenario. But it never works on my side: it always fails to retrieve the Moon constants. Are we sure of the 0.4.2 behavior with respect to the latest() helper ?

[ChristopherRabotin]

Yes, you'll want to have a single PCA with all of your frames defined, that's the simplest.

You're right: it shouldn't be EARTH but EARTH_J2000 (the first one is the celestial object, the second one is the frame). Another thing I just thought of: will you need transformations between the WGS84 frame and other frames? If so, if the WGS84 frame does not have an ID used by other datasets, the Almanac won't know how to transform it. I think it would be best to also add a test in this code about how you plan on using it so that you can be sure that it works as expected. Also, I'll admit that I have not used the feature to create new frames yet in production (I only use SPICE data), so this will be learning experience for me as well.

What is the error you're getting with the moon frame definitions? It's hosted on the same storage server, it should just work... I hope to work on nyx-space/anise#294 this week which will fix issues where, for whatever reason, the data can't be downloaded and the lock file persists. Is that the problem you're seeing?

[gwbres]

will you need transformations between the WGS84 frame and other frames?
If so, if the WGS84 frame does not have an ID used by other datasets, the Almanac won't know how to transform it

Yes, if ANISE cannot transform any frame to any frame, it is almost pointless attempting any progress on the topic.
In other words, if we introduce new frames, we need to be able to convert from one of those to the standards (at least that's the minimum). We might not need the otherway around (ECEF_2000 -> GNSS)

The reason for this, the way I see it, is we build the Orbit from kepler parameters very precisely, and in specific GNSS frames: this is what GNSS constellations describe. And each constellation has its own reference frame. Eventually, we want to work in a single reference frame, for example ITRF93 and we need to bring everything back to that common frame.
It simplifies everything, and is actually the most correct operation we can think of. The SV attitutes are correctly described (correct interpretation of the TX orbit) and we apply valid conversions, to a unique context that facilitates everything and is easily understandable

[gwbres]

I think it would be best to also add a test in this code about how you plan on using it
so that you can be sure that it works as expected

Although I understand and agree, I don't see what test might be relevant.
The application is frame description (so being able to build those frames) and convert them, either between them, or back to ITRF. Maybe we can use the navigation equation, like the Bancroft solver, that gathers 4 TX orbits and deduces 1 RX orbit, but I'm not sure it buys us anything. Considering we know how to describe an ellipsoid and the frames already work well, I would expect this will work even on first attempt

[gwbres]

What is the error you're getting with the moon frame definitions?

argh sorry, it was once again the "lock file error", and I have a tendency not to read the error message in its entirety

whatever reason, the data can't be downloaded and the lock file persists

I do : )
I thought it came from me calling your API from different programs (rinex-cli, gnss-rtk testbench, this new gnss program.. ie., different file ""creators"") while they all use the same storage location, but there might be something i'm missing here

[ChristopherRabotin]

I thought it came from me calling your API from different programs (rinex-cli, gnss-rtk testbench, this new gnss program.. ie., different file ""creators"") while they all use the same storage location, but there might be something i'm missing here

Yes, that's correct, and that's the whole point. These files should have a fixed unique name so if you already have say the de440s.bsp file in your user's ANISE folder, no need to redownload all ~200 MB of it.

Concerning the test, here's what I'm thinking:

1. Load the latest Almanac
2. Modify the planetary constants to add your ellipsoid
3. Define an arbitrary Orbit in that frame which has the newly defined ellipsoid
4. Try to transform that Orbit into another frame (e.g. IAU_EARTH) and check that it does not fail.

I worry that step 4 would fail. In that case, I can think of two options:

1. The Almanac also contains a dataset of hardcoded rotations in the format of an Euler Parameter, which is just a unit Quaternion. You could define a new Quaternion in the Euler parameter dataset that perform the identity rotation from the 39901 frame to the J2000 (id 1) frame. I think that this would allow the full transformation from your custom frame to any other loaded frame because you'd be defining the rotation from your custom frame to a loaded frame.
2. If that does not work, the only alternative I can think of is not a good one: overwrite an existing frame number. If that is the only solution, then I'll need to work on a better one within ANISE.

Let me know how that goes, there's definitely a solution out there, but this is my first time thinking about this problem outside of the SPICE files.

[gwbres]

So if I understand correctly, we just need to make sure whether an external solution (ie., no modification of ANISE on your side) can answer the requirement of introducing new frame rotations.

Could you spend a couple of minutes helping me getting a version that compiles ?
The current state does not, because it says we cannot push Frames but only PlanetaryData into the planetary data set that latest() has generated. I'm trying to convert the Frame into PlanetaryData but I'm not even sure this is the right thing to do. PlanetaryData has many fields that have no obvious source from a Frame.

.save_as() works fine.

If I run the current test, it fails to load this new almanac with :

when loading data set from bytes, Data checksum differs from expected checksum encountered when loading as planetary data

If I try this instead:

 82         let mut bytes = Vec::<u8>::new();
 83         // load gnss_pck.pca
 84         let almanac = Almanac::default();
 85         let mut fd = File::open("pck_gnss.pca")
 86             .unwrap_or_else(|e| panic!("Failed to read GNSS PCA: does file exist? {}", e));
 87         fd.read_to_end(&mut bytes)
 88             .unwrap_or_else(|e| panic!("Failed to read GNSS PCA: does file exist? {}", e));
 89         let pck = PlanetaryDataSet::from_bytes(bytes);
 90         almanac.with_planetary_data(pck);

so I can use with_planetary_data with a compatible structure, it fails with :

DataSetIntegrity { action: "loading data set from bytes", source: ChecksumInvalid { expected: 1654977453, computed: 2243600445 } }

[ChristopherRabotin]

So if I understand correctly, we just need to make sure whether an external solution (ie., no modification of ANISE on your side) can answer the requirement of introducing new frame rotations.

Yes, that's correct. And it should be possible because ANISE does not ship with any frames.

Concerning the integrity error, I forgot that the save_as function does not update the checksum of the data. As a little bit of background, ANISE is designed to fly on spacecraft, and these can encounter single bit flips on the flash or in RAM. To prevent ANISE from loading a corrupted file, the CRC32 checksum of the data is checked to conform with what is expected, and whomever runs ANISE onboard a spacecraft is expected to regularly call the scrub() function to force re-reading the whole data (flash memories have forward error correction that, like in telco, allows them to fix single bits as they read them). With all of this background said, you need to add the following before calling save_as.

my_pck.set_crc32(my_pck.crc32());

Concerning the attempt to push a frame into a planetary dataset, could you provide an example of what you're trying to do? I wrote this function quite some time ago, so I forgot a little bit myself. But it seems that if you have a frame, you can use the almanac to find the planetary data, then pass the frame to it using to_frame, and then the planetary data will be set in the frame. Something like this

let original_earth_itrf93 = Frame(EARTH, ITRF93); // Won't have any planetary data but that's OK.
let my_planetary_data = almanac.planetary_data.get_by_name("GPS WGS84").unwrap();
let my_custom_shape_for_itrf93 = my_planetary_data.to_frame(original_earth_itrf93).unwrap();

assert_eq!(my_planetary_data.shape, my_custom_shape_for_itrf93.shape.unwrap());

[gwbres]

As a little bit of background, ANISE is designed to fly on spacecraft, and these can encounter single bit flips on the flash or in RAM.

Very interesting, I wanted to ask you about what part of your ecosystem might eventually be embedded. Especially since you recently brought up that you guys are discussing using your toolkit for the missions to come (I presume, very complex decision making with lots of consequences). I presumed you used Nyx for mission planning, but embedding it would be yet quite an achievement.

FYI, I'm specialized in embedded software, especially embedded linux and FPGA stuff. Of course I build higher level software, but it is rarely the case in profesionnal contexts, and not where the critical stuff happens. I have stopped working on these topic outside work about 10 years ago. I guess the hardware and instrumentation complexity (that I personally find very tiresome) led me to it. I also have never put anything into space (or at least, not that I'm aware of) but i'm aware of the environmental issues. Although, my employer did and has done it in the past, even to this day some colleagues of mine do work on stuff that will eventually be launched into space.

could you provide an example of what you're trying to do?

it's located in the planetary/main and associated test bench (WIP). Thank you very much, I will give it a try this weekend

[gwbres]

set_crc32 does not take any argument.
It works: i can now open the planetary data set that I generate.

I don't think you understood the problem. The problem is in the definition of the new frame. To push it into the dataset, we need to convert a Frame (what I'm building/defining) into a PlanetaryData (what can be pushed in the table).

Simply build the program and run the test:

cd planetary
cargo build -r
./target/release/planetary  # builds pck_gnss.pca (local file)
cargo test -r # open pck_gnss and try to use new definitions

To me, it looks like either PlanetaryData misses a from_frame() or Frame misses a to_planetary_data()

[ChristopherRabotin]

Very interesting, I wanted to ask you about what part of your ecosystem might eventually be embedded. Especially since you recently brought up that you guys are discussing using your toolkit for the missions to come (I presume, very complex decision making with lots of consequences). I presumed you used Nyx for mission planning, but embedding it would be yet quite an achievement.

Correct, at the moment, we're only using it on ground software, and there are no plans yet to fly ANISE as-is. However, the design is such that it could fly on embedded Linux, so far from real embedded / RTOS. In my experience, Linux has all of the tools one needs to confidently fly spacecraft, and is much simpler and cheaper to use than other true embedded platforms (even though true embedded platforms are needed for hardware control).

FYI, I'm specialized in embedded software, especially embedded linux and FPGA stuff.
I had no idea! Radio work mostly?

I don't think you understood the problem. The problem is in the definition of the new frame. To push it into the dataset, we need to convert a Frame (what I'm building/defining) into a PlanetaryData (what can be pushed in the table).

I think I understand, but I also think I didn't explain something: frames are not defined in ANISE. They are all built at runtime provided a given central object and an orientation ID. To fetch the correct shape for a given frame, the Almanac will return it when you call from_from_uid with a Frame instance as a parameter. It will search for the correct planetary data on request.

As an example, the IAU_EARTH frame is built without any planetary data (no shape, no gravity parameter). Then, when you call frame_from_uid, the planetary data is loaded for you, and now the frame has both a shape and a planetary data. That's why all you need to do is define the planetary data, load it in the Almanac, and then call frame_from_uid.

Let me know if that works for you. One limitation I'm thinking of right now, is that ANISE follows the same approach that SPICE uses, and there is only one planetary data per central object. For example, in the NASA provided "tpc" file, the radius axes of the Earth are defined for the Earth itself by its ID of 399. I wonder if that could be a problem for you where you have possibly different models of the shape of the Earth depending on the constellation you're using.

[gwbres]

on embedded Linux, so far from real embedded / RTOS

Xenomai is a real time declination of the linux kernel, personnaly I have never used it. I presume the best scenario is to always have an accelerator (dedicated computer) for critical paths. And the manager is a computer on its own. Side note, last year a colleage of mine pointed out an FPGA architecture that was built to fly in space (the combination of IP cores and Hardware cores), where everything is redundant, to avoid the bit corruption problem. I can try to find a link if there's any point

However, the design is such that it could fly on embedded Linux

oh yes, it's very easy to embed rust these days. I recommend using buildroot for that, which is much simpler than yocto. When it comes to performances, it seems yocto always out performs buildroot, they can probably access better toolchains.
I have never needed performance on the linux (management) side, it's always done in the accelerator (FPGA, GPU..)

I had no idea! Radio work mostly?

I did lots of radio indeed, mostly timing and metrology oriented. It's less frequent these days, I mostly work on video signals, timing, and always linux 🐧 / FPGA 🔌

As an example, the IAU_EARTH frame is built without any
planetary data (no shape, no gravity parameter). Then, when you call
frame_from_uid, the planetary data is loaded for
you, and now the frame has both a shape and a planetary data. That's why all you need to do is define the planetary data,
load it in the Almanac, and then call frame_from_uid. [...] the planetary data is loaded for you, and now the frame has both a
shape and a planetary data. That's why all you need to do is define the planetary data, load it in the Almanac, and then call
frame_from_uid.

OK i was not expecting that internal behavior: that's very important.

Yet, planetary_data_set.to_frame does not return an Error, alwas a Frame, which gives the feeling that augmenting a Frame using the Almanac is always feasible.

if I'm following correctly, let frame = Frame(Earth, ITRF93) is the only option to build a very basic frame, that we will augment using the Almanac. This does not exist, I presume it's Frame::new you meant. It does compile (see the latest version), but planetary_data_set.to_frame(Uid) expects an Uid, not a frame.

I updated the code, check it out. Currently it fails saying WGS84 does not exist in the planetary set, so I'm assuming the definition of the LUT is still wrong:

cargo build -r # update
./target/release/gnss-planetary # build the custom PCK
cargo test # exploitation attempt

I wonder if that could be a problem for you

it will be clearer if I run into it

you have possibly different models of the shape of the Earth depending on the constellation you're using.

Yet it still kind of puzzles me a little bit. If you take a look at the shape values, they're very close to each other and differ at 10E-5. I have not done the maths or am not advanced enough, but to me it looks like a misinterpretation at this level could be neglected. It might just be as stupid as the GPS constellation was introduced 25 years prior all the others. Reading RTKLib has helped me a lot, I would not have the Ephemeris solver without it (too few documentation, RINEX specs do not explain that either). Possibly very similar to you + SPICE. It would help if I could understand how they approach the initial orbits and if a conversion happens at some point prior internal navigation

[gwbres]

The "problem" also applies to SP3 file parsing, which contain 3D coordinates expressed in high precision reference frames like IGS20 and ITRF20. And I am in the process of upgrading SP3 to Orbit as well. So my current solution is to pretend it is ITRF93, but I'm unsure what is the magniture of the error introduced by that.

You said at somepoint that future versions will contain ITRF20 (newest definition). But that is only a partial solution. SP3 files may be expressed in a variaty of reference systems, IGS20 and ITRF20 being only the most common, which we should define rapidly (maybe by this PR/topic).

My current best scenario, is that GNSS provides frame definitions that we can easily pull to define Orbits, and Rinex::navigation::ephemeris needs to take advantage of that and use the correct orbit definition, for any SV

[ChristopherRabotin]

As I understand it, after speaking with Nat Bachman from NAIF, ITRF93 and ITRF2020 are very very close to either other (a few milliradians difference). Spacecraft flown out of NASA use the ITRF93 frame. So let me know what the solutions look like with 93, and if it doesn't work well, we can develop the code to build the Chebyshev interpolations from the EOP data.

…

On Sun, Sep 1, 2024, 05:33 gwbres ***@***.***> wrote: The "problem" also applies to SP3 file parsing, which contain 3D coordinates expressed in high precision reference frames like IGS20 and ITRF20. And I am in the process of upgrading SP3 to Orbit as well. So my current solution is to pretend it is ITRF93, but I'm unsure what is the magniture of the error introduced by that. You said at somepoint that future versions will contain ITRF20 (newest definition). But that is only a partial solution. SP3 files may be expressed in a variaty of reference systems, IGS20 and ITRF20 being only the most common, which we should define rapidly — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEZV2FYFYRAP7KQ4UUVRTDZUL3PVAVCNFSM6AAAAABM33FWSWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGI4TSNRSGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gwbres]

OK very good information.

So let's say if we can apply a high precision reference frame like ITRF93 from the "same class", the error can be neglected (and it might be valid of the vast majority of applications).

But I doubt it is the case when we consider "other classes". IGS and IGb seem to come up quite often, IGS has its own reference frame - for which I have never found the parameters, and I don't even know what IGb is yet