Development guide

[jgromes]

Hello everyone,

I'm looking into developing for GNURadio using gr-satellites as the starting point - I'm mainly interesting in writing some encoder counterparts for the existing decoder blocks. I'm using gr-satellites as the starting point becuase the decoders here are known to be working (and because I have very little idea about developing for gnuradio).

Perhaps somewhat naively, I though that I could e.g. add an AX.100 encoder by just copying the relevant files, renaming them to "ax100_encoder_*", changing the internals, editing all realted CMake files and rebuilding gr-satellites from source.

This however seems to have failed at some point, because I now get the following error (with PYTHONPATH set to /usr/local/lib/python3/dist-packages/):

Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import satellites
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/satellites/__init__.py", line 47, in <module>
    from .bindings.satellites_python import *
ModuleNotFoundError: No module named 'satellites.bindings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/dist-packages/satellites/__init__.py", line 49, in <module>
    from .satellites_python import *
ImportError: /usr/local/lib/python3.10/dist-packages/satellites/satellites_python.cpython-310-x86_64-linux-gnu.so: undefined symbol: _Z17bind_ax100_encodeRN8pybind117module_E

I think that it's related to generating Python bindings for the C code, but I don't really know how to do that (and was hoping CMake would take care of it). On a whim, I tried running python/bindings/bind_oot_file.py, but that produces rather strange errors (and don't know if I'm even using it correctly):

$ python3 python/bindings/bind_oot_file.py --module ax100_encode --output_dir python --src lib --filename include/satellites/ax100_encode.h 
INFO Parsing source file "/home/jgromes/gnuradio/gr-satellites/include/satellites/ax100_encode.h" ... 
In file included from <built-in>:440:
<command line>:3:9: error: macro name must be an identifier
#define -U__PIE__ 1
        ^
1 error generated.
Traceback (most recent call last):
  File "/home/jgromes/gnuradio/gr-satellites/python/bindings/bind_oot_file.py", line 57, in <module>
    bg.gen_file_binding(args.filename)
  File "/usr/lib/python3/dist-packages/gnuradio/bindtool/core/generator.py", line 197, in gen_file_binding
    raise(e)
  File "/usr/lib/python3/dist-packages/gnuradio/bindtool/core/generator.py", line 184, in gen_file_binding
    header_info = parser.get_header_info(self.namespace)
  File "/usr/lib/python3/dist-packages/gnuradio/blocktool/core/parseheader_generic.py", line 330, in get_header_info
    decls = parser.parse(
  File "/usr/lib/python3/dist-packages/pygccxml/parser/__init__.py", line 51, in parse
    declarations = parser.read_files(files, compilation_mode)
  File "/usr/lib/python3/dist-packages/pygccxml/parser/project_reader.py", line 264, in read_files
    return self.__parse_file_by_file(files)
  File "/usr/lib/python3/dist-packages/pygccxml/parser/project_reader.py", line 292, in __parse_file_by_file
    decls = reader.read_file(header)
  File "/usr/lib/python3/dist-packages/pygccxml/parser/source_reader.py", line 303, in read_file
    return self.read_cpp_source_file(source_file)
  File "/usr/lib/python3/dist-packages/pygccxml/parser/source_reader.py", line 322, in read_cpp_source_file
    xml_file = self.create_xml_file(ffname)
  File "/usr/lib/python3/dist-packages/pygccxml/parser/source_reader.py", line 263, in create_xml_file
    raise RuntimeError(
RuntimeError: Error occurred while running CASTXML xml file does not exist

I searched the documentation, as well as the issues/discussions but didn't find much info on how to develop for gr-satellites. Is there some sort of "development guide"? Though perhaps what I'm actually looking for is a development guide for GNURadio in general, so if anyone has any pointers, that would be really appreciated.

Thanks!

[daniestevez]

Hi Jan, these are the pybind11 bindings, which are required for every C++ block starting with GNU Radio 3.9. Usually running gr_modtool bind block_name will handle correctly the generation of bindings. However, it's good to have an idea about how these work, in case there are problems. There is a bind function per block in its own C++ file (example), which needs to be listed in CMakeLists.txt, and then all of these are called in python_bindings.cc.

This topic is not specific to gr-satellites. It applies to GNU Radio >= 3.9 C++ blocks in general. I took a quick look at the GNU Radio Wiki and I couldn't find something that explains pybind11 in enough detail. The C++ block tutorial assumes that you create the block with gr_modtool add and never touch the block public header (if you do you need to update the bindings). If you go to the documentation channel on the GNU Radio Matrix chat, maybe you can persuade someone to improve the documentation, or at least give the feedback that the documentation seems to be lacking.

[jgromes]

Hi Dani, thank you very much for the reply. I already did most of the things you mentioned, except for gr_modtool bind, doing that in the root of the gr-satellites repository literally resolved everything - guess I was not expecting it to be this straight-forward.

Thank you also for the link to GNURadio OOT tutorial, I will definitely need to read up on that. This is pretty much just an excuse to get to know GNURadio better.

By the way, are encoders for the various implemented decoders something you would be intersted in including in gr-satellites? Or is it out of scope for the project?

[daniestevez]

By the way, are encoders for the various implemented decoders something you would be intersted in including in gr-satellites? Or is it out of scope for the project?

I would be interested in including this. It would be in-scope, no doubt. This is something that has never been implemented for several reasons: not many satellites allow uplink to people besides the owners; it can be tricky to get a transmitter that has good compatibility with many SDR hardware, due to the differences in handling burst transmissions and the potential need to control external hardware.

However, there have been already some efforts to include some uplink support. There are the varlen_packet_framer and varlen_packet_tagger that Glenn Richardson was using for uplink and upstreamed. At some point I also had an ESA summer of code in space proposal to add some uplink features, but this never got selected.

I don't have a clear idea about how to include an end-to-end uplink solution integrated in the gr_satellites command line tool and components architecture, but there's nothing wrong with having a handful of blocks which are not used in this architecture. There are already many examples of these, such as the Doppler Correction block or the CCSDS blocks.

[jgromes]

OK, thanks for the feedback - I'll open a dedicated issue/PR once I have something worth while. My main use case (which perhaps could also benefit other satellite teams) is ground support equipment, it's simply much easier to do basic functional testing/hardware-in-the-loop tests with SDR rather than the actual GS hardware.

There's also the potential use-case of support for end-of-life transceivers. For example, the GomSpace AX.100 is built on the AX5043 transceiver which has been marked as end-of-life last year if I'm not mistaken, so once the stocks run out, SDRs will be the only game in town for the satellites still flying those.

[jgromes]

@daniestevez Hello again, I had some time to work on this, and managed to implement an OOT block for encoding and framing for the GomSpace AX100. Testing against the decoder it seems to be working. All the changes currently reside in my fork (https://github.com/vzlu/gr-satellites/tree/ax100_encode), not ready for PR yet (sorry).

I tried to follow the implementation logic of the decoding chain, so now there's a couple new blocks:

1. u482c_encode which takes the input packets and performs length encoding, FEC and scrambling
2. pdu_insert_bytes which basically acts as opposito to pdu_head_tail - this is used to add the preamble and sync word
3. ax100_framer which connects it all together - takes in the packets as message and produces a stream of bytes.

However, I'm unsure where exactly should we make the cut between the ax100_framer block and the next modulation steps. For example, before the FSK modulation, you still need to unpack the bytes into bits and convert to float. I think it makes sense to have that as a different hierarchical block from the ax100_framer, because it could be reused for other framing protocols. On the other hand, the original ax100_deframer block already contains the digital slicer, so its input is floats. See the flowchart below.

Another question is do you think it makes sense to implement a new FSK Modulator block, analogous to the currently implemented FSK Demodulator? If so, then I'm afraid it's a bit out of my area of expertise. Right now I'm doing FSK modulation using the the builtin gnuradio blocks which I sort of just stitched together until I arrived at a configuration that demodulates afterward.

[daniestevez]

Thanks for the update!

The drawback of using PDUs for a transmitter chain is that these are not able to pass backpressure to the source. Backpressure is required in many applications. In my opinion a transmitter chain in GNU Radio should be implemented using a tagged stream (which is not the same as tagged stream blocks; these often bring more problems than they solve).

the original ax100_deframer block already contains the digital slicer

Deframer blocks in gr-satellites are standardized to work with a float output because some of them take advantage of working with soft symbols. For a framer this does not apply, and probably the most natural output is bits (or dibits if the output alphabet has four element, etc.). Unpacking outside of the framer has the disadvantage that you are pushing the responsibility for handling the unpacking configuration somewhere else: some protocols transmit bytes in MSB order and others in LSB order. Also, outputting packed bytes doesn't work if the number of bits in your packets is not a multiple of 8.

Another question is do you think it makes sense to implement a new FSK Modulator block, analogous to the currently implemented FSK Demodulator?

It depends on what you're after. At the beginning, gr-satellites had GRC flowgraphs. Only when it became impossible to maintain dozens of different flowgraphs I created the demodulators and deframers with the idea of being able to instantiate a flowgraph in Python from a YAML configuration. If you have a single scenario in mind (as in symbol rate, sample rate, pulse shape and deviation), then probably the in-tree GFSK Mod block or a few other blocks accomplishing a similar function are fine.

[jgromes]

The drawback of using PDUs for a transmitter chain is that these are not able to pass backpressure to the source

The issue is that I will most likely have to start with PDUs - the original source of the data will be packets sent from some control application.

In my opinion a transmitter chain in GNU Radio should be implemented using a tagged stream

As far as I can tell that should be the case - I forgot to mention that the final step in the framer block is pdu_to_tagged_stream, so the output should be (I think) a tagged stream of bytes, which then gets processed further.