Ability to write to a pipe from memray.Tracker

[jhance]

Is there an existing proposal for this?

• I have searched the existing proposals

Is your feature request related to a problem?

I want to write to a fifo instead of a file but memray complains if the file I point it to already exists. The end problem I am trying to solve is that I want to stream the result to a distributed file system instead of place it on the disk.

Describe the solution you'd like

I'd like to write to a pre-existing fifo pipe.

Alternatives you considered

If memray could just give me the bytes in an iterator that would be fine too.

[godlygeek]

As things stand today, this isn't simple. Memray expects to be able to mmap the output file, and pipes cannot be mmapped.

We do support binding to a TCP port, waiting for a client to connect, and then writing the captured data over that socket - https://bloomberg.github.io/memray/api.html#memray.SocketDestination

That's intended for use by the memray live TUI, but you might be able to use that for your needs by spawning a thread or process that connects to that TCP port and writes everything it receives to your FIFO.

[sarahmonod]

After talking about it a bit, we decided we would address this in two separate ways:

1. Add documentation for writing a wrapper script that smuggles the capture file out: Explain how to run memray and send file over the network for containers #598
2. Add a way for the Tracker API to take a callback for user post-processing: Add callback to the Tracker API to post-process the capture file #599

Closing this in favor of the other two.