Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On files bigger then 33MBs #77

Open
lnee94 opened this issue Sep 9, 2023 · 1 comment
Open

On files bigger then 33MBs #77

lnee94 opened this issue Sep 9, 2023 · 1 comment

Comments

@lnee94
Copy link

lnee94 commented Sep 9, 2023

First, the problem would be telling if a group of cimbars are linked that could be done by having the first byte be the control byte
this could look something like this 10000000 and have a second byte for a index number if the indexed bit is one this model assumes that each cimbar is not mixed with each other, so cimbar 1 can't have images from cimbar 2.
Second, using a file container. The most popular is zip which just so happens to have a split function
zip in.zip --out new.zip -s 33m this will output a zip file with a bunch of z01 z02... files so having a index number is nessisery. (note to resembel run zip -F new.zip --out out)
Note there should be a mode where you can make a cimbar as long as you want, but you have to order them your self.
The index number plus zip would set the max size to, 8448MB with muiltable files with names and the ability to encrypt the files.

@sz3
Copy link
Owner

sz3 commented Sep 16, 2023

Something like that should work, yeah. Right now we don't transfer any information that could be used to correlate across files, but (1) it's possible to embed the information in the payload itself, or (2) we could modify the protocol as you're suggesting. (3) another option is to use the "skippable frame" functionality of zstd to attach some metadata to the transfer.

I was wary of including a filename field in cimbar initially, but I think it may be reasonable for future versions, and it might be useful for this.

@sz3 sz3 mentioned this issue Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants