Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunking for uBAM #140

Open
rhpvorderman opened this issue Oct 7, 2024 · 2 comments
Open

Chunking for uBAM #140

rhpvorderman opened this issue Oct 7, 2024 · 2 comments

Comments

@rhpvorderman
Copy link
Collaborator

Currently chunking only works for FASTQ. See marcelm/cutadapt#811

@marcelm
Copy link
Owner

marcelm commented Oct 7, 2024

Oh, interesting, I guess this needs to be done on the bgzip-level?

@rhpvorderman
Copy link
Collaborator Author

No, not really. Bgzip is just concatenated gzips. There is no requirement for the bgzips to be split at the bam record level. A bam record can start in one block and end in another, even if it could fit entirely in a block of its own. Nanopore records often will exceed the maximum size of a bgzip block.

So we can just decompress the whole thing as one big filestream and parse the records out. We already do this for single-end. For chunking we can make use of the fact that BAM records store their block sizes at the beginning. So there is no need to read the entire block. Chunking should be much faster than for FASTQ.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants