Feature: Parallel merge

[giang-nghg]

Batch merging currently process batches sequentially in order to maintain the ordering of samples in the original input. But in cases this is not necessary, batches can be processed in parallel to speed up the entire process.

[Zhicheng-Liu]

This is of low priority, imho. There are work around for some users who have access to compute clusters. It would be more efficient in terms of overall time to break down a large number of files to be merged into multiple chunks and merge them in separate jobs in multiple steps. If designed carefully, the ordering of samples in the original input can be still easily maintained.

[Zhicheng-Liu]

Even with batch processing in parallel, the ordering of samples can still be maintained. That's the contract we promised and we should not break it. Most users would expect that imho.

[martinghunt]

I wouldn't care about the order of the samples in the final merged VCF file.