New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for drums/percussion? #30
Comments
@tripathiarpan20 -- I found your comment interesting, so I took a short dive into the literature. There's a niche, and interesting, sub-sub-field of Music Information Retrieval (MIR) called Automatic Drum Transcription (ADT). Here's a literature review of ADT. The authors of that review describe different "drum transcription tasks" -- drum-only transcription (DTD) and drum-plus-accompaniment transcription (DTM) seem particularly relevant. If you want to "solve" drum encoding, you could look at some of the methods in the more recently referenced papers in the mentioned literature review and give them a try! Ref 80 appeared to have high scoring metrics, but might not work for drum kits with more than a kick, snare, and hi-hat. The authors (of ref 80) also have a GitHub repo, and a demo site linked! For another approach, you might find https://github.com/magenta/mt3 interesting/useful. Unfortunately, the related paper doesn't focus too heavily on drums, so you might find the Finally, perhaps we could make use of Facebook's demucs NMP(E-GMD)
original track -------> drum-only track -----------> drum-only MIDI I'll give this a try, and post on the results. Luckily, since NMP is so light it probably trains much faster than huge models, And who knows, maybe |
@tripathiarpan20 no plans at the moment, but will let you know if that changes. @jugoodma 's comment is great, and points to some open source drum transcription options. Here are two more open source systems I'm aware of: |
Hi @jugoodma and @rabitt , To be frank I am not familiar with how the instrument class is predicted in the NMP pipeline, but if retraining the Basic Pitch's architecture on Drum dataset for DTD along with devising the suitable posteriorgram post-processing works, I believe that it would make the domain of instruments in this project truly whole (afaik). Good luck on the process and keep us updated :D. Elsewhere, I also tried Meanwhile, perhaps Spotify could develop a lightweight demixing model which might benefit from end-to-end deep learning that uses CQT for preprocessing (rather than Mel spectrograms as in past demixing methods) in the future? Any feedback from anyone else is welcome too! |
@jugoodma Did you get around to attempting retraining as described above?
original track -------> drum-only track -----------> drum-only MIDI |
Hi!
Thanks for this amazing open-source work, I'm really enjoying using it. :)
I noticed that Basic Pitch works great for tracks with single mono/polyphonic instrument for most instruments, however it is unable to encode drums at all.
I understand that MIDI encoding for drums/percussion instruments is somewhat different compared to the rest of the instruments, but are there any future plans to add support for percussion instruments?
The text was updated successfully, but these errors were encountered: