knotgrass / attention Star 29 Code Issues Pull requests several types of attention modules written in PyTorch transformers pytorch transformer attention attention-mechanism softmax-layer multi-head-attention multi-query-attention grouped-query-attention scale-dot-product-attention Updated May 13, 2024 Python
M-e-r-c-u-r-y / pytorch-transformers Star 14 Code Issues Pull requests Collection of different types of transformers for learning purposes transformers pytorch multi-head-attention einsum-notation multi-query-attention Updated Jan 30, 2020 Jupyter Notebook