Code

documentation for task#2

Code

Open

natek-1 wants to merge 10 commits intoLAION-AI:mainLAION-AI/snac-to-llama3:mainfrom natek-1:mainnatek-1/snac-to-llama3:mainCopy head branch name to clipboard

+5,536-4,060

Commits

Commits on Jun 20, 2024

• updated folder sturcture

  natek-1

  committed
  5109bb3

• created a prilimilary versoin of vocabulary

  natek-1

  committed
  95f97ed

Commits on Jun 22, 2024

• started hugging face implimentation of BPE

  natek-1

  committed
  50c02f8

Commits on Jun 29, 2024

• udpated readme file:

  natek-1

  committed
  56c336d

• updated codec to token

  natek-1

  committed
  f2b1cc2

Commits on Jul 1, 2024

• created the split of string and list of codes

  natek-1

  committed
  fb18089

Commits on Jul 4, 2024

• updated pair_freqs method for task

  natek-1

  committed
  09844af

Commits on Jul 5, 2024

• added the algorith

  natek-1

  committed
  c0b15ed

Commits on Jul 6, 2024

• added tokenize method

  natek-1

  committed
  e5df127

Commits on Jul 10, 2024

• fixing some bugs

  natek-1

  committed
  b18c52d