Add a fix for special added tokens

[mokeddembillel]

This PR fixes the decoding of manually added tokens such as \n in this case. The latter is getting decoded as Ċ.

Generation before the fix:

Prompt: Once upon a time in a land far away,
there was a kingdom ruled by a wise and just king. The kingdom was known for its beauty and prosperity, and the people lived in peace and harmony.ĊĊOne day, a terrible drought struck the land, and the crops began to wither and die. The king, worried about the well-being of his people, called upon his wise council to find a solution. The council, after much deliberation, decided to send a group of brave knights to search for a magical spring that was said to have the power to bring rain to the kingdom.

Generation after the fix:

Prompt: Once upon a time in a land far away,
there was a kingdom ruled by a wise and just king. The kingdom was known for its beauty and prosperity, and the people lived in peace and harmony.

One day, a terrible drought struck the land, and the crops began to wither and die. The king, worried about the well-being of his people, called upon his wise council to find a solution. The council, after much deliberation, decided to send a group of brave knights to search for a magical spring that was said to have the power to bring rain to the kingdom.