the special tokens of XLNet is different from BERT

[lytum]

hallo everyone,

may i ask you, if the special tokens of XLNet are same as BERT? We all know, the special tokens of BERT are [CLS] and [SEP]. and many public introduction of XLNet also use [CLS] and [SEP] as the special tokens. but I noticed that the official documents of XLNet which use and as their classification and seperate tokens. which is right?

[wangcongcong123]

Have a look at this.
https://huggingface.co/transformers/model_doc/xlnet.html#xlnettokenizer
Hope it helps.

[lytum]

Have a look at this.
https://huggingface.co/transformers/model_doc/xlnet.html#xlnettokenizer
Hope it helps.

Thanks for your information. yes, I also looked at this official documentation. They use
sep_token='< sep >', cls_token='< cls >', as the XLNet special tokens. However, there are many public introduction for fine tuning XLNet, which they still use the same tokens as BERT. such as:

https://github.com/Shivampanwar/Bert-text-classification/blob/master/XLNet/xlnet_experimentation.ipynb

https://mccormickml.com/2019/09/19/XLNet-fine-tuning/

so i am confused.