Use VISTEC-TP-TH-21 dataset for Thai

[wannaphong]

Hello! Thank you for new word segmentation. I think you should use VISTEC-TP-TH-21 dataset. It is CC-BY-SA and is the largest social media domain datasets for Thai text processing.

VISTEC-TP-TH-21: https://github.com/mrpeerat/OSKut/tree/main/VISTEC-TP-TH-2021

[wannaphong]

You can see more about Thai dataset at https://nlpforthai.com/tasks/word-segmentation/

[sffc]

CC @younies @SahandFarhoodi

[0saurabh0]

@sffc, for the clarification, the intention is to train a new model using this dataset or to retrain an existing one?

[sffc]

The intention is to design a model that outperforms existing models on one or more axes including memory usage, model size, accuracy, and performance. Re-training using the new data set is also something we should explore, but that is separate from the chosen model.