Missing license #35
Description
@wangkuiyi, I appreciate this excellent tokenizer library in C++. But I noticed that the repository is missing license information. Would you be able to add licensing terms to the package?
I see there have been previous requests about this (refs: #34, #31, #32), including one where a community member proposed a license file.
Please let me know if there are any specific reasons the license hasn't been added or if you need any assistance in determining/reviewing appropriate licensing terms.