Hi all
In Ukrainian module we have some abbreviations with period added to the dictionary. Properly handling them required some non-trivial work in tokenization logic but it was definitely worth the effort.
Though now I have some rules that use such words in tokens and LT complains:
token [2], contains “р.” that contains token separators, so can’t possibly be matched
I wonder if it makes sense to add a flag allowing either language or token to allow dots so warning is not shown.
Thanks
Andriy