Very bad performance when integrating into Java application

Development
1

We are evaluating LanguageTool to integrate it into our own (Spring based) Java application, but we only need a basic spell check (plus user specific words to ignore)

So I followed Embedding LanguageTool in Java applications | dev.languagetool.org and created a simple main class:

Language de = Languages.getLanguageForShortCode("de-DE");
JLanguageTool lt = new JLanguageTool(de);

String toCheck = "Dieses Vorgehen scheint zielführend";
List<String> wordsToIgnore = Arrays.asList("foobar");

for (Rule rule : lt.getAllRules()) {
    if (rule instanceof SpellingCheckRule scr) {
        scr.addIgnoreTokens(wordsToIgnore);
    } else {
        lt.disableRule(rule.getId());
    }
}

List<RuleMatch> matches = lt.check(toCheck);

When I run the above code, the call lt.check() takes about 4-5 seconds which is unexpected (especially because only a single rule is active).

Is this because I am calling this only once from within the main() method and LanguageTool needs some “warming up” time?

If this is expected, is there a way to speed this up? We don’t expect to have strings that are longer than approx. 1000 characters to check.

Disabling of the rules also takes 1-2 seconds. But as JLanguageTool is not thread safe, we would need to do this for each “check request”, or is there a way to avoid that?

2

Yes, there’s a lot of warm-up time. The easiest fix/workaround is to initialize LT and check some fake text before you start sending real texts.

3

Thanks for the answer. But the documentation says, JLanguageTool isn’t thread safe and I need to create a new instance each time I want to check a text.

4

Please try creating a new JLanguageTool each time, but using the same language object. That should be fast.

5

I have tried disabling all non-dictionary based spelling rules and using the same language object per-request and am still seeing very slow performance. For medium length text in the range of 500-5000 characters it can take 5-35s to spellcheck the text. Do I need to allocate more ram to the server, do I need to tweak the settings, is there anything else that may help speed up this service?

Are there any smart things I can do on the frontend to split the text up into smaller chunks and then do some caching on the server for the most common words being spellchecked? Now I’m curious how web browsers and text editors do it all so fast and so seamlessly cause this sure is not that.

6

@dajms3 Is that even after warm-up? For which language is this?

7

This is for en-US and it is after warmup. For text of about 6800 characters, it takes 13s for the first spellcheck, and about 7.5s for additional spellchecks. A quick spellcheck of around 680 characters runs disproportionately faster, about 200ms. This is with only dictionary spelling rules enabled and calling the library with jLanguageTool.check(String text) I am using the package language-en 6.8

Seems like spellchecking is resource dependent, as machines with less memory run significantly slower than machines with more memory. The server, which originally only had the memory it needed, runs it about 3x as slow, vs a developer machine with more free memory runs it faster.

8

This might also be caused by the number of errors, as generating the suggestions for spelling errors is somewhat slow.

9

Had some more time to look at this, and your last comment about number of errors seems to be accurate. After cold start and adding all of the “errors” to our “dictionary”, it was only taking about 1-2s round trip to spellcheck content with approx. 7000 characters and minimal errors.

Is there a way to configure the spellchecker to generate fewer or less accurate suggestions per-word? We only use the top 5 suggestions and discard the rest.

I greatly appreciate the insights you were already able to provide. I may be able to squeeze a little extra performance out of this by caching spellchecking results / suggestions on the server.