and I get this error when I run languagetool (the GUI doesn’t start, it’s not just a warning):
WARN o.l.l.i.DefaultLanguageIdentifier fastText not configured - language detection performance will be degraded. See https://dev.languagetool.org/http-server#starting-from-command-line for instructions.
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by com.google.common.util.concurrent.AbstractFuture$UnsafeAtomicHelper (file:/usr/share/java/languagetool/libs/guava.jar)
WARNING: Please consider reporting this to the maintainers of class com.google.common.util.concurrent.AbstractFuture$UnsafeAtomicHelper
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
java.lang.RuntimeException: java.lang.RuntimeException: Could not activate rules
at org.languagetool.gui.LanguageToolSupport.reloadLanguageTool(LanguageToolSupport.java:258)
at org.languagetool.gui.LanguageToolSupport.init(LanguageToolSupport.java:299)
at org.languagetool.gui.LanguageToolSupport.<init>(LanguageToolSupport.java:106)
at org.languagetool.gui.Main.createGUI(Main.java:464)
at org.languagetool.gui.Main.access$1700(Main.java:140)
at org.languagetool.gui.Main$7.run(Main.java:1199)
at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:723)
at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:702)
at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
Caused by: java.lang.RuntimeException: Could not activate rules
at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:94)
at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:84)
at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:74)
at org.languagetool.gui.LanguageToolSupport.reloadLanguageTool(LanguageToolSupport.java:252)
... 14 more
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/de/grammar.xml'
at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
at org.languagetool.Language.getPatternRules(Language.java:711)
at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
... 19 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 513; JAXP00010004: Die akkumulierte Größe von Entitys ist "100.025" und überschreitet den Grenzwert "100.000", der von "jdk.xml.totalEntitySizeLimit" festgelegt wurde.
at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)
at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:178)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.checkLimit(XMLEntityScanner.java:1009)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.checkEntityLimit(XMLEntityScanner.java:968)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanContent(XMLEntityScanner.java:1072)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanContent(XMLDocumentFragmentScannerImpl.java:1575)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2872)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:635)
at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:551)
at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:890)
at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:826)
at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:134)
at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1225)
at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326)
at java.xml/javax.xml.parsers.SAXParser.parse(SAXParser.java:197)
at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:76)
... 22 more
Does adding -Djdk.xml.totalEntitySizeLimit=125000 on the command line help as a workaround? Otherwise, I think JDK24 isn’t supported yet by LT, and you’ll need to use e.g. JDK 17.
Adding -Djdk.. unfortunately doesn’t work (tried it also by adding it to /usr/bin/languagetool (see below for workaround)).
Previous version was 23 (gitlab.archlinux . org/archlinux/packaging/packages/java-openjdk/-/commits/main) btw.
My workaround:
sudo pacman -S jre21-openjdk (v21 is the latest non-24 version with long term support, I guess) (installing a concurrent different version is possible and supported on Arch (see))
I don’t use archlinux-java, as this would change the java version globally
Instead I edited /usr/bin/languagetool and changed the very last line 71 from "$JAVA_HOME/bin/java" -cp $CP -Dsun.java2d.uiScale=4 $GUI_command "${argv[@]}"
to "$JAVA_HOME/usr/lib/jvm/java-21-openjdk/bin/java" -cp $CP -Dsun.java2d.uiScale=4 $GUI_command "${argv[@]}"
Running languagetool works again
The -Dsun.java2d.uiScale=4 is also added by me to fix the GUI being tiny on my 4K screen
I can get all DE unit tests pass except one by adding these two properties: -Djdk.xml.totalEntitySizeLimit=0 -Djdk.xml.entityExpansionLimit=0
It seems that they changed the limit in jdk24 from 5 million to 100k:
I’ve pushed a commit to mitigate that problem in a jdk24 branch
I’ve set the xml limits at more global level and also updated guava to remove a warning.
Unfortunately there’s another warning coming from jna and even updating it to the latest version didn’t make all warnings go away so I didn’t push it.
Note: there are still other 2 places trying to set xml limits for the SAX parser and we may want to consolidate it.
Note: with jdk24 I have one test in [de] failing and also we have some French tests failing starting with jdk21 (I think there were some regressions since we pushed support for jdk21) so I could not verify it all the way.
It would be nice to make LT 6.6 jdk24-compatible.
Thanks for fixing it in v6.6. There are still a bunch of warnings, but the GUI / correction works now.
WARN o.l.l.i.DefaultLanguageIdentifier fastText not configured - language detection performance will be degraded. See https://dev.languagetool.org/http-server#starting-from-command-line for instructions.
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by com.google.common.util.concurrent.AbstractFuture$UnsafeAtomicHelper (file:/usr/share/java/languagetool/libs/guava.jar)
WARNING: Please consider reporting this to the maintainers of class com.google.common.util.concurrent.AbstractFuture$UnsafeAtomicHelper
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::load has been called by com.sun.jna.Native in an unnamed module (file:/usr/share/java/languagetool/libs/jna.jar)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled
It doesn’t seem to be just an Arch Linux issue, but a general OpenJDK-24 issue. Several WritingTool users have reported the same problem, and I was able to reproduce it on Linux Mint with OpenJDK-24. LanguageTool fails to initialize.
You can find a link to a log file here: https://github.com/user-attachments/files/19502023/wt.log
Please let me know when the issue is resolved. Then I can pass it on to my users.
So I had to port some regex fixes for JDK 19 from another branch and add few other changes.
I’ve rebased jdk24 branch on master and pushed it in.
With the exception of German the languagetool-standalone tests all pass on JDK 17, 21, and 24 now.
I see only one unsafe warning from Java about jna, but even upgrading it to the latest version (5.17.0) does not help. The LT works properly though so I guess we’ll have to just ignore it for now.
I’ll see if I can fix German tests so we can merge this into ‘master’ otherwise some languages would be broken on JDK >=19. @dnaber I would appreciate if you could help me with [de], thanks.
@SteVio1989 Do you remember what the issue was with JDK 24?
[ERROR] Errors:
[ERROR] org.languagetool.rules.de.GermanPatternRuleTest.testRules
[ERROR] Run 1: GermanPatternRuleTest>PatternRuleTest.lambda$testGrammarRulesFromXML$1:536->PatternRuleTest.testCorrectSentences:822->PatternRuleTest.addError:502 » PatternRuleTestFailure Test failure for rule ABKUERZUNG_LEERZEICHEN[3] in file /org/languagetool/rules/de/grammar.xml (line -1): Did not expect error in:
Das ist richtig, s. u.
Analyzed token readings: [/SENT_START*] Das[der/ART:DEF:AKK:SIN:NEU*,der/ART:DEF:NOM:SIN:NEU*,der/PRO:DEM:AKK:SIN:NEU*,der/PRO:DEM:NOM:SIN:NEU*,der/PRO:PER:AKK:SIN:NEU*,der/PRO:PER:NOM:SIN:NEU*,O] [ /null*] ist[sein/VER:3:SIN:PRÄ:NON,sein/VER:AUX:3:SIN:PRÄ,O] [ /null*] richtig[richtig/ADJ:PRD:GRU,richtig/ZUS,O] ,[,/PKT*,O] [ /null*] s[s/null,O] .[./PKT*,O] [ /null*] u[u/ABK:wort=und:PAR,O] .[./SENT_END*,./PKT*,O]
Unfortunately, the SentenceRangeTest is not correct, the sentence range fix change to it is not what our client apps expected. And this also was the reason we reverted the JDK19 change 1 day later last year.
This is what our clients expect: commit from JDK19 Branch. I just updated the main branch so that this test case is also included there.
Seems to be difficult to update all our clients to support the approach from the sentence range fix commit here.
Unfortunately, we did not have time to address this issue.
@arysin if you can resolve the issue with the SentenceRangeTest, I would create a test-server for our frontend team to test against it, before merging again to the main branch.
@dnaber I am not aware of specific issues with JDK24 as this was only released a few months ago, and we did not test it (because of the known issues with >=JDK19