# Performance: `/v2/check` significantly slower than Java-LanguageTool (~7× high…er CPU time per character)
**Environment:** LingoTweaker `0.1.0-alpha.1` (`lt-cli serve --addr 127.0.0.1:8081`, `LT_DATA_DIR=<repo>/data`, release build), Debian 13, 4 cores, 8 GB RAM, nginx reverse proxy.
**Baseline for comparison:** LanguageTool (Java) 6.8 on the same machine, same nginx, same test texts.
## TL;DR
In a 5-minute load test (4 parallel workers, mixed text lengths, German), LingoTweaker is slightly faster on **short** texts but dramatically slower on **medium/long** texts. The CPU time consumed per request is **~7× higher** than with Java-LanguageTool — while the total CPU load is nearly identical (~67% of 4 cores). In other words, the machine is CPU-saturated in both cases, but LingoTweaker completes ~6.6× fewer requests.
| Metric | Java-LT 6.8 | LingoTweaker 0.1.0-alpha.1 | Factor |
|---|---:|---:|---:|
| Throughput (requests / 5 min, 4 workers) | **862** | 130 | 6.6× less |
| CPU time / request (avg) | **936 ms** | 6,733 ms | **7.2× more** |
| Total CPU time (5 min) | 807 s (66.4% of 4 cores) | 875 s (67.7% of 4 cores) | ~same |
| Memory (RSS, at end) | 2,805 MB | **1,646 MB** | better |
| Peak memory (RSS) | 2,923 MB | 3,741 MB¹ | more |
| Max text length without error | ~124,500 chars² | **60,000 chars** (hard limit, HTTP 400) | limited |
| Stability under load | OOM crash on a ~124k-char text | stable (no crashes in the test) | better |
¹ The peak was reached before the measured run, during size calibration; during the 5-minute run lt-cli stayed stable at 1.64–1.70 GB.
² Java-LT crashed with `java.lang.OutOfMemoryError: Java heap space` on a single ~124,500-char text (heap limit ~2.4–2.9 GB).
## Latency per text length (response time, client perspective, incl. nginx)
Weights in the test: short 40%, medium 25%, long 20%, very_long 15%. All values from the 5-minute runs, 0 errors in both.
| Category | Text size | Java-LT n | Java median | Java p95 | Lingo n | Lingo median | Lingo p95 | Lingo/Java (median) |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| short | ~127 chars | 336 | **89 ms** | 157 ms | 60 | **72 ms** | 131 ms | 0.81× (Lingo faster) |
| medium | ~1,240 chars | 237 | **260 ms** | 442 ms | 25 | **492 ms** | 911 ms | 1.9× |
| long | ~12,450 chars | 164 | **2,021 ms** | 2,860 ms | 23 | **8,160 ms** | 10,157 ms | 4.0× |
| very_long | ~31,100 chars | 125 | **5,883 ms** | 7,320 ms | 22 | **46,859 ms** | 52,651 ms | 8.0× |
Full distributions (min/mean/median/p90/p95/p99/max) are in the attached JSON reports.
### CPU time per character (better comparison metric, since it is text-dependent)
Roughly estimated from the medians (response time ≈ CPU time under saturation):
| Size | Java-LT | LingoTweaker | Factor |
|---|---:|---:|---:|
| 127 chars | ~0.7 ms/char | ~0.57 ms/char | 0.8× |
| 1,240 chars | ~0.21 ms/char | ~0.40 ms/char | 1.9× |
| 12,450 chars | ~0.16 ms/char | ~0.65 ms/char | 4.0× |
| 31,100 chars | ~0.19 ms/char | ~1.51 ms/char | 8.0× |
The gaps between sizes (Java: 0.21 → 0.16 → 0.19) are measurement noise under saturation; the trend is flat for both, but strongly rising for LingoTweaker. **Hypothesis:** the pattern-rule engine (`lt-pattern`) and/or the tagger (`lt-tagger`, `lt-disambig`) do not scale linearly with text length — e.g. due to repeated regex/backtracking over growing windows, or a lack of incremental processing. A `perf record`/flame graph during a single `very_long` check would reveal the hotspot.
## Observed API behavior (relevant for interoperability)
1. **Only long language codes accepted:** `language=de` → `501 {"error":{"message":"The language de is not supported by this LingoTweaker build yet."}}`; `language=de-DE` → 200. Java-LT accepts both (`de`, `de-DE`). Since the v2 API documentation uses short codes, this breaks existing client code.
2. **Text only read from the POST body:** `/v2/check?language=de-DE&text=...` (query string) → `400 {"error":{"message":"Missing required parameter: language"}}`, i.e. query parameters are not read. Java-LT reads the text from the query *and* the body.
3. **Hard text limit of 60,000 characters:** longer → `400 {"error":{"message":"Text too long: limit is 60000 characters"}}`. Java-LT is (practically) limited only by the heap.
4. **Very slow single responses:** ~50k chars > 60 s → nginx `504 Gateway Time-out` (default `proxy_read_timeout 60s`). With `proxy_read_timeout 300s`, 50k-char texts got through; ~56k chars caused a brief `502` (lt-cli unresponsive for ~30–60 s, no process crash, `NRestarts=0`).
## Test methodology (reproducible)
Attached script: **`lt_perf_test.py`** (Python 3, dependency: `requests`; CPU/memory measurement requires `sudo` access for `systemctl show`).
### Invocation
```bash
# LingoTweaker (long code de-DE, POST body)
python3 lt_perf_test.py lt-cli [report.json]
# Java-LanguageTool (short code de, URL query)
python3 lt_perf_test.py languagetool [report.json]
```
The script automatically sets `LANGUAGE` and `USE_POST_BODY` based on the service name and writes a JSON report (default `/home/jochen/<service>_perf_result.json`).
### Configuration in the script
```python
LT_URL = "https://lt.fluid8.de/v2/check" # nginx URL; for a direct test: http://127.0.0.1:8081/v2/check
DURATION_SECONDS = 300 # 5 minutes
CONCURRENCY = 4 # parallel workers
REQUEST_TIMEOUT = 90 # s per request
CATEGORIES = [
("short", 20), # ~127 chars
("medium", 200), # ~1,240 chars
("long", 2000), # ~12,450 chars
("very_long",5000), # ~31,100 chars
]
```
Texts: a fixed sentence corpus (~1,800 German words, with deliberate grammar errors), cycled in random order up to the target word count, 10 variants per category, one picked at random per request. Distribution: short 40% / medium 25% / long 20% / very_long 15%.
### What is measured
- **Latency** per request (`time.perf_counter` around `requests.post`, client perspective incl. nginx/TLS)
- **CPU** of the backend process: `CPUUsageNSec` via `sudo systemctl show <service> --property=CPUUsageNSec,MemoryCurrent,MemoryPeak` (baseline before start, a sample every 5 s, final after the end; delta = test consumption)
- **Memory** of the same process: `MemoryCurrent` / `MemoryPeak` (systemd RSS)
Service names on our server: `languagetool.service` (Java: `java -cp languagetool-server.jar org.languagetool.server.HTTPServer --port 8081 --config /var/www/languagetool/languagetool.conf`, OpenJDK 21.0.12) and `lt-cli.service` (Rust: `/opt/lingotweaker/repo/target/release/lt-cli serve --addr 127.0.0.1:8081`, `Environment=LT_DATA_DIR=/opt/lingotweaker/repo/data`). Both run as user `languagetool`, sandboxed, with `MemoryAccounting`.
### nginx prerequisites (needed for long texts)
```nginx
server {
...
client_header_buffer_size 32k;
large_client_header_buffers 4 1m; # for Java-LT with query string > 8kB
...
location / {
proxy_pass http://localhost:8081;
proxy_read_timeout 300s; # LingoTweaker "very_long" > 60s
proxy_send_timeout 300s;
...
}
}
```
Additionally: a rate-limit exception for the test domain (otherwise 429 from ~20 req/s + burst 100 per IP).
### Measurement uncertainties / notes
- Both processes were **CPU-saturated** during the test (~67% of 4 cores), i.e. latencies include time waiting in the thread queue; CPU time per request is the more robust metric.
- LingoTweaker only produced 130 samples (vs. 862 for Java) because of the long single responses — p95/p99 of the long categories are based on 22–25 samples.
- Java-LT baseline: the process was freshly started and clears its cache between runs; the LingoTweaker peak memory comes from the pre-run calibration.
- For a clean A/B comparison, run both tests back to back (the services are mutually exclusive via `Conflicts=`), each freshly started, 300 s each.
### Suggested follow-up measurements in the repo
1. **Micro-benchmark:** `cargo run -p lt-cli -- examples --lang de` or `Engine::check` directly in a `#[bench]` (criterion) for the four text sizes, without the HTTP layer — separates engine performance from axum/network overhead.
2. **Flame graph:** `perf record -g` during a single 31k-char check → top frames of the `lt-pattern`/`lt-tagger`/`lt-disambig` crates.
3. **Scaling curve:** text lengths 1k/5k/10k/20k/40k/60k, n=20 each, sequential (1 worker) → ms/char per size; shows whether the scaling is super-linear.
4. **Check parity:** run the same example sentences from the `<example>` tests against both engines (matches, not just runtime) — performance optimization must not lose rules.
## Attachments
- `lt_perf_test.py` — performance script (see above)
- `lt_perf_result.json` — full report Java-LT 6.8 (5 min, 862 requests)
- `lingotweaker_perf_result.json` — full report LingoTweaker 0.1.0-alpha.1 (5 min, 130 requests)
[lingotweaker_perf_result.json](https://github.com/user-attachments/files/32540384/lingotweaker_perf_result.json)
[lt_perf_result.json](https://github.com/user-attachments/files/32540386/lt_perf_result.json)
[lt_perf_test.py](https://github.com/user-attachments/files/32540385/lt_perf_test.py)