"Add Term" fails if dictionary size > php memory_limit

[sarahmccuan]

Describe the bug
If a dictionary is larger than the php memory_limit value, e.g. because of data import, querying it by clicking on a text word fails and results in an error (see screenshot).

To Reproduce
Steps to reproduce the behavior:

1. Load a large dictionary (>128 mb) - in this case I loaded both large German dictionaries
2. Load a text to read
3. Click on an unknown word & click "Add" button
4. Popup loads for a minute, then errors

in php logs:

GET /api/v1/terms/for-edit?term_id=1&ord=3
  → PHP Fatal error: Allowed memory size ... exhausted in PreparedStatement.php on line 209

<b>Fatal error</b>: Allowed memory size...{"error":true,"message":"An unexpected error occurred",...}

Note, headers are flushed so this shows up as a 200.

Server (please complete)
Server
LWT version v003004002
Web Server Apache/2.4.68 (Debian)
Server Software Apache/2.4.68
Server Location http://localhost:8010
PHP Version 8.4.24
Database
Database name learning-with-texts
Database Size 209.6 MB
MySQL Version 12.1.2-MariaDB-ubu2404
Client API
REST API Version 3.4.2
REST API Release date 2026-08-16

• OS: Windows
• Browser: Vivaldi (Chromium)

---

** Additional info**

I think this is b/c the whole database is getting pulled into memory

SELECT WoID, WoTextLC, WoStatus, WoLemmaLC FROM words WHERE WoLgID = ? AND WoTextLC <> ?

I'll do a bit of poking at different ways to resolve, might make sense to just raise the memory limit & accept that it'll be a slower lookup. Will open a PR if I can come up with something that seems reasonable.

[HugoFara]

Did you use the standard German dicts proposed with the app? I tested them recently with #250 , so I'm a bit surprise by the issue. I'll have to retest, and it's as you suggest, maybe we will just have to increase the database size to accommodate the dict size.

In the original LWT, users would create all words by hand. Maybe the size is not making much sense now that we are ingesting a full dictionary from the start, but let me know for the solution that seems be to you.

[sarahmccuan]

Yes, I installed both the big standard dicts & then started getting this error. I think that's probably why it's hitting the limits like this.

[HugoFara]

Oh I understand now. One dictionary is normally enough, so there is no real reason to get the two for normal use cases. I'll survey in case a very big dictionary (English) could raise the same issue. Even there, there should be a clear message for the user rather than an error.

Happy to hear your thoughts if you have another suggestion.

[HugoFara]

The issue turned out to be a bit different, and easier to solve. When you import a word, it links it to similar terms, so that it is faster during retrieval. With a full dictionary import, it caused a temporary spike in memory (174 MB on 200 k terms, #279). I filtered terms that have 0 chance to be similar before processing. It should work fine now.

[HugoFara]

Fixed in 3.5.0, closing.