Feature request: detach chunks from unloaded world instances to reduce memory retained by stale references

[KamikotoTkach]

Problem

When using AdvancedSlimePaper with many dynamic worlds, unloaded world instances can still retain a large amount of memory if some plugin keeps a strong reference to the old world object.

In practice, many Minecraft plugins are not designed well for dynamically created and unloaded worlds. They may accidentally keep references to World, Chunk, Location, scheduled tasks, listeners, cached objects, or other objects connected to a world. Because of this, even after the world is unloaded, the old world instance may still be reachable from GC roots through a third-party plugin.

The issue is that the retained world instance appears to keep strong references to its loaded chunks and related chunk data. As a result, a single stale world reference can retain much more memory than just the world object itself.

This becomes a serious problem when working with many temporary or instanced worlds. In my case, memory pressure already starts becoming noticeable at around 100 retained unloaded worlds. Ideally, accidentally retained unloaded world objects should be cheap enough that even a much larger number of stale world references would not immediately fill the heap.

Proposed solution

After a Slime world is unloaded, the world instance should release/detach as much heavy runtime state as possible, especially strong references to chunks and chunk-related data.

In other words, once the world is no longer loaded, the old world object should become more like a lightweight "dead shell" / placeholder:

• no strong references to loaded chunks;
• no strong references to chunk sections, entities, block entities, ticking data, or other heavy per-world runtime state where possible;
• only minimal metadata required for diagnostics or safety checks should remain;
• any attempt to use the unloaded world instance should either fail safely or behave consistently with Paper’s existing unloaded-world behavior.

Optional diagnostic command

It would also be very useful to have a diagnostic command for finding unloaded world instances that are still being retained in memory.

For example, a command could list unloaded Slime worlds that are no longer registered as active worlds, but whose old world instances are still reachable.

This would make it much easier to identify problematic third-party plugins that keep references to unloaded worlds or other stale world-related state.

[davidmayr]

This would inherently break one of the rules I always keep in mind when doing ASP work:

• Affect worlds as little as possible, act exactly like a vanilla world would

We could theoretically hook world unload into the SlimeInMemoryWorld and mess up its state, BUT:

ASP is not responsible for fixing user error, besides Location objects use a WeakReference and are not affected by this issue. Users need to avoid keeping Entity, World and Chunk References. Location is fine, its just going to lose its world and get it set to null.

Whilst It's possible to overlook references from time to time, the sheer extensiveness of this issue shows a huge part of plugin devs have a major issue with understanding garbage collection and memory in their systems. I believe it's the wrong approach to hold developers hands for such a fundamental thing in the Java language. Instead, customers/users of plugins should report these issues to the respective developers so they learn from their mistakes and become better developers. We would just be fixing symptoms, not their cause.

I was considering researching about making a separate runtime analysis plugin for common Minecraft memory leaks that would help users solve this problem and other issues (like holding player references, which can also cause SWM memory leaks) without having to consult Eclipse Memory Analyzer or similar.

I'm going to get a second opinion on my statement here, but for me this really is a non-issue on our side and should be fixed on the API consumer end. Although reducing all those "memory leak" reports that just turn out to not be an issue on our end (essentially every time) would be a win.

[KamikotoTkach]

Thanks for the detailed explanation. I understand the reasoning behind keeping ASP behavior as close to vanilla as possible, and I agree that plugins holding strong references to World, Chunk, Entity, etc. after unload are ultimately doing something wrong.

The part I am struggling with is the practical side of this. Reporting these issues upstream is the right long-term direction, but in production it often does not give server owners a usable path forward.

For example, I already opened an issue for MythicMobs about stale world retention roughly two weeks ago, and there has been no response so far. I also seem to have a similar issue with ItemsAdder, although I still need to diagnose it better before filing a proper report. From the current heap dumps it looks like custom ticking blocks may be involved. I also saw WorldGuard in the retention graph; that one looks more like some temporary debounce cache judging by the name, so I am less sure about it, but it still shows how many different places this type of issue can come from.

To keep the production server stable, I ended up writing a separate workaround plugin for the MythicMobs case. It uses reflection to clear world references held by MythicMobs and cancels tasks related to unloaded worlds. That is obviously not a good solution. It is fragile, version-dependent, and can break on any MythicMobs update, but without doing something like this the stale world references keep accumulating heavy world state in memory.

That is why I do not think “report it to the plugin developer” fully solves the problem in practice. Many plugins are paid or closed-source, maintainers may take a long time to respond, and proving the issue usually requires heap dumps and MAT knowledge. Even if one plugin gets fixed, another plugin can have the same class of bug later.

ASP/SWM makes this problem much more visible because dynamic worlds are one of the main reasons people use it. On a regular server with a small fixed set of worlds, a bad reference might stay unnoticed for a long time. With short-lived instanced worlds, the same mistake gets multiplied by every loaded/unloaded world and can become a production blocker very quickly.

I also think there is a difference between loaded-world behavior and already-unloaded-world behavior. While a world is loaded, I fully agree that ASP should affect it as little as possible and behave like a vanilla world. After the world is unloaded, though, continuing to use the old World object is already invalid plugin behavior. In that state, it would be very helpful if the old object either became cheaper to retain or at least was easier to diagnose.

The runtime analysis plugin idea you mentioned sounds very useful, and I would definitely use it. It would probably reduce many false memory leak reports against ASP, because users would have a supported way to distinguish ASP-side issues from third-party plugins holding stale world/player/entity references.

At the same time, diagnostics alone would not have fixed the MythicMobs situation for me. Even if such a tool had existed and clearly told me “MythicMobs is retaining this unloaded world”, I would still have been stuck waiting for an upstream fix or writing the same reflection-based workaround. I can do that because I am a developer and can read heap dumps, inspect internals, and patch around it locally. Most server owners cannot. For them, even a perfect diagnostic result may only confirm that their production server has a problem they cannot actually fix.

That is why I still think some form of mitigation would be valuable, even if it is not the default behavior. The root cause is on the plugin side, but the operational impact lands on people running dynamic-world servers. In real setups, this class of bug is common enough that server owners need more than just “find the bad plugin and report it”.

So yes, I understand why this may not belong as a default behavior change in ASP. I just think this is common enough in real dynamic-world setups that having either official diagnostics or some safe/optional mitigation would make a big difference.

[davidmayr]

Maybe I'm biased here because I write all my plugins myself... (Even wrote a custom model engine. Zero paid plugins on my server except for Vulcan AntiCheat (although we currently don't even have that on our dev network and prod is currently in maitnenance sooo), which I'd be open to remove or replace with alternatives at any time and players likely wouldn't notice.).

I also think there is a difference between loaded-world behavior and already-unloaded-world behavior. While a world is loaded, I fully agree that ASP should affect it as little as possible and behave like a vanilla world. After the world is unloaded, though, continuing to use the old World object is already invalid plugin behavior. In that state, it would be very helpful if the old object either became cheaper to retain or at least was easier to diagnose.

Generally anything we do to the server creates a maintenance burden for us for something that's not our fault. Clearing SlimeInMemoryWorld#chunkStorage is an option, which would at least release a lot of the NBT stuff and should be semi low-effort and only a minor maintenance burden finding the correct hook to mess up the data.

But then again, the problem is not solved. Every state minecraft itself tracks would remain. And I definitely would not touch any state we do not maintain (so essentially anything outside SlimeInMemoryWorld is a taboo to touch from my side). So instead of you noticing the problem early, you are going to notice it only once your server actually has players that continuously fill up the memory. It's eventually going to happen, but It's going to be much harder to reproduce the less frequently it occurs. (Before you mention automatic restarts, that point can be brought up for this issue too IMO)

I'm still waiting for some more opinions on this, but I think my personal stance on it remains. We're not really responsible for your choice in plugin author and that's also one of the reasons I don't rely on any closed source plugins but my own. I get why that's frustrating, especially with plugins you have paid for. But It's also not really our job to care about third-parties mistakes. That being said, I'd not complain with 2 weeks wait time yet, that's still very low for most bug reports 🤷

[davidmayr]

That being said, I'm probably actually starting to research on better ways to detect leaks for Minecraft plugins (that are easy to understand for the average nobody) the next few days if I find the time to do so.

Obviously it's almost impossible to reliably auto-detect any leak, but at least for many Minecraft objects, we do know if those objects should exist or not.

[kyngs]

I think I agree with David here.

The thing is, as long as the plugin holds a strong reference to the World, the world stays in memory no matter what we do to it. Clearing chunkStorage just makes the leaked object smaller, but the leak itself is still there. The world is still reachable through the plugin, and I think all of MC's own per-level state hangs off it? So we'd be turning a large memory leak into a smaller one, not actually removing it.

In my opinion, the slower leak could be wrose. Right now, this fails quickly: if a plugin is leaking world objects, under real load you're going to find out pretty fast, capture a heap dump, and trace the offender. If we strip the world down to a shell, the same bug only shows up later under real player load, maybe after several days or weeks, maybe never if the server restarts periodically. And once that happens, you're back where we started.

This problem is also basically as old as SWM itself. Dynamic worlds have always made plugin reference leaks more visible because you're loading and unloading constantly, so any mistake gets multiplied. It's not new, and it's never really been our bug to fix. Going down the road of fixing plugin bugs sets a pretty dangerous precedent imo, because at that point we're officially working around other people's bugs. Today it's chunkStorage, next version someone wants us to clear something else because their plugin leaks differently, and there's no end to it.

I do get op's frustration though. Reporting it upstream doesn't help much when it's a paid, closed, or abandoned plugin and the issue has already been sitting for two weeks.

However, I'm wondering how we could make this more transparent to users. We can definitely detect that a leak is happening using something like PhantomReference, but realistically, without a heap dump, we can't really identify the offending plugin. Maybe this could be combined with maintaining a documented list of problematic plugins and comparing it against the plugins loaded on the server? But that would impose a significant maintaining burden because we would have to keep track of problematic plugins.

[kyngs]

Also, thinking about this more, this is something that could be handled by an external plugin.

Nothing really stops someone from creating a plugin that listens for Bukkit world unload events, checks whether the world is a slime world, gets the ASP handle, and clears the chunk storage.

And since this is doable without any involvement on our side, it makes even less sense for us to take on the burden of implementing this natively in ASP.