posted in Selfhosted

Why is my home server using so much RAM for cache + buffer?

I have noticed that my home server is strangely using lots of swap (~5 GB), despite having only a few lightweight processes running and loads of RAM installed (32 GB).

Upon configuring Grafana + Prometheus, I noticed a trend where cache + buffer will progressively increase until swap starts to be used. My system and services combined will use ~8 GB RAM. Upon rebooting, the cache + buffer will start anywhere from 3–10 GB, progressively ramp up to ~25 GB in 1–2h, where swap will start to be needed (~3 GB). See the image attached for reference.

My swap filesystem is on an expensive (to me) SSD, and I would like to reduce its wear by as much as possible. I understand that swap can introduce only minimal wear on SSDs depending on its nature and that it can be harmless, but I am still not sure what is causing this behavior (and why) and whether I should worry about it or not. So I figured I should investigate what is happening here.

My main question is, how can I figure out what is causing this behavior? Is it expected? I am looking for guidance from others who are more experienced than me in the topic.

A little bit about my system:

I am running Debian 12 on an NVMe SSD containing the root partition (btrfs) and docker services. I also have two HDDs, one with persistent data (ext4), and the other with backups (ext4). This is majoritarily a single-user machine. I tried using the following kernel parameters, but it hasn’t helped:

vm.swappiness=10
vm.vfs_cache_pressure=200

My docker services are:

  • *arr stack
  • jellyfin
  • nextcloud
  • immich
  • open-webui + ollama
  • pi-hole
  • invidious
  • romm
  • nginx proxy manager
  • grafana + prometheus
  • other minor services that I don’t think are doing much (uptime-kuma, stirlingpdf, vaultwarden, etc)
enPage

Replying to an earlier post

Yeah that’s completely normal, looks absolutely fine to me. For more info on how linux uses memory I suggest reading https://www.linuxatemyram.com/

To add: that link suggests looking if your swap usage is changing. Linux is quite happy to stuff some memory that isn’t actually used in swap. If it was right about it not being used you should see the swap usage not changing much, so your SSD isn’t really being used either apart from the initial swap-out. If your swap usage keeps fluctuating wildly there is more going on which might warrant further investigation.

www.linuxatemyram.comHelp! Linux ate my RAM!

Replying to an earlier post

The amount of swap used is not a good indicative, you need to check if there is a big exchange of data per second/minute. This is the only indicative of an out of memory system.

Sometimes, some regions of data memory “age” in ram without any access for a long periods of time, the the kernel here has two options, it could destroy the region knowing it could recreate it when needed (with some cpu overhead) or moved this to a swap file when the ram structure already in the swap file and release than section.

Which regions are good candidates for this? Buffers, specially in the fs, code region used for processes or even data sections of a long sleeping process…

Checking your data, if those 5gb are created over a long period of time I would not care a lot about it. Remeber how big the swap is, isn’t that important vs real traffic (in or out) to it

Replying to an earlier post

Huh…


  System load:  1.26                Temperature:           57.0 C
  Usage of /:   20.4% of 454.86GB   Processes:             527
  Memory usage: 44%                 Users logged in:       1
  Swap usage:   0%                  IPv4 address for eno1: 192.168.1.190

Ubuntu 22.04.5 - 32 GB RAM - 53 total containers

cat /proc/sys/vm/swappiness: 60 (pretty much default)

cat /proc/pressure/cpu: some avg10=0.68 avg60=0.62 avg300=0.67 total=960097774
full avg10=0.00 avg60=0.00 avg300=0.00 total=0
cat /proc/pressure/memory: some avg10=0.00 avg60=0.00 avg300=0.00 total=0
full avg10=0.00 avg60=0.00 avg300=0.00 total=0
cat /proc/pressure/io: some avg10=1.16 avg60=4.88 avg300=3.85 total=2831425045
full avg10=1.11 avg60=4.54 avg300=3.58 total=2653626377

Replying to an earlier post

Well, here I have 41 containers and:

free -hm
               total        used        free      shared  buff/cache   available
Mem:            31Gi       5.7Gi       426Mi       207Mi        25Gi        25Gi
Swap:           31Gi       3.7Gi        28Gi

As well as

cat /proc/sys/vm/swappiness: 10

and

$ cat /proc/pressure/cpu
some avg10=0.00 avg60=0.00 avg300=0.00 total=302436406
full avg10=0.00 avg60=0.00 avg300=0.00 total=0

$ cat /proc/pressure/memory
some avg10=0.00 avg60=0.00 avg300=0.00 total=64263542
full avg10=0.00 avg60=0.00 avg300=0.00 total=63043120

$ cat /proc/pressure/io
some avg10=1.99 avg60=1.67 avg300=1.61 total=1844790000
full avg10=1.90 avg60=1.59 avg300=1.56 total=1788451770

Replying to an earlier post

41 containers:

Are you noticing any stalling or stuttering when running apps or using multiple apps? If ‘no’ then I’d agree with the ones that are telling you it’s fine. Dats how Linux do. I’ve always been told to let Linux handle that unless it’s causing problems for you. And even then, after exhausting all other things that may cause this. Now whether that is sage advice, I couldn’t comment to that, however and anecdotally, it’s served me well.

Replying to @⁨A9nWGzYt@lemmy.dbzer0.com⁩

I disagree with most here.

You need to reduce swappiness even more, and tweak some other variables (like the memory “threshold” Linux starts to swap, and disk cache aggressiveness) to stop Linux from swapping so opportunistically under such a light load.

IMO, Linux is configured for “old” systems by default: slow HDDs, and constrained RAM pools, where disk IO really, really needs caching, and where idle background processes take a large fraction of RAM.

You have superfluous RAM for your workload. And very fast disk IO that isn’t such a hindrance to apps anyway. And a disk you don’t want to wear. This is the opposite scenario: you don’t want Linux to swap unless it absolutely has to.

For reference, this is part of my config. It’s rather niche and you probably shouldn’t use it, but you should consider looking up the variables:

# Keep min reserve reasonable for 8GB usable space
vm.min_free_kbytes = 262144        # Lower absolute minimum to 256MB

vm.watermark_scale_factor = 10     # Lower to 0.1% (which is ~128MB on 128GB)

# Disable watermark boosting completely
vm.watermark_boost_factor = 0

# Normal-ish metadata pressure so desktop doesn't stutter on disk reads
vm.vfs_cache_pressure = 120

# Allow reasonable swapping of inactive anonymous desktop pages, could be lower
vm.swappiness = 10

# Dirty bytes limits to limit caching
vm.dirty_background_bytes = 67108864
vm.dirty_bytes = 268435456

# Disable compaction & proactive scans to stop freezing with large portions of RAM mlocked
vm.compaction_proactiveness = 0
vm.compact_unevictable_allowed = 0

vm.page-cluster = 0 # 4kb pages for SSD

My system still uses RAM as disk cache with this config, it just won’t go out of its way to swap just to keep that cache, especially I lower swappiness to 1-3.

I also have a 1GB zram pool, prioritized over ssd swap. But you should make yours even larger (maybe 4GB?). This will intercept anything that does swap first.

I also start some applications with systemd-run and specify memory caps and swap limits (often forbidding them from swapping entirely).

It makes a night-and-day difference for some workloads on my system, that would otherwise swap pointlessly, just tank performance and even de-stabilize the system.

Replying to an earlier post

I agree that hitting swap is not ideal, but messing with swappiness sysctl is almost never the answer and very often leads to more problems than it solves.

If you want to tune for workload without needing fairly deep understanding of how Linux manages memory, use a sysctl that sets a whole system behaviour like CPU governor presets.

IMO, Linux is configured for “old” systems by default: slow HDDs, and constrained RAM pools, where disk IO really, really needs caching, and where idle background processes take a large fraction of RAM.

Not really a matter of opinion, and not true since kernel 5.1 when ssds became first class citizens.

Replying to @⁨non_burglar@lemmy.world⁩

I mean… it’s an issue. OP’s system is swapping the SSD when it clearly should not be doing it. My system had horrible usability issues in some scenarios without a lot of reconfiguration, and thrashed my SSD for no benefit.

The default heuristic is not good for our systems, and OP’s situation is a pretty common scenario. That’s a fact. How is that not an issue?

And support for TRIM and whatever else was added for SSD support is not really the issue, here. That’s working fine.

Replying to @⁨brucethemoose@lemmy.world⁩

when it clearly should not be doing it.

No, sorry. We don’t know enough to say this. What were the options when configuring ext4? What record size is grafana using? Mariadb or postgres? Is there contention with another db? Is there a mounted NFS path in the same ext4 partition with weird block settings? There are lots of legitimate config reasons for op’s problem, but there are so many parts of op’s setup we don’t know, we similarly can’t possibly know what the issue is.

Swappiness works as intended in most default scheduler configurations and seeing high swap is a symptom, not an issue. It is almost always more productive to find out why the swapping is happening.

Not sure what you’re getting at with TRIM.

Replying to an earlier post

Seeing high cache and buffer numbers is usually a good sign, as it means the system is making full use of the memory you have. It’ll automatically deallocate it if something actually needs the memory, but until then it’s using it to store data you might need.

Swap’s a funny one, and whilst you can tune it to an astonishing degree, the decisions the kernel makes aren’t always what you’d want. With this little actual memory usage, compared to the installed RAM, you might be able to run without swap at all, unless you want something like hibernate. If you want to test that, you can run (as root) swapoff -a, which should disable all swap devices, pushing needed pages back in to RAM, but only until you reboot. If it’s stable, you can consider removing the swap partition(s) later.

Replying to an earlier post

I agree with you that my server specs are a little bit excessive for my usual workload. I have disabled swap in the past with swapoff -a and removed the partition from /etc/fstab under a similar usage, and no issues were observed.

I am just leaving it there now for the rare instance when I run the occasional lightweight local LLM model (on CPU) for light tasks such as spell checking, which might need some RAM and I don’t risk crashing due to OOM issues.

However, I still want to understand what is happening here to learn a little bit about my system. My intuition can be wrong here, but I find it strange that my laptop with similar specs and the same OS (but reasonably distinct workloads, to be fair) almost never swaps nor reaches those ridiculous levels of buffer + cache usage, so I think I might find one “offender” container to blame.

Replying to an earlier post

Very basically, what’s happening is that every file that is read (including the binaries and libraries that make up the services you run, etc) is loaded into memory and stays there until that memory needs to be reclaimed for something else. It’s a good thing. It means that the next time that file is needed, it can be accessed directly in RAM rather than reaching out to the filesystem.

Replying to an earlier post

The cache usage being higher on a server than a laptop seems fairly likely as more disk accesses are occurring. When data is loaded from disk, the linux kernel will keep it in memory even after the original requester is done with it, on the principal that it might need it again. This is the cache you’re seeing. If the kernel detects high memory utilisation it will free cache pages before anything else, so you actually want to see near 100% memory utilisation all the time, as it means fewer accesses to disk.

The swap usage is harder to diagnose without more detailed diagnostic work, but you can see from the graph you posted that, even what swap is being used, you have a little free memory. This may well be the kernel preemptively moving little used memory pages to the swap cache so tgat they can be evicted from memory quickly if needed. You could investigate thus by adjusting the ‘swapiness’ value closer to 0, to see if that delays the swap usage and reduces the peak.

Replying to @⁨A9nWGzYt@lemmy.dbzer0.com⁩

No point in messing with swappiness settings, trust me. You want to address the reason this happens in the first place, not disable or reduce swap.

I’ve had a nearly identical scenario at work. The reason? A data verification process that went through 100s of thousands of files to run their checksums and check integrity. This was working as intended, but the system alerts were firing because of that high buffer/cache utilization. So check which processes have highest IO and you might find the answer. Swap is just a side effect.

Replying to an earlier post

I didn’t look too closely at details, but generally that’s relatively normal the more individual applications you have running, especially with the overhead of them all being virtualized. Often the overhead of running that many virtualized operating systems can be problematic depending on how the container author chose and configured the container’s OS. A lot of people seem to just randomly pick an OS for their containers just so they can distribute their application with docker as easily as possible, without considering the consequences.

Here are the options off the top of my head if disk wear is your primary concern:

  1. Turn swap off at the host level. Note, though, that you may end up in bottlenecks if multiple of them do very memory intensive tasks at the same time, like image and video processing, and docker can’t swap out the VMs that are idle to the disk and be able to use the RAM fully for those active tasks. For some of those apps you can configure when they do maintenance tasks and reduce the likelihood.

  2. Look at each individual VM in docker and see which are using the most swap and force docker to limit their allowed resources. You can even tell docker to make an application not use swap by setting the memory and memory-swap values to the same value or reduce their swap usage by setting memory-swapiness to a low number, or you can install those applications to the host system and configure the host to reduce swap or disallow swap usage through systemd or whatever rather than running them in docker. You may also find one of the applications you’re running has a memory leak issue. Good OSs often handle that, but who knows what IS the container is running and the docker layer is concealing from the host kernel and not handling quite as well as it being installed on “bare metal” might. Virtualization adds a lot of variables for these kinds of low level resource control scenarios. So just trying it on the host directly might reveal something if you’re interested in digging like I usually am, lol.

  3. Get a second, small, inexpensive drive and set that up as the swap partition for the host.

  4. Look into what database systems these are using. Often if they each are running their own separate DBMS containers, the DBMS maintenance overhead can cause some additional churn. I have a single instance of PostgreSQL and one of MariaDB running on a separate server that all of my applications connect to. Some of these applications may be not only spinning up a whole VM for the application, but an entirely other VM for their database server. Though I think some of these probably use sqlite which is just a file, I think manynof these use PostgreSQL and/or MySQL/MariaDB by default. This also helps because I only need a single backup for all of my databases. The negative is that if that DB server goes down, then all the apps go down too, but if you already have everything on one server that might not be as big of a deal. I have a lot of smaller servers running smaller subsets of applications based on their resource needs. Helps me conserve electricity rather than running one or two big servers that are almost always highly active. But I enjoy optimizing that kind of stuff, it’s tedious.

Replying to an earlier post

cache is good, but you shouldn’t have that much swap usage with that overall memory utilization. 8t seems the kernel is making a decision to move a significant amount of memory to swap to have just a little bit more cache. that could make sense in some situations, especially with lots of RAM and a slow disk that is frequently accessed… but probably not on your system. I see you have already reduced swappiness to 10, so maybe all that was snapped out is really rarely used memory, but I wouldn’t want even that on an SSD so you could try setting swappiness to 0. that way it should only use the swap when there is no more cache to free, I think, but the behaviour of this sysctl was not always clear like that.

docs.kernel.org/admin-guide/sysctl/vm.html?highli…

Replying to @⁨A9nWGzYt@lemmy.dbzer0.com⁩

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

Fewer Letters More Letters NFS Network File System, a Unix-based file-sharing protocol known for performance and efficiency SSD Solid State Drive mass storage VPS Virtual Private Server (opposed to shared hosting)

[Thread #79 for this comm, first seen 17th Aug 2026, 20:30] [FAQ] [Full list] [Contact] [Source code]

Hachyderm.ioImran Nazar ~ عمران نزر (@Two9A@hachyderm.io)2.93K Posts, 306 Following, 268 Followers · Front-end #typescript developer and general keyboard tapper; author of Internet RFC 7168 (Hypertext Coffeepot Control Protocol for teapots); occasional plumber, more than occasional #c64 #retrocomputing enthusiast, terrible at classical #piano. Header photo is a verdant scene looking over the landscape near Buxton, England, taken from a single-track country road.

Replying to @⁨A9nWGzYt@lemmy.dbzer0.com⁩

https://www.linuxatemyram.com/

This is normal. Linux caches disk reads so that you don’t have to go to her/SSD to reread blocks. If your app isn’t using the ram Linux will use it instead.

Zero reason to be concerned, because if your app allocates more memory Linux will evict the cached pages to make sure your apps have plenty of space.

In this case your apps have memory that isn’t being touched (sometimes a sign of a memory leak), and Linux has decided that caching your disk access is more advantageous than keeping the unused application memory in ram.

If you have apps that you don’t use often and are really slow to access when you finally do, keep lowering that swappiness value or just remove swap all together if it annoys you.

Looking at your graph you have a lot of ram that you don’t really need… I would just disable swap entirely. Otherwise you are just wasting write cycles on your SSD.

www.linuxatemyram.comHelp! Linux ate my RAM!