Where did my freed memory (not) go?

Yapfast's whole deal is not eating your RAM, so I measure it A LOT. One night (at around 4 AM specifically) I was running a ten person test call, and one step of that run hid the window, to see how much memory the window was costing.

The app's heap in use dropped from 78 MiB to 55, great. The memory the kernel counted for the process went from 97.8 MiB to.. 97.3.

malloc vs the kernel

Two things are counting memory here, and they very much disagree. The kernel's number is the process's anonymous memory, every page it has that no file backs (its heap, its threads' stacks and so on). You can read it per mapping in /proc/PID/smaps, and unlike the binary's code, which the kernel can drop and read back from disk, it has nowhere to go but RAM or swap.

The heap number comes from malloc itself, which counts how much it has handed out and not gotten back yet. glibc prints that for each arena if you call malloc_stats(), as the bytes it took from the system and the bytes in use. Yapfast has a test hook that calls it (my measurement scripts write commands into a file, and stats is one of them), so I asked it at every step of the run:

/// Print the allocator's per-arena totals to standard error: memory held from
/// the system against memory in use.
fn allocator_stats() {
    #[cfg(all(target_os = "linux", target_env = "gnu"))]
    {
        unsafe extern "C" {
            fn malloc_stats();
        }
        // SAFETY: malloc_stats only reads allocator state and writes to stderr.
        unsafe { malloc_stats() };
    }
}

Here's what it said, in MiB. "Held" is what the arenas took from the system, "in use" is what Yapfast actually has allocated, and "anonymous" is the kernel's number:

The window	Arenas	Held	In use	Anonymous
Open, in the call	47	91.4	78.2	97.8
Hidden	47	91.4	54.8	97.3
Destroyed	47	91.4	54.2	97.8
Destroyed, after malloc_trim	47	89.6	54.5	73.1
Opened again	47	91.2	77.5	96.7

Going down it, the first row looks normal enough, apart from glibc holding 13 MiB more than Yapfast has in use. Hide the window and "in use" drops by 23 MiB, but "held" doesn't move at all, and neither does the kernel's 97 MiB. Destroy the window completely and it's the same thing. Yapfast freed the memory, glibc got it back, and then glibc just.. kept it, and the kernel still counts every byte of it against us.

Then one malloc_trim takes 25 MiB off the kernel's number while "held" barely changes (what? hold that thought). Oh, forgot to mention, there are 47 arenas.

47 arenas

The answer's in mallopt(3).¹ glibc doesn't want every thread fighting over one lock, so it hands out arenas. The first time a thread allocates, it gets an arena of its own, up to eight per CPU core on 64-bit, and past that threads start sharing. Every arena keeps its own free chunks around for the next time its threads need them.

Yapfast runs dozens of threads (Tokio's workers, the audio threads, the call's connections, whatever else), so it ended up with 47 arenas, each one sitting on its own little pile of free memory. Add the piles up and you get 13 MiB, which is the gap between 91.4 held and 78.2 in use. So THAT's where the 13 MiB went :D

Yep! Such a smart chicky!!! mallopt(M_ARENA_MAX, n) does exactly that, and so does the MALLOC_ARENA_MAX environment variable. Yapfast calls it as the very first thing in main:

fn limit_heap_arenas() {
    #[cfg(all(target_os = "linux", target_env = "gnu"))]
    if std::env::var_os("MALLOC_ARENA_MAX").is_none() {
        const M_ARENA_MAX: std::ffi::c_int = -8;
        unsafe extern "C" {
            fn mallopt(parameter: std::ffi::c_int, value: std::ffi::c_int) -> std::ffi::c_int;
        }
        // SAFETY: this runs first in main, before Yapfast starts any other thread.
        unsafe { mallopt(M_ARENA_MAX, 2) };
    }
}

The -8 is just what glibc's malloc.h defines M_ARENA_MAX as. If you set MALLOC_ARENA_MAX yourself, yours wins, which is handy for trying other values without a rebuild (someone speed up the Rust compiler I'm begging you). And it has to run before Yapfast starts any other thread; that's what the SAFETY comment is about.

Same call again, now with the cap:

	Arenas	Held	In use	Anonymous	CPU, one core
glibc's defaults	47	91.4	78.2	97.8	5.8%
MALLOC_ARENA_MAX=2	2	79.3	78.1	89.6	5.6%
The cap in main	2	79.3	78.2	90.1	5.5%

The gap between held and in use went from 13 MiB to about 1, and the kernel's number dropped by 8 MiB, at the same CPU :D

Why freed memory stays

Even with two arenas, hiding the window still drops "in use" from 78 MiB to 55 while "held" stays at 79, and the kernel's number only goes from 90.1 MiB to 89.5. So, basically nothing, again; the 23 MiB Yapfast freed just sits in the two arenas now.

glibc does give memory back for big allocations, which can get a mapping of their own and go straight back to the kernel when you free them. Everything else lives inside an arena's heap, and glibc only returns the free space at the very top of that heap, once there's more than M_TRIM_THRESHOLD of it (128 KiB by default).² Anything freed below a chunk that's still in use stays mapped and resident, waiting to be reused. One tiny allocation that sticks around, sitting above megabytes of freed stuff, keeps ALL of it in your RAM.

And that's the weird malloc_trim row from the first table. malloc_trim(0) skips that rule, and it seems that since glibc 2.8 (thanks Claude) it walks every arena and tells the kernel it can have back any whole page that's free, even in the middle of a heap.³ glibc keeps the address range, so "held" barely moves, but the pages go back to the kernel, so its number drops. That's the 91.4 to 89.6 and the 97.8 to 73.1, finally making some damn sense.

Trimming it back

Yapfast now trims in three spots where it frees a big pile all at once, plus whenever my measurement scripts send trim, which is how I tell memory that's in use from memory glibc is just sitting on. I go by the allocator's numbers instead of PSS⁴ anyway; PSS jumps around by about 10 MiB between runs of the exact same build.

After READY

When Yapfast connects, Discord's gateway sends READY, a whole ass JSON message with all your servers, channels and DMs. Yapfast copies out what it needs and drops the rest, then trims before the session does anything else:

async fn connected_session(
    gateway: Gateway,
    ready: Value,
    requests: &mut mpsc::UnboundedReceiver<Request>,
    shared: &mut Shared,
) -> Ended {
    // ...
    let mut tracker = Tracker::from_ready(me, &ready, shared.names.clone());
    drop(ready);
    trim_startup_heap();
    // ...
    session.run(requests).await
}

trim_startup_heap itself is just malloc_trim(0), behind the same glibc check. I added it in the same commit where I killed two stupid copies: the gateway cloned every message it received, and the chat copied every loaded picture into its own pixel buffer, even for rows nowhere near the screen. With all three fixes, on a fake account⁵ with 100 servers and 300 DMs, a server view went from 220–223 MiB of RSS⁶ to 170–176 MiB, and the heap in use from 113 MiB to 75. I didn't measure the trim on its own (lazy), but going by the window and the call, without it a lot of the freed heap would just stay resident (very evil).

After a call

On the fake Discord server, in a call with the microphone on, the kernel's number went from 27 MiB to 47 MiB and stayed at 47 after I hung up, even though the heap in use had dropped back below where it started. Amazing.

Yep, same story as the window, so the voice worker now trims too, once the call is over:

while let Some(command) = commands.recv().await {
    match command {
        Command::Join(id) => {
            // ...
            let result = join(&calls, id, &mut state, /* ... */).await;
            if let Err(error) = result {
                state.error = Some(error.to_string());
            }
            state.disconnected();
            trim_after_call();
        }
        // ...
    }
}

join runs the whole call, from joining until you leave, so the trim comes right after you hang up. Well, three seconds after:

pub(crate) fn trim_after_call() {
    #[cfg(all(target_os = "linux", target_env = "gnu"))]
    tokio::task::spawn_local(async {
        tokio::time::sleep(std::time::Duration::from_secs(3)).await;
        unsafe extern "C" {
            fn malloc_trim(pad: usize) -> std::ffi::c_int;
        }
        // SAFETY: malloc_trim only returns unused allocator pages to the OS.
        unsafe { malloc_trim(0) };
    });
}

Why exactly three seconds you might ask? malloc_trim can only give back memory that's already been freed, and a call doesn't free everything the moment you hang up. Its audio threads and connections shut down on their own a bit after. Trim right away and most of the call is still allocated. A call on real Discord had freed all of it within two seconds, so three leaves a little room. Six seconds after leaving, the kernel's number was down to 31 MiB after the first call, and 26 after a second one.

Closing the window

The ten person call from the start was me checking what a tray could save, and hiding the window saved nothing, which was confusing and annoying af. Hiding a Slint window only makes it invisible, so the UI, the data it shows and the renderer all stay in memory.

Now that there's a tray, closing the window drops it completely, renderer and all, and half a second later Yapfast trims again:

pub(super) fn release_memory() {
    trim();
    #[cfg(windows)]
    {
        // ...
        // SAFETY: both act on this process's own heap and working set; -1 for
        // both sizes asks Windows to take out every page it can.
        unsafe {
            HeapCompact(GetProcessHeap(), 0);
            SetProcessWorkingSetSize(GetCurrentProcess(), usize::MAX, usize::MAX);
        }
    }
}

On the GPU renderer, the first close to the tray took the kernel's number from 86.6 MiB to 62.9, 24 MiB actually back. That was on the fake Discord with five servers, not the ten person call, so it's not exactly the same 23 MiB. On Skia Software it's only 2.3 MiB, because there was way less to give back in the first place.

What about Windows?

Most of this doesn't run there. The arena cap and the trims are behind #[cfg(all(target_os = "linux", target_env = "gnu"))], because mallopt and malloc_trim only exist in glibc. Windows has its own heap, and a musl build would use musl's allocator. Closing to the tray is the one exception: on Windows it compacts the heap and empties the working set instead, which drops the window's pages out of Task Manager's count.⁷ I've only measured the allocator on Linux so far, so whether Windows hoards freed memory the same way, who the fuck knows (not me, yet). My friends use it on Windows, though, and it seems to be doing just fine even after hours of scrolling and calls. One of them actually managed to get vanilla Discord to 3.6 GB (?????) while trying to compare them, truly an Electron moment.

The bourgeois GPU

As for the 97 MiB @Pipi asked about, here's the ten person call on Skia's software renderer instead of the GPU one, glibc on its defaults:

Ten person call, window open	Heap in use	Anonymous	PSS
GPU	78.2	97.8	246.8
Skia Software	18.1	32.1	131.1

The GPU renderer has 60 MiB more heap, all of it the graphics driver and Skia's GPU state (how it splits between those two, I don't know yet, sorry :( ). Skia Software is the default renderer now, and the GPU one is a toggle away, so you can pick your poison.

If you know glibc's malloc better than me and I got something wrong here, PLEASE tell me, on the Discord server or in an issue.

Thanks for reading, stay lean (RAM wise), and stay blazingly fast <3

Footnotes

1. Under M_ARENA_MAX in mallopt(3). On a 32-bit system it's two per core. ↩

2. At first, anyway. Every time you free a big allocation that had its own mapping, glibc raises the size it gives mappings to, up to that allocation's size, and the trim threshold follows at twice that. So the longer an app runs, the more freed memory glibc can sit on before it trims. Setting any of these options yourself turns that off; it's under M_MMAP_THRESHOLD in mallopt(3). ↩

3. malloc_trim(3). Before 2.8 it only trimmed the top of the main arena. ↩

4. Proportional set size: what the process has in RAM, with every page it shares with other processes split between them. ↩

5. On the fake Discord server I test against, testing/fake_discord.ts. It can make up accounts like this one, so no real Discord account gets touched. ↩

6. Resident set size: everything the process has in RAM, counting pages it shares with other processes in full. ↩

7. Task Manager's Memory column is the private working set. Emptying it moves the window's pages to the standby list, and they come back if anything touches them again. ↩