Geoff Chappell - Software Analyst
Evidently, I don’t get much done these days. Among the many pointless diversions that have nothing to do with work (except for interfering with it) and the several welcome diversions for paid work, I also got diverted by a report on Twitter about Windows hanging when one process reserves all of another’s virtual address space.
There is indeed a long-standing problem of spectacularly poor performance in this particular case of resource exhaustion. Especially interesting is that what is ordinarily a useful aid, namely the VAD bitmap, becomes the problem. At any given time, some of the user-mode address space has its allocation status optimised into the VAD bitmap for ready access. On searching for clear bits but finding none, the Memory Manager expands the bitmap to account for more of the address space, and then repeats its search. In the problem case, however, all the address space that is newly optimised into the bitmap is reserved. All the new bits are set. Each repeated search still finds no clear bits and the Memory Manager falls into a cycle of expanding and re-searching, ever slower as the bitmap gets ever larger. Eventually, the bitmap is as large as it can possibly be, now “optimising” allocation from the whole of user-mode address space. The expansions stop. There isn’t really a hang, but the realisation takes orders of magnitude more time than any real-world user should ever have patience for.
Or so goes my rough summary. Plainly Microsoft has something to look into there and the computer industry’s security researchers might usefully reopen discussion of when misbehaviour from exhaustion of resources becomes a security issue. But that’s all for the many who get to think through such things as salaried work.
For this website, it’s just a little story of what it was that got me referring to my notes on the Memory Manager and to what I’ve yet published of them. It turns out that I, and apparently everyone else who writes on the Internet about Windows, have largely left the VAD bitmap alone even in my own notes. At first this suprised me, since the VAD bitmap must be well known. Against this is that it’s not fundamental to an understanding of virtual address space in Windows: it’s just an optimisation of the allocation algorithms. Also looking neglected, and again not just by me, is the wide-ranging re-architecture of the Working Set List. The start of this for the 1607 edition of Windows 10 is recorded among my own published pages, but then I left it alone. Obviously there’s much here that could do with some fresh attention!
Sadly, since I am these days less capable of giving anything fresh attention, the need for it applies not just to my pages about memory management. Add that I have since mid-2020 aimed to rearrange my pages on kernel structures around what we know of which headers Microsoft defines them in, and I fear I shall easily be overwhelmed. Still, I’ll try to do what I can. Who’s to know how long this revisit will hold my attention against all those other diversions, but let’s see where it runs…