1.. _highmem: 2 3==================== 4High Memory Handling 5==================== 6 7By: Peter Zijlstra <a.p.zijlstra@chello.nl> 8 9.. contents:: :local: 10 11What Is High Memory? 12==================== 13 14High memory (highmem) is used when the size of physical memory approaches or 15exceeds the maximum size of virtual memory. At that point it becomes 16impossible for the kernel to keep all of the available physical memory mapped 17at all times. This means the kernel needs to start using temporary mappings of 18the pieces of physical memory that it wants to access. 19 20The part of (physical) memory not covered by a permanent mapping is what we 21refer to as 'highmem'. There are various architecture dependent constraints on 22where exactly that border lies. 23 24In the i386 arch, for example, we choose to map the kernel into every process's 25VM space so that we don't have to pay the full TLB invalidation costs for 26kernel entry/exit. This means the available virtual memory space (4GiB on 27i386) has to be divided between user and kernel space. 28 29The traditional split for architectures using this approach is 3:1, 3GiB for 30userspace and the top 1GiB for kernel space:: 31 32 +--------+ 0xffffffff 33 | Kernel | 34 +--------+ 0xc0000000 35 | | 36 | User | 37 | | 38 +--------+ 0x00000000 39 40This means that the kernel can at most map 1GiB of physical memory at any one 41time, but because we need virtual address space for other things - including 42temporary maps to access the rest of the physical memory - the actual direct 43map will typically be less (usually around ~896MiB). 44 45Other architectures that have mm context tagged TLBs can have separate kernel 46and user maps. Some hardware (like some ARMs), however, have limited virtual 47space when they use mm context tags. 48 49 50Temporary Virtual Mappings 51========================== 52 53The kernel contains several ways of creating temporary mappings: 54 55* vmap(). This can be used to make a long duration mapping of multiple 56 physical pages into a contiguous virtual space. It needs global 57 synchronization to unmap. 58 59* kmap(). This permits a short duration mapping of a single page. It needs 60 global synchronization, but is amortized somewhat. It is also prone to 61 deadlocks when using in a nested fashion, and so it is not recommended for 62 new code. 63 64* kmap_atomic(). This permits a very short duration mapping of a single 65 page. Since the mapping is restricted to the CPU that issued it, it 66 performs well, but the issuing task is therefore required to stay on that 67 CPU until it has finished, lest some other task displace its mappings. 68 69 kmap_atomic() may also be used by interrupt contexts, since it is does not 70 sleep and the caller may not sleep until after kunmap_atomic() is called. 71 72 It may be assumed that k[un]map_atomic() won't fail. 73 74 75Using kmap_atomic 76================= 77 78When and where to use kmap_atomic() is straightforward. It is used when code 79wants to access the contents of a page that might be allocated from high memory 80(see __GFP_HIGHMEM), for example a page in the pagecache. The API has two 81functions, and they can be used in a manner similar to the following:: 82 83 /* Find the page of interest. */ 84 struct page *page = find_get_page(mapping, offset); 85 86 /* Gain access to the contents of that page. */ 87 void *vaddr = kmap_atomic(page); 88 89 /* Do something to the contents of that page. */ 90 memset(vaddr, 0, PAGE_SIZE); 91 92 /* Unmap that page. */ 93 kunmap_atomic(vaddr); 94 95Note that the kunmap_atomic() call takes the result of the kmap_atomic() call 96not the argument. 97 98If you need to map two pages because you want to copy from one page to 99another you need to keep the kmap_atomic calls strictly nested, like:: 100 101 vaddr1 = kmap_atomic(page1); 102 vaddr2 = kmap_atomic(page2); 103 104 memcpy(vaddr1, vaddr2, PAGE_SIZE); 105 106 kunmap_atomic(vaddr2); 107 kunmap_atomic(vaddr1); 108 109 110Cost of Temporary Mappings 111========================== 112 113The cost of creating temporary mappings can be quite high. The arch has to 114manipulate the kernel's page tables, the data TLB and/or the MMU's registers. 115 116If CONFIG_HIGHMEM is not set, then the kernel will try and create a mapping 117simply with a bit of arithmetic that will convert the page struct address into 118a pointer to the page contents rather than juggling mappings about. In such a 119case, the unmap operation may be a null operation. 120 121If CONFIG_MMU is not set, then there can be no temporary mappings and no 122highmem. In such a case, the arithmetic approach will also be used. 123 124 125i386 PAE 126======== 127 128The i386 arch, under some circumstances, will permit you to stick up to 64GiB 129of RAM into your 32-bit machine. This has a number of consequences: 130 131* Linux needs a page-frame structure for each page in the system and the 132 pageframes need to live in the permanent mapping, which means: 133 134* you can have 896M/sizeof(struct page) page-frames at most; with struct 135 page being 32-bytes that would end up being something in the order of 112G 136 worth of pages; the kernel, however, needs to store more than just 137 page-frames in that memory... 138 139* PAE makes your page tables larger - which slows the system down as more 140 data has to be accessed to traverse in TLB fills and the like. One 141 advantage is that PAE has more PTE bits and can provide advanced features 142 like NX and PAT. 143 144The general recommendation is that you don't use more than 8GiB on a 32-bit 145machine - although more might work for you and your workload, you're pretty 146much on your own - don't expect kernel developers to really care much if things 147come apart. 148