Fixing Memory ProblemsThis chapter is about finding bugs in C/C+ programs with the help of a memorydebugger. A memory debugger is a runtime tool designed to trace and detect bugsin C/C+ memory management and access. It does not replace a general debugger.In the following sections, we will describe the memory access bugs that typicallyoccur in C/C+ programs, introduce memory debuggers, and show with two exampleshow these tools find bugs. We will then show how to run memory and sourcecode debuggers together, how to deal with unwanted error messages by writing asuppression file, and what restrictions need to be considered.4.1 Memory Management in C/C+ Powerful but DangerousThe C/C+ language is able to manage memory resources, and can access memorydirectly through pointers. Efficient memory handling and “programming close to thehardware” are reasons why C/C+ replaced assembly language in the implementationof large software projects such as operating systems, where performance andlow overhead play a major role. The allocation of dynamic memory (also known asheap memory) in C/C+ is under the control of the programmer. New memory isallocated with functions such as malloc() and various forms of the operator new.Unused memory is returned with free() or delete.The memory handling in C/C+ gives a large degree of freedom, control, andperformance, but comes at a high price: the memory access is a frequent source ofbugs. The most frequent sources of memory access bugs are memory leaks, incorrectuse of memory management, buffer overruns, and reading uninitialized memory.3334 4 Fixing Memory Problems4.1.1 Memory LeaksMemory leaks are data structures that are allocated at runtime, but not deallocatedonce they are no longer needed in the program. If the leaks are frequent or large,eventually all available main memory in your computer will be consumed. The programwill first slow down, as the computer starts swapping pages to virtual memory,and then fail with an out-of-memory error. Finding leaks with a general debugger isdifficult because there is no obvious faulty statement. The bug is that a statement ismissing or not called.4.1.2 Incorrect Use of Memory ManagementA whole class of bugs is associated with incorrect calls to memory management:freeing a block of memory more than once, accessing memory after freeing it,or freeing a block that was never allocated. Also belonging to this class is usingdelete instead of delete for C+ array deallocation, as well as usingmalloc() together with delete, and using new together with free().4.1.3 Buffer OverrunsBuffer overruns are bugs where memory outside of the allocated boundaries is overwritten,or corrupted. Buffer overruns can occur for global variables, local variableson the stack, and dynamic variables that were allocated on the heap with memorymanagement.One nasty artifact of memory corruption is that the bug may not become visibleat the statement where the memory is overwritten. Only later, another statement inthe program will access this memory location. Because the memory location has anillegal value, the program can behave incorrectly in a number of ways: the programmay compute a wrong result, or, if the illegal value is in a pointer, the program willtry to access protected memory and crash. If a function pointer variable is overwritten,the program will do a jump and try to execute data as program code. Thekey point is that there may be no strict relation between the statement causing thememory corruption and the statement triggering the visible bug.4.1.4 Uninitialized Memory BugsReading uninitialized memory can occur because C/C+ allows creation of variableswithout an initial value. The programmer is fully responsible to initializeall global and local variables, either through assignment statements or through the4.2 Memory Debuggers to the Rescue 35various C+ constructors. The memory allocation function malloc() and operatornew also do not initialize or zero out the allocated memory blocks. Uninitializedvariables will contain unpredictable values.4.2 Memory Debuggers to the RescueThe above categories of memory access bugs created a need for adequate debuggingtools. Finding bugs related to leaked, corrupted, or uninitialized memory witha conventional debugger such as GDB turned out to be unproductive. To deal withmemory leaks in large software projects, many programmers came up with the sameidea. They created memory management functions/operators with special instrumentationto track where a memory block was allocated, and if each block wasproperly deallocated at the end of the program.Since everybody had the same memory bugs in their C/C+ programs, and sinceeverybody improvised with custom instrumentation to track down at least some ofthese bugs, a market for a tool called memory debugger was created. The most wellknowntool is Purify, released in 1991 by Pure Software. Purifys name has sincebecome synonymous with memory debugging. There is also Insure+, Valgrind, andBoundsChecker, among others. See the tools Appendix B.4 starting on page 198 forreferences and the survey in Luecke06 for a comparison of features.Memory debuggers do detailed bookkeeping of all allocated/deallocated dynamicmemory. They also intercept and check access to dynamic memory. Somememory debuggers can check access to local variables on the stack and statically allocatedmemory. Purify and BoundsChecker do this by object code instrumentationat program link time, Insure+ uses source code instrumentation, and Valgrind executesthe program on a virtual machine and monitors all memory transactions. Thecode instrumentation allows the tools to pinpoint the source code statement where amemory bug occurred.The following bugs are detectable by a memory debugger: Memory leaks Accessing memory that was already freed Freeing the same memory location more than once Freeing memory that was never allocated Mixing C malloc()/free()with C+ new/delete Using delete instead of delete for arrays Array out-of-bound errors Accessing memory that was never allocated Uninitialized memory read Null pointer read or writeWe will show in the next section how to attach a memory debugger to your program,and how the tool finds and reports bugs.36 4 Fixing Memory Problems4.3 Example 1: Detecting Memory Access ErrorsOur first example is a program that allocates an array in dynamic memory, accessesan element outside the final array element, reads an uninitialized array element, andfinally forgets to deallocate the array. We use the public domain tool Valgrind onLinux as the memory debugger, and demonstrate how the tool automatically detectsthese bugs. This is the code of our program main1.c:1 /* main1.c */2 #include <stdio.h>3 int main(int argc, char* argv) 4 const int size=100;5 int n, sum=0;6 int* A = (int*)malloc( sizeof(int)*size );78 for (n=size; n>0; n-) /* walk through A100.A1 */9 An = n; /* error: A100 invalid write*/10 for (n=0;n<size; n+) /* walk through A0.A99 */11 sum += An; /* error: A0 not initialized*/12 printf ("sum=%dn", sum);13 return 0; /* mem leak: A */14 We compile the program with debug information and then run under Valgrind:> gcc -g main1.c> valgrind -tool=memcheck -leak-check=yes ./a.outIn the following sections we go through the error list reported by Valgrind.4.3.1 Detecting an Invalid Write AccessThe first and perhaps most severe error is a buffer overrun: the accidental writeaccess to array element A100. Because the array has only 100 elements, thehighest valid index is 99. A100 points to unallocated memory that is locatedjust after the memory allocated for array A. Valgrind thus reports an “invalid write”error:=11323= Invalid write of size 4=11323= at 0x8048518: main (main1.c:9)=11323= Address 0x1BB261B8 is 0 bytes after a block=11323= of size 400 allocd=11323= at 0x1B903F40: malloc=11323= (in /usr/lib/valgrind/vgpreload_memcheck.so)=11323= by 0x80484F2: main (main1.c:6)The string "=11323=" refers to the process ID and is useful when Valgrind ischecking multiple processes 1. The important piece of information is that an invalid1 Valgrind will, per default, check only the first (parent) process that has been invoked. Use option-trace-children=yes to check all child processes as well.4.3 Example 1: Detecting Memory Access Errors 37write occurs in line 9 of main1.c. There is also additional information revealingthe address of the closest allocated memory block and how it was allocated. Thememory debugger guesses that the invalid write in line 9 is related to this memoryblock. The guess is correct because both belong to the same array A.Note that Valgrind is able to catch an out-of-array-bounds errors only when thearray is allocated as dynamic memory with malloc() or new. This is the case inthe example with the statement in line 6:6 int* A = (int*)malloc( sizeof(int)*size );If the example were instead written as int Asize in line 6, then A would bea local variable located on the stack and not on the heap. It turns out that Valgrinddoes not detect such an error but Purify is able to catch it. This shows that not allmemory debuggers will report exactly the same errors.修正內(nèi)存問題本章是關(guān)于利用內(nèi)存BUG調(diào)試器的幫助，來發(fā)現(xiàn)C或C+程序中的BUG。內(nèi)存調(diào)試器是在C或C+程序管理或存儲時(shí)用于追蹤和發(fā)現(xiàn)BUG的運(yùn)行時(shí)工具。它不能取代一個(gè)正規(guī)的調(diào)試器。在一下的章節(jié)，我們將描述幾個(gè)代表性的在C或C+程序中處理的內(nèi)容BUG，介紹內(nèi)容調(diào)試器，并且介紹兩個(gè)內(nèi)存如何發(fā)現(xiàn)BUG的例子。我們?nèi)缓髮⒀菔救绾问箖?nèi)存與編碼調(diào)試器一起運(yùn)行，怎么去處理不需要的錯(cuò)誤信息通過寫文件，需要去考慮程序需要的限制。4.1 內(nèi)存管理在C或C+中強(qiáng)健但是危險(xiǎn)C或C+語言能夠管理內(nèi)存資源，并且通過指針直接讀取內(nèi)存。高效的內(nèi)存處理和“直接控制硬件”是規(guī)劃大型例如操作系統(tǒng)的軟件工程取代匯編語言的原因，高性能和低限制是它作為一個(gè)重要角色的原因。動態(tài)內(nèi)存的分配（眾所周知的堆內(nèi)存）在C或C+中是在程序員的控制之下的。新內(nèi)存被分配擔(dān)任起例如malloc（）和各種有程序員構(gòu)成的新的指針。不被使用的內(nèi)存被free（）或delete回收。4.1.1 內(nèi)存泄漏內(nèi)存泄漏是在運(yùn)行時(shí)的內(nèi)存分配結(jié)構(gòu)，但是它們只需要我們分配一次，我們卻分配了多次。如果內(nèi)存泄漏經(jīng)常發(fā)生或者十分巨大，最后你電腦的主要內(nèi)存將被耗盡。程序首先會變慢，然后電腦開始使用分頁技術(shù)虛擬內(nèi)存，然后以內(nèi)存溢出錯(cuò)誤宣告失敗。用普遍的調(diào)試器發(fā)現(xiàn)內(nèi)存溢出錯(cuò)誤是困難的因?yàn)樗鼪]有一個(gè)明顯的錯(cuò)誤聲明。BUG會丟失或不能被聲明。4.1.2 錯(cuò)誤的內(nèi)存管理用法一套完整BUG集合與錯(cuò)誤聲明對于內(nèi)存管理來說：解除一個(gè)錯(cuò)誤的內(nèi)存超過一次，在解除以后訪問內(nèi)存，或解除一個(gè)從分配過的阻塞內(nèi)存。用delete代替delete在C+數(shù)組中也屬于這個(gè)類，也可以用malloc（）和delete一起使用，用new和free（）一起使用。4.1.3 緩存溢出緩存溢出內(nèi)存被重載時(shí)超出范圍的BUG，或損壞。緩存溢出會由屬性引起，在堆中的局部變量，通過內(nèi)存管理分配在棧上的動態(tài)變量。在內(nèi)存中令人討厭的事是內(nèi)存重載的時(shí)候BUG沒有被聲明。只有在以后，另一個(gè)在問題中的聲明將存取在內(nèi)存中。因?yàn)閮?nèi)存含有一個(gè)不合法的變量，內(nèi)存會在一下幾個(gè)方面表現(xiàn)出異常：這個(gè)問題得出一個(gè)錯(cuò)誤的結(jié)果，或者如果一個(gè)不合法的值在指針上，這個(gè)問題將試圖存取和保護(hù)內(nèi)存。如果在存取的過程中一個(gè)動態(tài)指針可變，程序?qū)⑻幚硭凑粘绦虼a。鍵指針不是引起內(nèi)存溢出和嚴(yán)重問題的關(guān)鍵。4.1.4 非原始內(nèi)存BUG讀非原始內(nèi)存是因?yàn)镃或C+允許創(chuàng)建變量在沒有一個(gè)原始值的情況下。程序員有完全的責(zé)任給所有的屬性和局部變量賦值，也可以通過聲明，或各種C+的構(gòu)造器。內(nèi)存引導(dǎo)功能malloc（）和操作new也不能夠定義變量或者是內(nèi)存為空。未定義的變量中包含不穩(wěn)定的值。4.2 內(nèi)存調(diào)試解決上述問題處理BUG需要?jiǎng)?chuàng)建一個(gè)強(qiáng)大的BUG處理工具。尋找BUG與漏洞有關(guān)，不完善的，只有普通BUG調(diào)試器GDB內(nèi)存變得不安全。應(yīng)付大型軟件工程中的系統(tǒng)漏洞，很多程序員都提出了相同的意見。他們建立內(nèi)存管理機(jī)制為一個(gè)被分配阻塞內(nèi)存利用不同的機(jī)制，并且確保是否每個(gè)阻塞內(nèi)存在最后都得到了分配。自從每個(gè)人在各自的C或C+程序中遇到相同的內(nèi)存BUG，自從每個(gè)人用傳統(tǒng)的內(nèi)存處理器追蹤到很少的幾個(gè)BUG，一系列叫內(nèi)存調(diào)試器的工具誕生了。最總所周知的是Purify，在1991年被Pure軟件公司發(fā)布。Purify一直被用于一些同類的內(nèi)存調(diào)試器。也有Insure+，Valgrind，也有BoundsChecker，除此之外的一些其他同類工具。內(nèi)存調(diào)試器作用于所有被動態(tài)分配的內(nèi)存。他們同樣攔截和檢查分配過的內(nèi)存。一些內(nèi)存調(diào)試器能夠檢查一些被分配在棧上的局部變量和被分配過的內(nèi)存。Purify和BoundsChecker用代碼編譯工具完成這些工作在程序連接期間，Insure+利用源代碼編譯工具，Valgrind則把程序先執(zhí)行在虛擬機(jī)上并且監(jiān)聽所有的內(nèi)存處理。源代碼編譯工具允許一個(gè)BUG發(fā)生的時(shí)候精確的找到代碼發(fā)生的位置。一下的BUG被內(nèi)存調(diào)試器處理：1. 內(nèi)存泄漏2. 訪問已經(jīng)被釋放的內(nèi)存3. 釋放一段內(nèi)存空間不止一次4. 釋放從未被分配過的內(nèi)存5. 把C中的malloc（），free（）與C+中的 new/delette混雜在一起6. 在數(shù)組中用delete代替delete7. 數(shù)組越界錯(cuò)誤8. 訪問從未被分配過的內(nèi)存9. 未被定義過的內(nèi)存讀取10. 空指針讀或?qū)懳覀儗⒃谙乱徽鹿?jié)演示如何為你的程序附屬內(nèi)存調(diào)試器，內(nèi)存調(diào)試器如何發(fā)現(xiàn)和報(bào)告錯(cuò)誤。4.3 例1檢測內(nèi)存訪問錯(cuò)誤我們第一個(gè)例子是在動態(tài)內(nèi)存中分配一列，存取最后一個(gè)在數(shù)組序列外的元素，讀一個(gè)未定義的數(shù)組元素，最終忘記分配這個(gè)數(shù)組。我們用公共范圍工具Valgrind在Linux上作為內(nèi)存調(diào)試器，演示內(nèi)存調(diào)試器如何自動的解決這些BUG。4.3.1 檢查一個(gè)無用的寫操作第一個(gè)最大最嚴(yán)重的錯(cuò)誤是緩存溢出：對這個(gè)數(shù)組以外的寫操作。因?yàn)檫@個(gè)數(shù)組只有100個(gè)元素，最大的有效索引是99。100指針超過了這個(gè)數(shù)組的有效范圍。一次報(bào)告一個(gè)無用的寫錯(cuò)誤。=11323= Invalid write of size 4=11323= at 0x8048518: main (main1.c:9)=11323= Address 0x1BB261B8 is 0 bytes after a block=11323= of size 400 allocd=11323= at 0x1B903F40: malloc=11323= (in /usr/lib/valgrind/vgpreload_memcheck.so)=11323= by 0x80484F2: main (main1.c:6)這個(gè)字符串，應(yīng)用一個(gè)過程ID，當(dāng)Valgrind檢查過程的時(shí)候是十分必要的。最重要的信息塊是在第9行的寫操作。這依然有信息補(bǔ)充說明根據(jù)以離內(nèi)存最近的區(qū)域和它是怎么被分配的。內(nèi)存調(diào)試器猜測第9行與內(nèi)存阻塞有關(guān)。這個(gè)猜測被修正因?yàn)閮蓚€(gè)都屬于數(shù)組A。在Valgrind中內(nèi)存中數(shù)組下標(biāo)越界錯(cuò)誤只有當(dāng)動態(tài)內(nèi)存被定義為malloc()或new的情況下。這個(gè)是第6行的實(shí)例: 6 int*A = (int * )malloc (sizeof(int) * size)如果例子中書寫Asize在第6行，A將被定義在一個(gè)局部變量上，堆而不是棧上。它表現(xiàn)出Valgrind不能夠發(fā)現(xiàn)這樣的錯(cuò)誤但是Purity可以捕獲到它們。它說明不是所有的內(nèi)存調(diào)試器都能捕獲到同一個(gè)錯(cuò)誤。