Skip to content

Commit

Permalink
Release 1.1.0
Browse files Browse the repository at this point in the history
Changed d-scripts and processing logic to only trace malloc/free/realloc/calloc and abandoned tracing new/delete
  • Loading branch information
ppissias committed Nov 12, 2018
1 parent bead8ce commit 8b5b6cd
Show file tree
Hide file tree
Showing 14 changed files with 2,583 additions and 420 deletions.
141 changes: 103 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,80 @@
# Dtleakanalyzer
# DtLeakAnalyzer

A tool for supporting identifying memory leaks with dtrace

### Overview

This tool is intended to be used in supporting memory leak investigations with dtrace. It consists of
- a set of D scripts (that attach to a process and start tracing)
- a Java program that analyzes the traces and produces a report

It re-uses the .d script defined in https://blogs.oracle.com/openomics/investigating-memory-leaks-with-dtrace moving most of the trace analysis to the newly created Java post-processing program. Moreover it tries to detect wrong delete operations (i.e. delete on memory that was allocated with new[])
- a **set of D scripts** (that attach to a process and start tracing)
- a **Java program that analyzes the traces** and produces a report

Overall there are **4 basic modes of usage**:
- **Single memory allocator tracing session**, where we collect once traces for a process and analyze them
- **Combination of multiple memory allocator tracing sessions**, where we combine multiple tracing sessions of different time lengths
- **Combination of multiple short-term and long-term traces memory allocator** (There are different d-scripts for the short-term and long-term traces. The short-term traces are used to train the system in order to interpret the long term traces)
- **Process memory growth analysis**

In summary it provides the following features
- **Single memory allocator tracing session**
-- presentation of call stacks that appear to be causing memory leaks
-- heuristic analysis for pointing out strongly suspected memory leaks
-- presentation of call stacks that appear to be freeing memory wrongly
-- heuristic analysis for pointing out stringly suspected wrong free call stacks
-- identification of double free operations
-- presentation of a combined call stack where the potential memory leaks are identified
- **Combination of multiple memory allocator tracing sessions** (on top of the features above)
-- combined presentation of the occurence of suspected call stacks for all trace files
-- heuristic analysis for pointing out very strongly suspected call stacks
- **Combination of multiple short-term and long-term traces** (on top of the features above)
-- heuristic analysis of long term traces
- **Process memory growth analysis**
-- presentation of call stacks that caused memory growth, including occurences and total size
-- presentation of a combined call stack where all calls that caused memory growth are presented

More information on the usage and capabilities of DtLeakAnalyzer can be found [in the DtLeakAnalyzer usage manual](resources/DtLeakAnalyzer.pdf)

### Running

First we collect traces from the running process
Detailed instructions can be found in [in the DtLeakAnalyzer usage manual](resources/DtLeakAnalyzer.pdf)

As a **single memory allocator tracing session** example, first we collect traces from the running process
i.e.
```
> trace_new_delete.d <PID> > traces.txt
> ./trace-memalloc.d 14291 > trace-memalloc.log
```
(press ctrl^C when we are done)
then we demangle names in the call stack (this is an optional step to show the call stacks of your program clearly)

then we tun the trace analysis program
```
> c++filt traces.txt > traces.dem.txt
> java -jar dtleakanalyzer.jar -f memalloc trace-memalloc.log trace-memalloc.log.report
```
then we tun the trace analysis program
The output of the trace analysis tool for this case is:
```
> java -jar dtleakanalyzer.jar traces.dem.txt traces.dem.txt.report
Output:
Trace analyser started on Mon Oct 08 13:40:45 CEST 2018
Processing wrong deletes [delete on memory that was allocated with new[]), found 0 instances
found 0 unique wrong delete stacks
Analyzing 2290 potential memory leaks
Processing completed.
Detected 108 potential memory leaks
Started memory allocator analysis for file memalloc trace-memalloc.log on:Thu Nov 08 08:03:54 CET 2018
Finished memory allocator analysis for file memalloc trace-memalloc.log on:Thu Nov 08 08:04:05 CET 2018
Call statistics
Found 142511 malloc calls
Found 0 calloc calls
Found 0 realloc calls
Found 142427 free calls
Double free issues
Found 0 double free stacks in total
Free non-allocated memory issues (may also be potential memory leaks)
Found 2617 stacks that freed memory that was not allocated during the period of the trace
Found 17 unique stacks that freed memory that was not allocated during the period of the trace
Found 495 unique stacks that correctly freed memory
Found 0 unique stacks that have never been found to correctly free memory
Memory leak issues
Found 2701 potential memory leaks in total
Found 44 unique potential memory leak stacks (suspects)
Found 553 unique stacks that allocated memory that was correctly freed
Found 1 unique stacks that were never correctly deleted/freed (strong suspects)
finished on Mon Oct 08 13:40:47 CEST 2018
```
This will produce a report (traces.dem.txt.report) which contains all useful information and potential memory leaks.

The report will be procuded in the specified file: trace-memalloc.log.report and will contain all relevant information about the identified call stacks and heuristics.

### Compiling

Expand All @@ -50,30 +86,59 @@ Extract the reporitory and run the compile.bat or .sh

```
c:\dev\projects\DTLeakAnalyzer>compile.bat
c:\dev\projects\DTLeakAnalyzer>javac -d classes src/DTLeakAnalyzer.java
c:\dev\projects\DTLeakAnalyzer>jar cvfm dtleakanalyzer.jar resources/manifest.txt -C classes .
added manifest
adding: DTLeakAnalyzer$1.class(in = 894) (out= 465)(deflated 47%)
adding: DTLeakAnalyzer$2.class(in = 839) (out= 451)(deflated 46%)
adding: DTLeakAnalyzer$3.class(in = 793) (out= 479)(deflated 39%)
adding: DTLeakAnalyzer$DTLeakLogEntry.class(in = 3509) (out= 1811)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakLogEntryType.class(in = 1112) (out= 582)(deflated 47%)
adding: DTLeakAnalyzer$DTLeakReportEntry.class(in = 703) (out= 431)(deflated 38%)
adding: DTLeakAnalyzer$DTLeakWrongDeleteEntry.class(in = 806) (out= 412)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakWrongDeleteReportEntry.class(in = 1020) (out= 506)(deflated 50%)
adding: DTLeakAnalyzer.class(in = 7823) (out= 3849)(deflated 50%)
adding: classes.txt(in = 0) (out= 0)(stored 0%)
adding: DTLeakAnalyzer$1.class(in = 561) (out= 380)(deflated 32%)
adding: DTLeakAnalyzer$10.class(in = 1475) (out= 709)(deflated 51%)
adding: DTLeakAnalyzer$11.class(in = 795) (out= 458)(deflated 42%)
adding: DTLeakAnalyzer$12.class(in = 799) (out= 461)(deflated 42%)
adding: DTLeakAnalyzer$13.class(in = 806) (out= 467)(deflated 42%)
adding: DTLeakAnalyzer$14.class(in = 1991) (out= 872)(deflated 56%)
adding: DTLeakAnalyzer$2.class(in = 561) (out= 380)(deflated 32%)
adding: DTLeakAnalyzer$3.class(in = 561) (out= 376)(deflated 32%)
adding: DTLeakAnalyzer$4.class(in = 838) (out= 458)(deflated 45%)
adding: DTLeakAnalyzer$5.class(in = 838) (out= 463)(deflated 44%)
adding: DTLeakAnalyzer$6.class(in = 838) (out= 459)(deflated 45%)
adding: DTLeakAnalyzer$7.class(in = 842) (out= 456)(deflated 45%)
adding: DTLeakAnalyzer$8.class(in = 806) (out= 463)(deflated 42%)
adding: DTLeakAnalyzer$9.class(in = 806) (out= 464)(deflated 42%)
adding: DTLeakAnalyzer$BrkStackOccurence.class(in = 1024) (out= 568)(deflated 44%)
adding: DTLeakAnalyzer$BrkTraceEntry.class(in = 3561) (out= 1856)(deflated 47%)
adding: DTLeakAnalyzer$BrkTraceEntryType.class(in = 990) (out= 523)(deflated 47%)
adding: DTLeakAnalyzer$DTGenericLeakLogEntry.class(in = 3648) (out= 1850)(deflated 49%)
adding: DTLeakAnalyzer$DTLeakAnalyzerFileType.class(in = 1029) (out= 519)(deflated 49%)
adding: DTLeakAnalyzer$DTLeakBrkLogEntry.class(in = 3586) (out= 1841)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakBrkReportEntry.class(in = 998) (out= 548)(deflated 45%)
adding: DTLeakAnalyzer$DTLeakCppLogEntry.class(in = 3531) (out= 1830)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakGenLogEntry.class(in = 3640) (out= 1873)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakLogBrkEntryType.class(in = 1018) (out= 521)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakLogCppEntryType.class(in = 1133) (out= 586)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakLogEntry.class(in = 3510) (out= 1828)(deflated 47%)
adding: DTLeakAnalyzer$DTLeakLogEntryType.class(in = 1112) (out= 586)(deflated 47%)
adding: DTLeakAnalyzer$DTLeakLogGenEntryType.class(in = 1128) (out= 580)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakMemAllocLogEntry.class(in = 3649) (out= 1866)(deflated 48%)
adding: DTLeakAnalyzer$DTLeakReportEntry.class(in = 799) (out= 454)(deflated 43%)
adding: DTLeakAnalyzer$DTLeakWrongDeleteEntry.class(in = 824) (out= 414)(deflated 49%)
adding: DTLeakAnalyzer$DTLeakWrongDeleteReportEntry.class(in = 1020) (out= 510)(deflated 50%)
adding: DTLeakAnalyzer$MemoryAllocationTraceEntryType.class(in = 1191) (out= 596)(deflated 49%)
adding: DTLeakAnalyzer$MemoryAllocatorTraceEntry.class(in = 3697) (out= 1873)(deflated 49%)
adding: DTLeakAnalyzer$StackOccurence.class(in = 1142) (out= 583)(deflated 48%)
adding: DTLeakAnalyzer$TraceFileType.class(in = 966) (out= 521)(deflated 46%)
adding: DTLeakAnalyzer.class(in = 36673) (out= 15321)(deflated 58%)
```
This will create the dtleakanalyzer.jar on the current folder.
This will create the dtleakanalyzer.jar executable jar on the current folder.

## Contributing




Please feel free to extend the project!



## License

Expand All @@ -88,6 +153,6 @@ This project is licensed under the MIT License (https://opensource.org/licenses/
## Acknowledgments



* Thanks to the authors of the original article on how to use dtrace for supporting memory leak investigations : https://blogs.oracle.com/openomics/investigating-memory-leaks-with-dtrace moving
* Thanks to Brendan Gregg for his excellent page on memory leaks and ways to identify them: http://www.brendangregg.com/Solaris/memoryflamegraphs.html
* Thanks to the authors of this article on how to use dtrace for supporting memory leak investigations : https://blogs.oracle.com/openomics/investigating-memory-leaks-with-dtrace

Binary file removed resources/Analyzing reports with dtleakanalyzer.pdf
Binary file not shown.
15 changes: 15 additions & 0 deletions resources/D scripts/execute-memalloc-loop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
for REP in 1 2 3 4 5; do
for SLTIME in 10 20 40 ; do
echo tracing process:$1 for :$SLTIME seconds. Repetition:$REP

./trace-memalloc.d $1 > ./trace-memalloc.$SLTIME.$REP &
dtracepid=$!

echo started dtrace process, pid:$dtracepid
sleep $SLTIME
kill -SIGINT $dtracepid
echo sent interrupt signal to dtrace process
wait $dtracepid
echo dtrace finished
done
done
15 changes: 15 additions & 0 deletions resources/D scripts/execute-memalloc-loop-proc
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
for REP in 1 2 3 4 5; do
for SLTIME in 0010 0020 0040 0080 0160 0320 0640 1280 2560; do
echo tracing process:$1 for :$SLTIME seconds. Repetition:$REP

./trace-memalloc-proc.d $1 > ./trace-memalloc-proc.$SLTIME.$REP &
dtracepid=$!

echo started dtrace process, pid:$dtracepid
sleep $SLTIME
kill -SIGINT $dtracepid
echo sent interrupt signal to dtrace process
wait $dtracepid
echo dtrace finished
done
done
41 changes: 0 additions & 41 deletions resources/D scripts/trace-malloc-free.d

This file was deleted.

110 changes: 110 additions & 0 deletions resources/D scripts/trace-memalloc-proc.d
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
#!/usr/sbin/dtrace -s

/*
* Dtrace script that logs the number of times call stacks allocated or freed memory
* The output of the script is further processed as described in
* https://github.com/ppissias/DTLeakAnalyzer
*
* Adapt the aggsize, aggsize and bufsize parameters accordingly if needed.
* Author Petros Pissias
*/

#pragma D option quiet
#pragma D option aggrate=100us
#pragma D option aggsize=100m
#pragma D option bufpolicy=fill
#pragma D option bufsize=100m


#!/usr/sbin/dtrace -s

pid$1::malloc:entry
{
self->trace = 1;
self->size = arg0;
}

pid$1::malloc:return
/self->trace == 1/
{
@allocstacks[ustack(50)] = count();
@counts["created"] = count();
counts["pending"]++;

self->trace = 0;
self->size = 0;
}


pid$1::realloc:entry
{
self->trace = 1;
self->size = arg1;
self->oldptr = arg0;
}

pid$1::realloc:return
/ (self->trace == 1) && (self->size == 0)/
{
/* this is same as free, size=0 */
@deallocstacks[ustack(50)] = count();
@counts["deleted"] = count();
counts["pending"]--;

self->trace = 0;
self->size = 0;
self->oldptr = 0;
}

pid$1::realloc:return
/ (self->trace == 1) && (self->size > 0)/
{
/* this is a deallocation and a new allocation */
@deallocstacks[ustack(50)] = count();
@allocstacks[ustack(50)] = count();

self->trace = 0;
self->size = 0;
self->oldptr = 0;
}

pid$1::calloc:entry
{
self->trace = 1;
self->size = arg1;
self->nelements = arg0;
}

pid$1::calloc:return
/self->trace == 1/
{
/* log the memory allocation */
@allocstacks[ustack(50)] = count();
@counts["created"] = count();
counts["pending"]++;

self->trace = 0;
self->size = 0;
self->nelements = 0;
}



pid$1::free:entry
{
@deallocstacks[ustack(50)] = count();
@counts["deleted"] = count();
counts["pending"]--;
}

END
{
printf("== FINISHED ==\n\n");
printf("== allocation stacks ==\n\n");
printa(@allocstacks);
printf("\n== deallocation stacks ==\n\n");
printa(@deallocstacks);
printf("\n== mem allocations vs deletions ==\n\n");
printa(@counts);
printf("number of allocations - number of deallocations: %d",counts["pending"]);
}
Loading

0 comments on commit 8b5b6cd

Please sign in to comment.