Merge branch 'develop'

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2017 Markus Gaasedelen
+Copyright (c) 2017-2018 Markus Gaasedelen
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,18 +1,19 @@
-# Lighthouse - Code Coverage Explorer for IDA Pro
+# Lighthouse - A Code Coverage Explorer for Reverse Engineers
 <p align="center">
-<img alt="Lighthouse Plugin" src="screenshots/overview.png"/>
+<img alt="Lighthouse Plugin" src="screenshots/overview.gif"/>
 </p>
 
 ## Overview
 
-Lighthouse is a code coverage plugin for [IDA Pro](https://www.hex-rays.com/products/ida/). The plugin leverages IDA as a platform to map, explore, and visualize externally collected code coverage data when symbols or source may not be available for a given binary.
+Lighthouse is a code coverage plugin for [IDA Pro](https://www.hex-rays.com/products/ida/), and [Binary Ninja](https://binary.ninja/). The plugin makes use of interactive disassemblers to map, explore, and visualize externally collected code coverage data when symbols or source may not be available for a given binary.
 
-This plugin is labeled only as a prototype and IDA / Qt code example for the community. 
+This plugin is labeled only as a prototype & code resource for the community. 
 
 Special thanks to [@0vercl0k](https://twitter.com/0vercl0k) for the inspiration.
 
 ## Releases
 
+* v0.8 -- Binary Ninja support, HTML coverage reports, consistent styling, many tweaks, bugfixes.
 * v0.7 -- Frida, C++ demangling, context menu, function prefixing, tweaks, bugfixes.
 * v0.6 -- Intel pintool, cyclomatic complexity, batch load, bugfixes.
 * v0.5 -- Search, IDA 7 support, many improvements, stability.
@@ -21,34 +22,56 @@ Special thanks to [@0vercl0k](https://twitter.com/0vercl0k) for the inspiration.
 * v0.2 -- Multifile support, performance improvements, bugfixes.
 * v0.1 -- Initial release
 
-## Installation
+# IDA Pro Installation
 
-Install Lighthouse into the IDA plugins folder.
+Lighthouse is a cross-platform (Windows, macOS, Linux) python plugin, supporting IDA Pro 6.8 and newer.
 
 - Copy the contents of the `plugin` folder to the IDA plugins folder
  - On Windows, the folder is at `C:\Program Files (x86)\IDA 6.8\plugins`
- - On MacOS, the folder is at `/Applications/IDA\ Pro\ 6.8/idaq.app/Contents/MacOS/plugins`
+ - On macOS, the folder is at `/Applications/IDA\ Pro\ 6.8/idaq.app/Contents/MacOS/plugins`
  - On Linux, the folder may be at `/opt/IDA/plugins/`
 
-The plugin is compatible with IDA Pro 6.8 --> 7.0 on Windows, MacOS, and Linux.
+It has been primarily developed and tested on Windows, so that is where we expect the best experience.
 
-## Usage
+# Binary Ninja Installation (Experimental)
 
-Lighthouse loads automatically when an IDB is opened, installing a handful of menu entries into the IDA interface.
+At this time, support for Binary Ninja is considered experimental. Please feel free to report any bugs that you encounter.
 
-<p align="center">
-<img alt="Lighthouse Menu Entries" src="screenshots/open.gif"/>
-</p>
+You can install Lighthouse & PyQt5 for Binary Ninja by following the instructions below.
 
-These are the entry points for a user to load and view coverage data.
+## Windows Installation
+
+1. Install PyQt5 from a Windows command prompt with the following command:
+
+```
+pip install --target="%appdata%\Binary Ninja\plugins\Lib\site-packages" python-qt5
+```
+
+2. Copy the contents of the `/plugin/` folder in this repo to your Binary Ninja [plugins folder](https://docs.binary.ninja/guide/plugins/index.html#using-plugins).
+
+## Linux Installation
+
+1. Install PyQt5 from a Linux shell with the following command:
 
 ```
-- File --> Load file --> Code coverage file...
-- File --> Load file --> Code coverage batch...
-- View --> Open subviews --> Coverage Overview
+sudo apt install python-pyqt5
 ```
 
-Batch load can quickly aggregate hundreds (thousands?) of collected coverage files into a single composite at load time.
+2. Copy the contents of the `/plugin/` folder in this repo to your Binary Ninja [plugins folder](https://docs.binary.ninja/guide/plugins/index.html#using-plugins).
+
+## macOS Installation
+
+¯\\\_(ツ)\_/¯
+
+# Usage
+
+Lighthouse loads automatically when a database is opened, installing a handful of menu entries into the disassembler.
+
+<p align="center">
+<img alt="Lighthouse Menu Entries" src="screenshots/open.gif"/>
+</p>
+
+These are the entry points for a user to load and view coverage data.
 
 ## Coverage Painting
 
@@ -58,6 +81,8 @@ Lighthouse 'paints' the active coverage data across the three major IDA views as
 <img alt="Lighthouse Coverage Painting" src="screenshots/painting.png"/>
 </p>
 
+In Binary Ninja, only the Disassembly and Graph views are supported.
+
 ## Coverage Overview
 
 The Coverage Overview is a dockable widget that provides a function level view of the active coverage data for the database.
@@ -141,6 +166,15 @@ Loaded coverage data and user constructed compositions can be selected or delete
 <img alt="Lighthouse Coverage ComboBox" src="screenshots/combobox.gif"/>
 </p>
 
+## HTML Coverage Report
+
+Lighthouse can generate a rudimentary HTML coverage report of the active coverage. 
+A sample report can be seen [here](https://rawgit.com/gaasedelen/lighthouse/master/testcase/report.html).
+
+<p align="center">
+<img alt="Lighthouse HTML Report" src="screenshots/html_report.gif"/>
+</p>
+
 # Collecting Coverage
 
 Before using Lighthouse, one will need to collect code coverage data for their target binary / application.
@@ -157,7 +191,7 @@ Example usage:
 ..\DynamoRIO-Windows-7.0.0-RC1\bin64\drrun.exe -t drcov -- boombox.exe
 ```
 
-## Intel Pin (Experimental)
+## Intel Pin
 
 Using a [custom pintool](coverage/pin) contributed by [Agustin Gianni](https://twitter.com/agustingianni), the Intel Pin DBI can also be used to collect coverage data.
 
@@ -167,7 +201,7 @@ Example usage:
 pin.exe -t CodeCoverage64.dll -- boombox.exe
 ```
 
-For convenience, binaries for the Windows pintool can be found on the [releases](https://github.com/gaasedelen/lighthouse/releases/tag/v0.7.0) page. MacOS and Linux users need to compile the pintool themselves following the [instructions](coverage/pin#compilation) included with the pintool for their respective platforms.
+For convenience, binaries for the Windows pintool can be found on the [releases](https://github.com/gaasedelen/lighthouse/releases/tag/v0.8.0) page. macOS and Linux users need to compile the pintool themselves following the [instructions](coverage/pin#compilation) included with the pintool for their respective platforms.
 
 ## Frida (Experimental)
 
@@ -186,9 +220,12 @@ Time and motivation permitting, future work may include:
 * ~~Asynchronous composition, painting, metadata collection~~
 * ~~Multifile/coverage support~~
 * Profiling based heatmaps/painting
-* Coverage & Profiling Treemaps
+* Coverage & profiling treemaps
 * ~~Additional coverage sources, trace formats, etc~~
-* Improved Pseudocode painting
+* Improved pseudocode painting
+* Lighthouse console access, headless usage
+* Custom themes
+* Python 3 support
 
 I welcome external contributions, issues, and feature requests.
 

dev_scripts/close_BINJA.bat

@@ -0,0 +1,9 @@
+REM - Clean up an existing or past 'test session' 
+taskkill /F /IM "binaryninja.exe"
+timeout 1
+REM del "..\..\testcase\*.id0" 
+REM del "..\..\testcase\*.id1" 
+REM del "..\..\testcase\*.id2" 
+REM del "..\..\testcase\*.nam" 
+REM del "..\..\testcase\*.til"
+REM del "..\..\testcase\*.$$$"

dev_scripts/reload_BINJA.bat

@@ -0,0 +1,18 @@
+set LIGHTHOUSE_LOGGING=1
+REM - Close any running instances of IDA
+call close_BINJA.bat
+
+REM - Purge old lighthouse log files
+del /F /Q "C:\Users\user\AppData\Roaming\Binary Ninja\lighthouse_logs\*"
+
+REM - Delete the old plugin bits
+del /F /Q "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\*lighthouse_plugin.py"
+rmdir "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\lighthouse" /s /q
+
+REM - Copy over the new plugin bits
+xcopy /s/y "..\plugin\*" "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\"
+del /F /Q "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\.#lighthouse_plugin.py"
+
+REM - Launch a new IDA session
+start "" "C:\tools\disassemblers\BinaryNinja\binaryninja.exe" "..\..\testcase\boombox.bndb"
+

dev_scripts/reload_BINJA_IDA.bat

@@ -0,0 +1,18 @@
+set LIGHTHOUSE_LOGGING=1
+REM - Close any running instances of IDA
+call close_BINJA.bat
+
+REM - Purge old lighthouse log files
+del /F /Q "C:\Users\user\AppData\Roaming\Binary Ninja\lighthouse_logs\*"
+
+REM - Delete the old plugin bits
+del /F /Q "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\*lighthouse_plugin.py"
+rmdir "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\lighthouse" /s /q
+
+REM - Copy over the new plugin bits
+xcopy /s/y "..\plugin\*" "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\"
+del /F /Q "C:\Users\user\AppData\Roaming\Binary Ninja\plugins\.#lighthouse_plugin.py"
+
+REM - Launch a new IDA session
+start "" "C:\tools\disassemblers\BinaryNinja\binaryninja.exe" "..\..\testcase\idaq.bndb"
+

dev_scripts/reload_IDA_7.bat

@@ -1,3 +1,4 @@
+set LIGHTHOUSE_LOGGING=1
 REM - Close any running instances of IDA
 call close_IDA.bat
 

dev_scripts/reload_IDA_71_ida.bat

@@ -0,0 +1,18 @@
+set LIGHTHOUSE_LOGGING=1
+REM - Close any running instances of IDA
+call close_IDA.bat
+
+REM - Purge old lighthouse log files
+del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\lighthouse_logs\*"
+
+REM - Delete the old plugin bits
+del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\*lighthouse_plugin.py"
+rmdir "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\lighthouse" /s /q
+
+REM - Copy over the new plugin bits
+xcopy /s/y "..\plugin\*" "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\"
+del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\.#lighthouse_plugin.py"
+
+REM - Launch a new IDA session
+start "" "C:\tools\disassemblers\IDA 7.1\ida.exe" "..\..\testcase\idaq71.idb"
+

dev_scripts/reload_IDA_7_ida.bat

@@ -1,3 +1,4 @@
+set LIGHTHOUSE_LOGGING=1
 REM - Close any running instances of IDA
 call close_IDA.bat
 

dev_scripts/reload_IDA_8.bat

@@ -1,3 +1,4 @@
+set LIGHTHOUSE_LOGGING=1
 REM - Close any running instances of IDA
 call close_IDA.bat
 

dev_scripts/reload_IDA_8_big.bat

@@ -0,0 +1,18 @@
+set LIGHTHOUSE_LOGGING=1
+REM - Close any running instances of IDA
+call close_IDA.bat
+
+REM - Purge old lighthouse log files
+del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\lighthouse_logs\*"
+
+REM - Delete the old plugin bits
+del /F /Q "C:\tools\disassemblers\IDA 6.8\plugins\*lighthouse_plugin.py"
+rmdir "C:\tools\disassemblers\IDA 6.8\plugins\lighthouse" /s /q
+
+REM - Copy over the new plugin bits
+xcopy /s/y "..\plugin\*" "C:\tools\disassemblers\IDA 6.8\plugins\"
+del /F /Q "C:\tools\disassemblers\IDA 6.8\plugins\.#lighthouse_plugin.py"
+
+REM - Relaunch two IDA sessions
+start "" "C:\tools\disassemblers\IDA 6.8\idaq.exe" "..\..\testcase\harness_ufs_pdf.instr.idb"
+

dev_scripts/reload_IDA_8_ida.bat

@@ -1,3 +1,4 @@
+set LIGHTHOUSE_LOGGING=1
 REM - Close any running instances of IDA
 call close_IDA.bat
 

dev_scripts/reload_IDA_95.bat

@@ -1,16 +1,17 @@
+set LIGHTHOUSE_LOGGING=1
 REM - Close any running instances of IDA
 call close_IDA.bat
 
 REM - Purge old lighthouse log files
 del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\lighthouse_logs\*"
 
 REM - Delete the old plugin bits
-del /F /Q "C:\tools\disassemblers\IDA 6.95\plugins\*lighthouse_plugin.py"
-rmdir "C:\tools\disassemblers\IDA 6.95\plugins\lighthouse" /s /q
+del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\*lighthouse_plugin.py"
+rmdir "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\lighthouse" /s /q
 
 REM - Copy over the new plugin bits
-xcopy /s/y "..\plugin\*" "C:\tools\disassemblers\IDA 6.95\plugins\"
-del /F /Q "C:\tools\disassemblers\IDA 6.95\plugins\.#lighthouse_plugin.py"
+xcopy /s/y "..\plugin\*" "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\"
+del /F /Q "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\plugins\.#lighthouse_plugin.py"
 
 REM - Relaunch two IDA sessions
 start "" "C:\tools\disassemblers\IDA 6.95\idaq64.exe" "..\..\testcase\boombox95.i64"

dev_scripts/test_headless.py

@@ -0,0 +1,14 @@
+import os
+import binaryninja
+
+target = os.path.abspath(os.path.join(
+ os.path.dirname(os.path.abspath(__file__)),
+ "..",
+ "testcase",
+ "boombox.exe"
+))
+
+print "Opening '%s' headlessly..." % target
+x = binaryninja.BinaryViewType["PE"].open(target)
+x.update_analysis_and_wait()
+print "DONE!"

plugin/lighthouse/binja_integration.py

@@ -0,0 +1,65 @@
+import logging
+
+from binaryninja import PluginCommand
+from lighthouse.core import Lighthouse
+from lighthouse.util.disassembler import disassembler
+
+logger = logging.getLogger("Lighthouse.Binja.Integration")
+
+#------------------------------------------------------------------------------
+# Lighthouse Binja Integration
+#------------------------------------------------------------------------------
+
+class LighthouseBinja(Lighthouse):
+ """
+ Lighthouse UI Integration for Binary Ninja.
+ """
+
+ def __init__(self):
+ super(LighthouseBinja, self).__init__()
+
+ def interactive_load_file(self, bv):
+ disassembler.bv = bv
+ super(LighthouseBinja, self).interactive_load_file()
+
+ def interactive_load_batch(self, bv):
+ disassembler.bv = bv
+ super(LighthouseBinja, self).interactive_load_batch()
+
+ def interactive_load_batch(self, bv):
+ disassembler.bv = bv
+ super(LighthouseBinja, self).open_coverage_overview()
+
+ def _install_load_file(self):
+ PluginCommand.register(
+ "Lighthouse - Load code coverage file...",
+ "Load individual code coverage file(s)",
+ self.interactive_load_file
+ )
+ logger.info("Installed the 'Code coverage file' menu entry")
+
+ def _install_load_batch(self):
+ PluginCommand.register(
+ "Lighthouse - Load code coverage batch...",
+ "Load and aggregate code coverage files",
+ self.interactive_load_batch
+ )
+ logger.info("Installed the 'Code coverage batch' menu entry")
+
+ def _install_open_coverage_overview(self):
+ PluginCommand.register(
+ "Lighthouse - Coverage Overview",
+ "Open the database code coverage overview",
+ self.interactive_load_batch
+ )
+ logger.info("Installed the 'Coverage Overview' menu entry")
+
+ # TODO/V35: No good signals to unload (core) plugin on
+ def _uninstall_load_file(self):
+ pass
+
+ def _uninstall_load_batch(self):
+ pass
+
+ def _uninstall_open_coverage_overview(self):
+ pass