Merge pull request #46 from jspahrsummers/librarization

Restructure like a proper Python library

.circleci/config.yml

@@ -38,7 +38,7 @@ jobs:
 
       - run:
           name: install dependencies
-          command: script/bootstrap
+          command: script/bootstrap --skip-license
 
       - save_cache:
           paths:
@@ -53,6 +53,10 @@ jobs:
           name: check formatting
           when: always
           command: script/reformat --diff
+
+      - run:
+          name: try installing
+          command: script/test_install --skip-license
 
 workflows:
   version: 2

CONTRIBUTING.md

@@ -2,6 +2,20 @@
 
 Thanks for wanting to contribute to this project!
 
+## Setting up your environment
+
+The included bootstrap script will set up a Python virtual environment and install the necessary dependencies, including the [Interactive Brokers API](http://interactivebrokers.github.io):
+
+```
+script/bootstrap
+```
+
+After bootstrapping, confirm that the environment works by running the included test suite:
+
+```
+script/test
+```
+
 ## Issues
 
 Bug reports and enhancement requests are more than welcome. However, if you're able to, a [pull request](#pull-requests) is much better!

README.md

@@ -3,46 +3,54 @@ Ingest portfolio and other data from multiple brokerages, and analyze it.
 
 **Table of contents:**
 
-1. [Getting started](#getting-started)
+1. [Installation](#installation)
 1. [Connecting to brokers](#connecting-to-brokers)
    1. [Interactive Brokers](#interactive-brokers)
    1. [Charles Schwab](#charles-schwab)
    1. [Fidelity](#fidelity)
    1. [Vanguard](#vanguard)
 1. [Extending `bankroll`](#extending-bankroll)
 
-# Getting started
+# Installation
 
-The included bootstrap script will set up a Python virtual environment and install the necessary dependencies, including the [Interactive Brokers API](http://interactivebrokers.github.io):
+To install `bankroll` as a Python package, simply run `pip` (or `pip3`, as it may be named on your system) from the repository root:
 
 ```
-script/bootstrap
+pip install .
 ```
 
-After bootstrapping, confirm that the environment works by running the included test suite:
+This will also make the command-line tool available directly:
 
 ```
-script/test
+bankroll --help
 ```
 
+The [Interactive Brokers API](http://interactivebrokers.github.io) must also be installed prior to using `bankroll`. This repository includes a helpful script to ease this process:
+
+```
+script/install_twsapi
+```
+
+If you do not want to install `ibapi` globally, you may wish to [set up a Python virtual environment](CONTRIBUTING.md#setting-up-your-environment) instead.
+
 # Connecting to brokers
 
 After being set up, `bankroll` can be used from the command line to bring together data from multiple brokerages.
 
 For example, to show all positions held in both Interactive Brokers and Charles Schwab:
 
 ```
-python3 bankroll.py \
+python -m bankroll \
   --twsport 7496 \
   --schwabpositions ~/Positions-2019-01-01.CSV \
   --schwabtransactions ~/Transactions_20190101.CSV \
   positions
 ```
 
-Run with `-h` to see all options:
+Run with `--help` to see all options:
 
 ```
-python3 bankroll.py -h
+python -m bankroll --help
 ```
 
 ## Interactive Brokers
@@ -56,7 +64,7 @@ Unfortunately, [one of IB's trading applications](https://interactivebrokers.git
 Once Trader Workstation or IB Gateway is running, and [API connections are enabled](https://interactivebrokers.github.io/tws-api/initial_setup.html#enable_api), provide the local port number to `bankroll` like so:
 
 ```
-python3 bankroll.py \
+python -m bankroll \
   --twsport 7496 \
   [command]
 ```
@@ -101,7 +109,7 @@ Under _Sections_, click _Trade Confirmations_ and enable everything in the dialo
 With the token and the query ID from your account, historical trades can be downloaded:
 
 ```
-python3 bankroll.py \
+python -m bankroll \
   --flextoken [token] \
   --flexquery-trades [query ID] \
   activity
@@ -120,7 +128,7 @@ The only section which needs to be enabled is _Change in Dividend Accruals_:
 Pass your existing token, and the new query's ID, on the command line:
 
 ```
-python3 bankroll.py \
+python -m bankroll \
   --flextoken [token] \
   --flexquery-activity [query ID] \
   activity
@@ -141,7 +149,7 @@ Click the "Export" link in the top-right:
 Then provide the paths of either or both these downloaded files to `bankroll`:
 
 ```
-python3 bankroll.py \
+python -m bankroll \
   --schwabpositions ~/path/to/Positions.CSV \
   --schwabtransactions ~/path/to/Transactions.CSV \
   [command]

bankroll/__init__.py

@@ -0,0 +1,3 @@
+from .analysis import *
+from .brokers import *
+from .model import *

bankroll.py → bankroll/__main__.py

@@ -1,19 +1,15 @@
 from argparse import ArgumentParser, Namespace
 from functools import reduce
 from ib_insync import IB
-from model import Activity, Instrument, Stock, Position, Trade, Cash, MarketDataProvider
+from bankroll import Activity, Instrument, Stock, Position, Trade, Cash, MarketDataProvider, analysis
+from bankroll.brokers import *
 from pathlib import Path
 from progress.bar import Bar
 from typing import Dict, Iterable, List, Optional
 
-import analysis
-import ibkr
-import fidelity
 import logging
-import schwab
-import vanguard
 
-parser = ArgumentParser()
+parser = ArgumentParser(prog='bankroll')
 
 parser.add_argument(
     '--lenient',
@@ -158,7 +154,12 @@ def printActivity(args: Namespace) -> None:
 activityParser = subparsers.add_parser(
     'activity', help='Operations upon imported portfolio activity')
 
-if __name__ == '__main__':
+
+def main() -> None:
+    global positions
+    global activity
+    global dataProvider
+
     args = parser.parse_args()
     if args.verbose:
         logging.basicConfig(level=logging.INFO)
@@ -217,3 +218,7 @@ def printActivity(args: Namespace) -> None:
 
     positions = list(analysis.deduplicatePositions(positions))
     commands[args.command](args)
+
+
+if __name__ == '__main__':
+    main()

analysis.py → bankroll/analysis.py

@@ -1,6 +1,6 @@
 from functools import reduce
 from itertools import groupby
-from model import Activity, Cash, DividendPayment, Trade, Instrument, Option, MarketDataProvider, Quote, Position
+from .model import Activity, Cash, DividendPayment, Trade, Instrument, Option, MarketDataProvider, Quote, Position
 from progress.bar import Bar
 from typing import Dict, Iterable, Optional, Tuple
 
@@ -9,19 +9,19 @@
 
 # Different brokers represent "identical" symbols differently, and they can all be valid.
 # This function normalizes them so they can be compared across time and space.
-def normalizeSymbol(symbol: str) -> str:
+def _normalizeSymbol(symbol: str) -> str:
     # These issues mostly show up with separators for multi-class shares (like BRK A and B)
     return re.sub(r'[\.\s/]', '', symbol)
 
 
 def _activityAffectsSymbol(activity: Activity, symbol: str) -> bool:
-    normalized = normalizeSymbol(symbol)
+    normalized = _normalizeSymbol(symbol)
 
     if isinstance(activity, DividendPayment):
-        return normalizeSymbol(activity.stock.symbol) == normalized
+        return _normalizeSymbol(activity.stock.symbol) == normalized
     elif isinstance(activity, Trade):
-        return (isinstance(activity.instrument, Option) and normalizeSymbol(
-            activity.instrument.underlying) == normalized) or normalizeSymbol(
+        return (isinstance(activity.instrument, Option) and _normalizeSymbol(
+            activity.instrument.underlying) == normalized) or _normalizeSymbol(
                 activity.instrument.symbol) == normalized
     else:
         return False

bankroll/brokers/__init__.py

@@ -0,0 +1,4 @@
+from . import fidelity
+from . import ibkr
+from . import schwab
+from . import vanguard

fidelity.py → bankroll/brokers/fidelity.py

@@ -1,9 +1,9 @@
-from csvsectionslicer import parseSectionsForCSV, CSVSectionCriterion, CSVSectionResult
 from datetime import date, datetime
 from decimal import Decimal
 from enum import IntEnum, unique
-from model import Activity, Cash, Currency, Instrument, Stock, Bond, Option, OptionType, Position, DividendPayment, Trade, TradeFlags
-from parsetools import lenientParse
+from bankroll.csvsectionslicer import parseSectionsForCSV, CSVSectionCriterion, CSVSectionResult
+from bankroll.model import Activity, Cash, Currency, Instrument, Stock, Bond, Option, OptionType, Position, DividendPayment, Trade, TradeFlags
+from bankroll.parsetools import lenientParse
 from pathlib import Path
 from sys import stderr
 from typing import Callable, Dict, List, NamedTuple, Optional, Set
@@ -13,7 +13,7 @@
 import re
 
 
-class FidelityPosition(NamedTuple):
+class _FidelityPosition(NamedTuple):
     symbol: str
     description: str
     quantity: str
@@ -23,11 +23,11 @@ class FidelityPosition(NamedTuple):
     costBasis: str
 
 
-InstrumentFactory = Callable[[FidelityPosition], Instrument]
+_InstrumentFactory = Callable[[_FidelityPosition], Instrument]
 
 
-def parseFidelityPosition(p: FidelityPosition,
-                          instrumentFactory: InstrumentFactory) -> Position:
+def _parseFidelityPosition(p: _FidelityPosition,
+                           instrumentFactory: _InstrumentFactory) -> Position:
     qty = Decimal(p.quantity)
     return Position(instrument=instrumentFactory(p),
                     quantity=qty,
@@ -36,7 +36,7 @@ def parseFidelityPosition(p: FidelityPosition,
 
 
 @unique
-class FidelityMonth(IntEnum):
+class _FidelityMonth(IntEnum):
     JAN = 1,
     FEB = 2,
     MAR = 3,
@@ -51,7 +51,7 @@ class FidelityMonth(IntEnum):
     DEC = 12
 
 
-def parseOptionsPosition(description: str) -> Option:
+def _parseOptionsPosition(description: str) -> Option:
     match = re.match(
         r'^(?P<putCall>CALL|PUT) \((?P<underlying>[A-Z]+)\) .+ (?P<month>[A-Z]{3}) (?P<day>\d{2}) (?P<year>\d{2}) \$(?P<strike>[0-9\.]+) \(100 SHS\)$',
         description)
@@ -64,7 +64,7 @@ def parseOptionsPosition(description: str) -> Option:
     else:
         optionType = OptionType.CALL
 
-    month = FidelityMonth[match['month']]
+    month = _FidelityMonth[match['month']]
     year = datetime.strptime(match['year'], '%y').year
 
     return Option(underlying=match['underlying'],
@@ -87,10 +87,10 @@ def parsePositions(path: Path, lenient: bool = False) -> List[Position]:
             endSectionRowMatch=["", ""],
             rowFilter=lambda r: r[0:7])
 
-        instrumentBySection: Dict[CSVSectionCriterion, InstrumentFactory] = {
+        instrumentBySection: Dict[CSVSectionCriterion, _InstrumentFactory] = {
             stocksCriterion: lambda p: Stock(p.symbol, currency=Currency.USD),
             bondsCriterion: lambda p: Bond(p.symbol, currency=Currency.USD),
-            optionsCriterion: lambda p: parseOptionsPosition(p.description),
+            optionsCriterion: lambda p: _parseOptionsPosition(p.description),
         }
 
         sections = parseSectionsForCSV(
@@ -100,14 +100,15 @@ def parsePositions(path: Path, lenient: bool = False) -> List[Position]:
 
         for sec in sections:
             for r in sec.rows:
-                pos = parseFidelityPosition(FidelityPosition._make(r),
-                                            instrumentBySection[sec.criterion])
+                pos = _parseFidelityPosition(
+                    _FidelityPosition._make(r),
+                    instrumentBySection[sec.criterion])
                 positions.append(pos)
 
         return positions
 
 
-class FidelityTransaction(NamedTuple):
+class _FidelityTransaction(NamedTuple):
     date: str
     account: str
     action: str
@@ -127,7 +128,7 @@ class FidelityTransaction(NamedTuple):
     settlementDate: str
 
 
-def parseOptionTransaction(symbol: str, currency: Currency) -> Option:
+def _parseOptionTransaction(symbol: str, currency: Currency) -> Option:
     match = re.match(
         r'^-(?P<underlying>[A-Z]+)(?P<date>\d{6})(?P<putCall>C|P)(?P<strike>[0-9\.]+)$',
         symbol)
@@ -146,9 +147,9 @@ def parseOptionTransaction(symbol: str, currency: Currency) -> Option:
                   strike=Decimal(match['strike']))
 
 
-def guessInstrumentFromSymbol(symbol: str, currency: Currency) -> Instrument:
+def _guessInstrumentFromSymbol(symbol: str, currency: Currency) -> Instrument:
     if re.search(r'[0-9]+(C|P)[0-9]+$', symbol):
-        return parseOptionTransaction(symbol, currency)
+        return _parseOptionTransaction(symbol, currency)
     elif Bond.validBondSymbol(symbol):
         return Bond(symbol, currency=currency)
     else:
@@ -159,8 +160,8 @@ def _parseFidelityTransactionDate(datestr: str) -> datetime:
     return datetime.strptime(datestr, '%m/%d/%Y')
 
 
-def forceParseFidelityTransaction(t: FidelityTransaction,
-                                  flags: TradeFlags) -> Trade:
+def _forceParseFidelityTransaction(t: _FidelityTransaction,
+                                   flags: TradeFlags) -> Trade:
     quantity = Decimal(t.quantity)
 
     totalFees = Decimal(0)
@@ -176,14 +177,14 @@ def forceParseFidelityTransaction(t: FidelityTransaction,
 
     currency = Currency(t.currency)
     return Trade(date=_parseFidelityTransactionDate(t.date),
-                 instrument=guessInstrumentFromSymbol(t.symbol, currency),
+                 instrument=_guessInstrumentFromSymbol(t.symbol, currency),
                  quantity=quantity,
                  amount=Cash(currency=currency, quantity=amount),
                  fees=Cash(currency=currency, quantity=totalFees),
                  flags=flags)
 
 
-def parseFidelityTransaction(t: FidelityTransaction) -> Optional[Activity]:
+def _parseFidelityTransaction(t: _FidelityTransaction) -> Optional[Activity]:
     if t.action == 'DIVIDEND RECEIVED':
         return DividendPayment(date=_parseFidelityTransactionDate(t.date),
                                stock=Stock(t.symbol,
@@ -203,7 +204,7 @@ def parseFidelityTransaction(t: FidelityTransaction) -> Optional[Activity]:
     if not flags:
         return None
 
-    return forceParseFidelityTransaction(t, flags=flags)
+    return _forceParseFidelityTransaction(t, flags=flags)
 
 
 # Transactions will be ordered from newest to oldest
@@ -223,6 +224,6 @@ def parseTransactions(path: Path, lenient: bool = False) -> List[Activity]:
             filter(
                 None,
                 lenientParse(
-                    (FidelityTransaction._make(r) for r in sections[0].rows),
-                    transform=parseFidelityTransaction,
+                    (_FidelityTransaction._make(r) for r in sections[0].rows),
+                    transform=_parseFidelityTransaction,
                     lenient=lenient)))