Implement more efficient way of submitting highlights

[marcus-crane]

At the moment, everything is just fired off to Readwise.

This is fine but as your Kobo highlights grow, the upload process will naturally become slower.

I was thinking that you could easily tell if a book hasn't changed (and finished books don't) by simply saving hashes for books and seeing if they've changed. We kind of need to retain what content has actually been submitted (not just number of bookmarks) since highlight positions and text annotations can change after the fact.

If they have changed then we ideally just figure out what items are new so perhaps we just hash each bookmark.

It all falls down if someone either is doing a first upload with a squillion highlights (these should be chunked to avoid rate limiting) and/or they use a new computer with a previous upload DB.

It's really cheap to make database calls but expensive (relatively speaking) to fire JSON across the web.

[marcus-crane]

The side effect of #19 relates here.

Upon submitting highlights, we need to persist the Readwise IDs to disc, probably mapped to VolumeID. Upon sending new highlights, we need to get the book from Readwise (to see if the title has updated) and use that title so we don't overwrite the users choice or send a duplicate entry.

[marcus-crane]

Oh, I dunno why it never occured to me but the sync history can just be stored on the Kobo device given it's effectively a USB 🤦 It also means that if you use October on a new computer, it can pick up where it left off instead of blinding syncing everything from scratch.

[marcus-crane]

I think this can be pushed to post v1.0.0. It's good enough for an initial release.

[oleestar]

The USB storage feature, to be able to use October on different devices, would be helpful. Now I make sure I only sync it from my desktop, but I would love to sync it also from my laptops.

Otherwise, there are usually two types of users:
a) Keep all the books on the Kobo, and therefore have a gigantic libarary of ebooks to sync
b) Keep only a few books on the Kobo, which are being read - so the library is small

I used to do a), but I found if I keep only a few books on the Kobo, I read more, and finish them faster, as there is more focus to read.