v0.9.5

Allow rate limit "burst" for TMDB API when using the default API key, allowing the limit to average to 1 second with a lot less blocking, which became more important as the default queue concurrency was reduced to 1.

What's Changed

• Increase rate limit burst when using default TMDB API key by @mgdigital in #296

Full Changelog: v0.9.4...v0.9.5

v0.9.4

This release includes 2 optimizations that should reduce the load that queue jobs place on the database, including for the 0.9.0 upgrade job.

It should also reduce instances of ERROR: canceling statement due to user request in the Postgres logs, but note that Bitmagnet does cancel some database queries because it sometimes tries 2 query strategies in a race and cancels the slowest one, so this message doesn't indicate a problem.

What's Changed

• Add local_search_enabled flag to classifier by @mgdigital in #291
• Reduce CTE strategy usage when not needed by @mgdigital in #292

Full Changelog: v0.9.3...v0.9.4

v0.9.3

This release reduces the default processor.concurrency config to 1 following user feedback that concurrency of 3 causes slowdowns on some systems. If your system can handle it, you may want to configure a higher value.

What's Changed

• Default processor.concurrency to 1 by @mgdigital in #284

Full Changelog: v0.9.2...v0.9.3

v0.9.2

Some users have reported a slowdown in the app that is probably due to the 0.9.0 reindex. Adding a configuration option to dial down the queue concurrency for torrent processing.

If you're affected, try setting the value to 1, either as an environment variable, e.g. PROCESSOR_CONCURRENCY=1, or in your config file:

processor:
  concurrency: 1

What's Changed

• Allow configuring processor concurrency by @mgdigital in #281

Full Changelog: v0.9.1...v0.9.2

v0.9.1

This release includes an optimization may improve reprocess performance on more modest hardware.

What's Changed

• Processor optimisation - reduce concurrency by @mgdigital in #277
• Bump github.com/vektah/gqlparser/v2 from 2.5.11 to 2.5.14 by @dependabot in #274
• Processor optimization by @mgdigital in #278

Full Changelog: v0.9.0...v0.9.1

v0.9.0

• The main improvement for v0.9.0 is ordering of results in the web UI and GraphQL API
• An --apisDisabled flag was added to the reprocess command to allow significantly faster reprocessing/classifying without any API calls (i.e. TMDB), without disabling this globally.
• Optimisations were made to the web UI that significantly improve responsiveness; the default pagination limit has therefore been increased to 20
• Reprocessing is now done fully within the queue (so you'll no longer see a progress bar when running bitmagnet reprocess, this is handled by the new process_torrent_batch task)

IMPORTANT: Some refactoring of the database was needed to allow for performant ordering. On first run, a database migration will be applied that should take no longer than a few minutes, and tasks will be added to the queue to reindex all torrents. The reindex tasks must complete before ordering will work correctly; if you have millions of torrents in your index this may take a day or more, but is significantly faster in this release than prior versions. Please ensure you have no pending items in your queue (at the /metrics endpoint) before opening any issues about incorrect ordering.

What's Changed

• Use fontsource packages for fonts to ensure deterministic build by @mgdigital in #257
• More Filters against cp. by @Ewoodss in #254
• Ordering by @mgdigital in #260
• Bump rexml from 3.2.6 to 3.2.8 in /bitmagnet.io by @dependabot in #259
• Split gen task to individual tasks by @mgdigital in #261
• Parallel search strategies by @mgdigital in #264
• Classifier runtime flags by @mgdigital in #265
• Bump nokogiri from 1.16.4 to 1.16.5 in /bitmagnet.io by @dependabot in #256
• ENH: Classification result protobuf includes content id and source by @rraymondgh in #266
• Add batch reprocessing queue job and 0.9.0 upgrade hook by @mgdigital in #267
• Increase slow threshold by @mgdigital in #268
• Don't track imported info hashes by @mgdigital in #269
• Add sponsor links to web UI by @mgdigital in #270
• Website updates by @mgdigital in #271
• Tweak build flags by @mgdigital in #272

New Contributors

• @Ewoodss made their first contribution in #254
• @rraymondgh made their first contribution in #266

Full Changelog: v0.8.0...v0.9.0

v0.8.0

Classifier reimplementation

The classifier has been re-implemented and now uses a DSL allowing for full customisation. Several bugs have also been fixed, and a preparatory database migration for search ordering has been added.

The classifier documentation can be found at https://bitmagnet.io/tutorials/classifier.html.

Build reimplementation

The build has been re-implemented using https://github.com/goreleaser/goreleaser, thanks to @KyleSanderson. A number of new platform-specific build artifacts are now available.

Seeders/Leechers bug

There has somehow been a bug present, since day 1, whereby seeders and leechers were being swapped by the DHT crawler! This is fixed in 0.8.x. The first beta included a migration that would fix existing data, but the query just takes too long (for larger indexes), and so it has been removed from the migration.

Until now, no-one noticed - but If you care and would like to fix this in your index, you can run the following query, replacing the time with the time at which you did the 0.8.x upgrade. Be warned, this query may take a while for larger indexes! In time, much of the data will naturally fix itself anyway as the S/L numbers get re-scraped by the crawler when the torrent is re-discovered.

update torrents_torrent_sources set seeders = leechers, leechers = seeders where source = 'dht' and  updated_at < '2024-04-30 00:00:00+00';

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240
• Fix slow migration by @mgdigital in #241
• Fix slow migration by @mgdigital in #242
• Keep a default placeholder value for published_at to avoid a breaking change by @mgdigital in #243
• Add an order by updated_at column, as there may be many records with the same published_at by @mgdigital in #244
• Add orphans option to reprocess command by @mgdigital in #245
• Use budgeted count for reprocess by @mgdigital in #246
• Bump golang.org/x/net from 0.20.0 to 0.23.0 by @dependabot in #239
• Bump express from 4.18.2 to 4.19.2 in /webui by @dependabot in #236
• Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /webui by @dependabot in #231
• Bump follow-redirects from 1.15.5 to 1.15.6 in /webui by @dependabot in #221
• Bump github.com/jackc/pgx/v5 from 5.5.2 to 5.5.4 by @dependabot in #218
• Add Nix shell and refactor actions by @mgdigital in #249
• Use goreleaser for build process (#212) by @mgdigital in #216
• Upgrade dependencies by @mgdigital in #250
• Don't run GHCR action on main by @mgdigital in #251
• Fix docsite plugin by @mgdigital in #252

New Contributors

• @josteink made their first contribution in #235
• @KyleSanderson made their first contribution in #212

Full Changelog: v0.7.14...v0.8.0

v0.8.0-beta.7

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240
• Fix slow migration by @mgdigital in #241
• Fix slow migration by @mgdigital in #242
• Keep a default placeholder value for published_at to avoid a breaking change by @mgdigital in #243
• Add an order by updated_at column, as there may be many records with the same published_at by @mgdigital in #244
• Add orphans option to reprocess command by @mgdigital in #245
• Use budgeted count for reprocess by @mgdigital in #246
• Bump golang.org/x/net from 0.20.0 to 0.23.0 by @dependabot in #239
• Bump express from 4.18.2 to 4.19.2 in /webui by @dependabot in #236
• Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /webui by @dependabot in #231
• Bump follow-redirects from 1.15.5 to 1.15.6 in /webui by @dependabot in #221
• Bump github.com/jackc/pgx/v5 from 5.5.2 to 5.5.4 by @dependabot in #218
• Add Nix shell and refactor actions by @mgdigital in #249
• Use goreleaser for build process (#212) by @mgdigital in #216
• Upgrade dependencies by @mgdigital in #250

New Contributors

• @josteink made their first contribution in #235

Full Changelog: v0.7.14...v0.8.0-beta.7

v0.8.0-beta.6

What's Changed

• Revamp build process by @KyleSanderson in #212

New Contributors

• @KyleSanderson made their first contribution in #212

Full Changelog: v0.7.14...v0.8.0-beta.6

v0.8.0-beta.5

What's Changed

• Revamp build process by @KyleSanderson in #212

New Contributors

• @KyleSanderson made their first contribution in #212

Full Changelog: v0.7.14...v0.8.0-beta.5