v2.3.1

Tip

Now is the best time to switch back to the :latest image tags if you are on :main for testing.

What's Changed

• Update TwitchDownloader version to fix new VOD ids being greater than what int32 supports lay295/TwitchDownloader#1057

  • You may need to restart the ArchiveTwitchChatWorkflow for any failed archives. Restarting ArchiveTwitchChatWorkflow will restart the entire chat archive flow (download, render, and move). To restart, click the failed workflow then click the "restart" button.
  • 

• fix : #416 oidc by @boringwolf in #417

New Contributors

• @boringwolf made their first contribution in #417

Full Changelog: v2.3.0...v2.3.1

v2.3.0

Tip

Now is the best time to switch back to the :latest image tags if you are on :main for testing.

New Features

Muted Segments

Going forward, all video (vod) downloads will also save the muted segments. These are saved to the database in the table muted_segments. They are also saved to the info.json file. These are not displayed in the frontend yet, I'm still trying to find a good way to display the segments. For now you can use the API or query the database.

/api/v1/vod/<id>?with_channel=true&with_chapters=true&with_muted_segments=true

Title Regex

Watched channels now have a new option called "title regex". Custom regex can be applied to either live streams or video downloads. This allows you to customize what is downloaded by applying regex to the title. The obvious use case is to ignore reruns as the title usually contains it, but it's all customizable to fit your needs.

To get started, edit a "watched channel" and click the plus button under Advanced > Title Regex

A new entry will be visible allowing you to enter a regex and apply some optional settings.

• negative: Invert the match, meaning you do not want the regex
• apply to video downloads: Apply the regex to video downloads. By default the regex is only applied to live streams. If this is checked then it will not be used for live streams. You need to make two regexes if you want to apply to both live stream and videos.

A title regex to not download any rerun livestreams would look like the following

For more information please see the wiki page https://github.com/Zibbp/ganymede/wiki/Watched-Channel-Title-Regex. If you have any useful regexes, please share in a discussion!

What's Changed

• feat: save muted segments by @Zibbp in #406
• feat: title regex filter by @Zibbp in #409
• build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0 by @dependabot in #410
• build: package updates by @Zibbp in #411
• build(deps): bump docker/setup-buildx-action from 3.2.0 to 3.3.0 by @dependabot in #403

Full Changelog: v2.2.0...v2.3.0

v2.2.0

What's Changed

• The Twitch API is slow to update when a stream goes offline, this is especially noticeable with a low "live check interval". #385 adds some logic to detect if a stream gets archived when it's truly offline, but the API hasn't updated yet. This includes stopping the queue/workflows and deleting the video entry.
• Add a "max video age" to watched channels when archiving previous VODs. VODs are only archived if they are less than X number of days old. (#395).
  • 
• Re-implement the dark/light mode theme toggle (#397).

Commits

• build(deps): bump docker/login-action from 3.0.0 to 3.1.0 by @dependabot in #393
• build(deps): bump docker/build-push-action from 5.1.0 to 5.3.0 by @dependabot in #392
• build(deps): bump docker/setup-buildx-action from 3.0.0 to 3.2.0 by @dependabot in #391
• build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 by @dependabot in #384
• build(deps): bump github.com/go-playground/validator/v10 from 10.16.0 to 10.19.0 by @dependabot in #381
• build(deps): bump github.com/google/uuid from 1.5.0 to 1.6.0 by @dependabot in #375
• build(deps): bump github.com/rs/zerolog from 1.31.0 to 1.32.0 by @dependabot in #374
• Delete archive if stream is ended but Twitch API is slow to update by @Zibbp in #385

Full Changelog: v2.1.0...v2.2.0

v2.1.0

Important

Temporal container changes

It's been reported that the sqlite version of the Temporal container that was previously shipped with Ganymede caused issues, see #339 and #353 for more information. To fix this, I'm now recommending everyone to switch to the more "production-like" temporal setup, which uses a database. To perform this change, follow the below.

No workflow data will be saved from the previous container, so ensure that nothing is actively being archived

1. Bring down Ganymede docker compose down
2. Perform the following change to your docker-compose.yml file. The new Temporal container requires access to your database, make sure to update the POSTGRES_USER and POSTGRES_PWD with the username and password of your ganymede-db container.

-  ganymede-temporal:
-    container_name: ganymede-temporal
-    image: ghcr.io/zibbp/ganymede-temporal:latest
-    restart: unless-stopped
-    volumes:
-      - ./temporal:/data
-    ports:
-      - 7233:7233
-      - 8233:8233 # web ui - optional
+  ganymede-temporal:
+    image: temporalio/auto-setup:1
+    container_name: ganymede-temporal
+    depends_on:
+      - ganymede-db
+    environment:
+      - DB=postgresql # this tells temporal to use postgres (not the db name)
+      - DB_PORT=5432
+      - POSTGRES_USER=ganymede
+      - POSTGRES_PWD=PASSWORD
+      - POSTGRES_SEEDS=ganymede-db # name of the db service
+    ports:
+      - 7233:7233

3. Optionally add the web-ui service if you wish to use that.

+  ganymede-temporal-ui:
+    image: temporalio/ui:latest
+    container_name: ganymede-temporal-ui
+    depends_on:
+      - ganymede-temporal
+    environment:
+      - TEMPORAL_ADDRESS=ganymede-temporal:7233
+    ports:
+      - 8233:8080

4. Delete the temporal directory
5. Bring everything back up docker compose up -d
6. Ensure the temporal container is working fine docker logs ganymede-temporal and check that the API container is able to connect docker logs ganymede-api

Refer to the docker-compose.yml file for a full example.

What's Changed

If you've been running the :main image for either the API or frontend, now is a good time to switch back to the :latest tag.

Frontend

• Bump package versions
• Place theater mode button next to full screen button
• Show thumbnail when video hasn't begun playing yet
• Add sharp and ensure /app directory is writable
• Fix bug the prevented the VOD title component from coming back after exiting theater mode

API

• Use debian as the x86 docker base image
• Re-run live thumbnail download 10 minutes after the archive starts

Lots of bug fixes and package version bumps:

• feat(workflow): update live stream archives with correct vod ids by @Zibbp in #351
• Fixes 354 by @Zibbp in #357
• Fixes 2024 02 04 by @Zibbp in #361
• fix(exec): check if livechatworkflowid is populated by @Zibbp in #364
• fix: add comments by @Zibbp in #365
• feat(workflow): re-download live thumbnail again after 10 minutes by @Zibbp in #372
• build(deps): bump golangci/golangci-lint-action from 3.7.0 to 4.0.0 by @dependabot in #368
• build(deps): bump golang.org/x/crypto from 0.17.0 to 0.19.0 by @dependabot in #367
• build(deps): bump docker/metadata-action from 5.0.0 to 5.5.1 by @dependabot in #363
• build(deps): bump github.com/mattn/go-sqlite3 from 1.14.17 to 1.14.22 by @dependabot in #362
• build(deps): bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 by @dependabot in #338
• build(deps): bump github.com/labstack/echo/v4 from 4.11.3 to 4.11.4 by @dependabot in #332
• build(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.17.0 by @dependabot in #366
• build(deps): bump actions/setup-go from 4 to 5 by @dependabot in #322
• build(deps): bump github/codeql-action from 2 to 3 by @dependabot in #326
• build(docker): use debian as base image by @Zibbp in #373

Full Changelog: v2.0.2...v2.1.0

v2.0.2

What's Changed

• feat(workflows): paginate executions by @Zibbp in #341
• fix(channel): allow specifying an external id when creating a channel… by @Zibbp in #344
• feat(activity/livevideo): attempt to update livestream archive with the external vod id by @Zibbp in #347
• Fix live stream chat update if the channel does not have any previous videos

Frontend

What's Changed

• feat(workflows): paginate workflows by @Zibbp in Zibbp/ganymede-frontend#19
• feat(admin/channel): allow setting external id when creating channel by @Zibbp in Zibbp/ganymede-frontend#20
• fix(chat): fallback emote tooltip text to fragment text when empty + use mantine tooltips by @Entrivax in Zibbp/ganymede-frontend#23
• fix(channel): add more elements per row on bigger screens by @Entrivax in Zibbp/ganymede-frontend#21
• fix(vod): try improve responsiveness of the vod page by @Entrivax in Zibbp/ganymede-frontend#22

New Contributors

• @Entrivax made their first contribution in Zibbp/ganymede-frontend#23

Full Changelog: v2.0.1...v2.0.2

v2.0.1

If you have not read the 2.0.0 release notes, please start there! https://github.com/Zibbp/ganymede/releases/tag/v2.0.0

v2.0.1 Changes

• add procps package to arm build
• Add retry logic to viper config parsing
• Do not run refreshConfig when worker starts
• Intelligently detect if the database needs to be seeded
• Remove unused config options

---

What's Changed

• fix(config): retry viper load by @Zibbp in #334

Full Changelog: v2.0.0...v2.0.1

v2.0.0

2.0.0

Merry Christmas everyone, I hope you enjoy this update! 🎅🎄

Breaking Change!

Important

Version 2.0.0 has a change to the docker compose that must be applied. The API container will not start until this change is applied

A few environment variables need to be added to the API compose service. As well as adding a new Temporal service.

  ganymede-api:
    container_name: ganymede-api
    image: ghcr.io/zibbp/ganymede:latest
    restart: unless-stopped
    environment:
      - TZ=America/Chicago # Set to your timezone
      - DB_HOST=ganymede-db
      - DB_PORT=5432
      - DB_USER=ganymede
      - DB_PASS=PASSWORD
      - DB_NAME=ganymede-prd
      - DB_SSL=disable
      - JWT_SECRET=SECRET
      - JWT_REFRESH_SECRET=SECRET
      - TWITCH_CLIENT_ID=
      - TWITCH_CLIENT_SECRET=
      - FRONTEND_HOST=http://IP:PORT
      # OPTIONAL
      # - OAUTH_PROVIDER_URL=
      # - OAUTH_CLIENT_ID=
      # - OAUTH_CLIENT_SECRET=
      # - OAUTH_REDIRECT_URL=http://IP:PORT/api/v1/auth/oauth/callback # Points to the API service
+      - TEMPORAL_URL=ganymede-temporal:7233
+      # WORKER
+      - MAX_CHAT_DOWNLOAD_EXECUTIONS=5
+      - MAX_CHAT_RENDER_EXECUTIONS=3
+      - MAX_VIDEO_DOWNLOAD_EXECUTIONS=5
+      - MAX_VIDEO_CONVERT_EXECUTIONS=3
    volumes:
      - /path/to/vod/storage:/vods
      - ./logs:/logs
      - ./data:/data
      # Uncomment below to persist temp files
      #- ./tmp:/tmp
    ports:
      - 4800:4000
+  ganymede-temporal:
+    container_name: ganymede-temporal
+    image: ghcr.io/zibbp/ganymede-temporal:latest
+    restart: unless-stopped
+    volumes:
+      - ./temporal:/data
+    ports:
+      - 7233:7233
+      - 8233:8233 # web ui - optional

See the full docker-compose.yml for a full example (see commit history for new additions).

Features

New Workflow System

I'm now utilizing Temporal to create and run workflows. This replaces the queue system with something that will hopefully be more robust.

There is now a new "Workflows" page that shows active and closed workflows.

The "Queue" page is still available which provides an easy-to-see status of an archive. Workflows are still in an early stage so if you'd like to see more information about the workflows I suggest exposing the "web ui" port of the Temporal container (8233) and visiting that. Manually restarting a task is now accomplished on the workflow page for that specific execution.

Most workflows now have auto retries, currently configured up to 3 retries before the workflow errors out. You can now be more strict on the maximum number of workflow types as seen in the new environment variables for the API container.

The worker is currently bundled in the API server container and they both run at once. In the future I will probably add support to break this up and allow for distributed worker nodes.

Getting the existing code moved over to Temporal workflows has been a massive lift. It will pay off over time though as it will be significantly easier to setup new workflows in the future. This change will likely introduce some bugs that I wasn't able to catch. If you encounter any issues, please create an issue with as many logs and information as possible.

Chapters / Categories

Requested in #317, Game categories are now saved to the info.json file as well as the database if the video has categories. These chapters / categories are visible in the video player.

A new workflow exists to update existing videos with available categories (if the video still exists and has categories).

A future update will allow creating, editing, and deleting chapters in the frontend.

Frontend Updates

I've given the frontend a new color scheme and updated some components, including the landing page.

Known Issues

• Live stream chat will not render if the channel does not have any videos.
  • TwitchDownloader now requires a video ID to render chat. I'm exploring possible workarounds for this.
• Workflows on the "workflows" page disappear after a day.
  • Temporal auto removes old workflows. I'm hoping to have a fix for this in the next release.
• Mobile player
  • The vertical layout for watching a video is a bit messed up right now. Rotate your device so it's in landscape and press the theater mode icon. This will play the video next to the chat player.

What's Changed

• fix(archive): remove delete vod by @Zibbp in #309
• build(deps): bump github.com/rs/zerolog from 1.30.0 to 1.31.0 by @dependabot in #303
• build(deps): bump golang.org/x/net from 0.15.0 to 0.17.0 by @dependabot in #310
• build(deps): bump github.com/go-playground/validator/v10 from 10.15.4 to 10.16.0 by @dependabot in #312
• build(deps): bump docker/build-push-action from 5.0.0 to 5.1.0 by @dependabot in #315
• build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 by @dependabot in #316
• feat: temporal queue by @Zibbp in #328
• docs: temporal docker by @Zibbp in #330

Full Changelog: v1.4.3...v2.0.0

v1.4.3

What's Changed

• build(deps): bump golang.org/x/crypto from 0.12.0 to 0.13.0 by @dependabot in #284
• build(deps): bump docker/metadata-action from 4.6.0 to 5.0.0 by @dependabot in #290
• build(deps): bump docker/setup-qemu-action from 2.2.0 to 3.0.0 by @dependabot in #291
• build(deps): bump docker/setup-buildx-action from 2.10.0 to 3.0.0 by @dependabot in #292
• build(deps): bump docker/login-action from 2.2.0 to 3.0.0 by @dependabot in #293
• build(deps): bump docker/build-push-action from 4.1.1 to 5.0.0 by @dependabot in #294
• build(deps): bump github.com/go-playground/validator/v10 from 10.15.3 to 10.15.4 by @dependabot in #296
• build(deps): bump entgo.io/ent from 0.12.3 to 0.12.4 by @dependabot in #298
• build(deps): bump golang.org/x/oauth2 from 0.11.0 to 0.12.0 by @dependabot in #285
• build(deps): bump github.com/go-co-op/gocron from 1.33.1 to 1.35.0 by @dependabot in #302
• feat: local view count by @Zibbp in #288
• fix(archive): return and delete queue if no stream data by @Zibbp in #301

Features

Local View Count

Views unique to Ganymede are now shown next to the source views.

Fixes

#301 fixes a scenario where there's a small window between various tasks that crashes the application if the stream goes offline (no data return by Twitch's API).

Full Changelog: v1.4.2...v1.4.3

v1.4.2

What's Changed

• build(deps): bump github.com/google/uuid from 1.3.0 to 1.3.1 by @dependabot in #272
• build(deps): bump golangci/golangci-lint-action from 3.6.0 to 3.7.0 by @dependabot in #275
• build(deps): bump docker/setup-buildx-action from 2.9.1 to 2.10.0 by @dependabot in #276
• build(deps): bump github.com/go-playground/validator/v10 from 10.15.0 to 10.15.3 by @dependabot in #277
• build(deps): bump github.com/swaggo/swag from 1.16.1 to 1.16.2 by @dependabot in #279
• build(deps): bump github.com/swaggo/echo-swagger from 1.4.0 to 1.4.1 by @dependabot in #280
• build(deps): bump actions/checkout from 3 to 4 by @dependabot in #281
• build(deps): bump github.com/go-co-op/gocron from 1.31.0 to 1.33.1 by @dependabot in #278
• feat: video schedule interval by @Zibbp in #283

Features

feat: video schedule interval by @Zibbp in #283

A new config variable has been added that allows you to customize how often the video schedule schedule runs.

Full Changelog: v1.4.1...v1.4.2

v1.4.1

What's Changed

• fix(archive): catch error if thumbnail update fails by @Zibbp in #269
• build(deps): bump github.com/go-co-op/gocron from 1.30.1 to 1.31.0 by @dependabot in #260
• build(deps): bump golang.org/x/crypto from 0.11.0 to 0.12.0 by @dependabot in #265
• build(deps): bump github.com/rs/zerolog from 1.29.1 to 1.30.0 by @dependabot in #261
• build(deps): bump github.com/go-playground/validator/v10 from 10.14.1 to 10.15.0 by @dependabot in #264
• build(deps): bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 by @dependabot in #266

Full Changelog: v1.4.0...v1.4.1