Handle GitHub changes to fetch latest `yt-dlp` #375

richardsondev · 2022-06-09T07:03:48Z

Issue

GitHub has removed the response body when redirecting a user during release downloads which has broken the yt-dlp runtime download. This is causing first time executions of the application to not download yt-dlp and return "Error! Binaries missing/corrupted" in the UI.

Background

The approach to grab the latest version of yt-dlp was relying on GitHub to return the redirect URL in the body response when fetching the latest binary. GitHub has changed to no longer return the URL in the response body and is only emitting it in the Location header now.

This PR

This PR is changing the logic to read the version from the Location header during the redirect.

In my testing the application now correctly detects the latest version number for yt-dlp and downloads it correctly.

After screenshot

ytdlVersion file

{"version":"2022.05.18","ytdlp":true}

Tests

olegshulyakov · 2022-06-18T12:50:34Z

@richardsondev Please also update modules/Filepaths.js set binary names like following:

case "darwin":
  this.ytdl = this.app.isPackaged ? path.join(this.unpackedPrefix, "binaries/yt-dlp_macos") : "binaries/yt-dlp_macos";
  ...
  break;
case "linux":
  this.ytdl = this.app.isPackaged ? path.join(this.persistentPath, "yt-dlp") : "binaries/yt-dlp";
  ...
  break;

StefanLobbenmeier · 2022-06-29T22:11:52Z

@richardsondev Please also update modules/Filepaths.js set binary names like following:

That would also require a change to getRemoteVersion. I agree that we should also get yt-dlp_macos and yt-dlp_macos_legacy now and I can make a pull request to also fix that, but until then we should merge this as soon as possible since it resolves so many issues.

StefanLobbenmeier · 2022-06-29T22:22:13Z

I can confirm that this fixes the issue 😄 LGTM

keli

I just tested this pr on Linux and can confirm it works.

42Willow · 2022-09-08T09:32:29Z

How can I download this? I need to download a youtube video by tomorrow because I need to be able to view it offline...

StefanLobbenmeier · 2022-09-08T11:40:45Z

@42Willow the quickest solution is to use https://github.com/jely2002/youtube-dl-gui/releases/tag/v2.3.1 instead, that version still packages youtube_dl.

Altnernatively I merged this PR in my own fork and made a release for macOS and Linux. Unfortunately the Windows Build is broken: https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/tag/v2.4.2

jrowberg · 2022-09-08T14:00:38Z

How can I download this? I need to download a youtube video by tomorrow because I need to be able to view it offline...

In the meantime, you can also download the yt-dlp binary manually and put it where it's supposed to be; I just did this successfully on my Windows installation. You can find yt-dlp here:

https://github.com/yt-dlp/yt-dlp/releases

On Windows, the yt-dlp.exe file should be placed in C:\Users\[YOURUSER]\AppData\Local\Programs\youtube-dl-gui\resources\app.asar.unpacked\binaries, assuming a standard installation.

claudiusraphael · 2022-09-08T20:02:54Z

Dunno if this is the right place, but stumbled over the same error in Windows 11 and therefore went to build this branch (richardson:handlegithubchangesytdlp) via npm under Node Current (v18.8.0):

PS C:\Users\claudiusraphael\Downloads\youtube-dl-gui-handlegithubchangesytdlp\youtube-dl-gui-handlegithubchangesytdlp> npm install
npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm WARN deprecated [email protected]: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
npm WARN deprecated [email protected]: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.

added 693 packages, and audited 694 packages in 1m

53 packages are looking for funding
  run `npm fund` for details

8 vulnerabilities (7 moderate, 1 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force`

Run `npm audit` for details.
npm notice
npm notice New minor version of npm available! 8.18.0 -> 8.19.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.19.1
npm notice Run npm install -g [email protected] to update!
npm notice
PS C:\Users\claudiusraphael\Downloads\youtube-dl-gui-handlegithubchangesytdlp\youtube-dl-gui-handlegithubchangesytdlp> npx electron-builder --win
  • electron-builder  version=22.9.1 os=10.0.22000
  • loaded configuration  file=package.json ("build" field)
  • writing effective config  file=dist\builder-effective-config.yaml
  • packaging       platform=win32 arch=x64 electron=11.5.0 appOutDir=dist\win-unpacked
  • downloading     url=https://github.com/electron/electron/releases/download/v11.5.0/electron-v11.5.0-win32-x64.zip size=78 MB parts=8
  • downloaded      url=https://github.com/electron/electron/releases/download/v11.5.0/electron-v11.5.0-win32-x64.zip duration=11.77s
  • downloading     url=https://github.com/electron-userland/electron-builder-binaries/releases/download/winCodeSign-2.6.0/winCodeSign-2.6.0.7z size=5.6 MB parts=1
  • downloaded      url=https://github.com/electron-userland/electron-builder-binaries/releases/download/winCodeSign-2.6.0/winCodeSign-2.6.0.7z duration=3.465s
  • building        target=nsis file=dist\Open Video Downloader Setup 2.4.0.exe archs=x64 oneClick=true perMachine=false
  • downloading     url=https://github.com/electron-userland/electron-builder-binaries/releases/download/nsis-resources-3.4.1/nsis-resources-3.4.1.7z size=731 kB parts=1
  • downloaded      url=https://github.com/electron-userland/electron-builder-binaries/releases/download/nsis-resources-3.4.1/nsis-resources-3.4.1.7z duration=1.487s
  • downloading     url=https://github.com/electron-userland/electron-builder-binaries/releases/download/nsis-3.0.4.1/nsis-3.0.4.1.7z size=1.3 MB parts=1
  • downloaded      url=https://github.com/electron-userland/electron-builder-binaries/releases/download/nsis-3.0.4.1/nsis-3.0.4.1.7z duration=1.683s
  • building block map  blockMapFile=dist\Open Video Downloader Setup 2.4.0.exe.blockmap

Build went fine and can confirm that the error doesn't appear anymore and that downloading videos works as it should.

If this comment should be applied as a "Review" instead, let me know - i wasn't sure about that.

StefanLobbenmeier · 2022-09-08T21:32:59Z

@claudiusraphael no worries, this repo is anarchy now anyway 😄

Good to know that the windows build still works locally, the CI build has some strange powershell in it that fails:

Run echo  > env.b64
  echo  > env.b64
  certutil -decode env.b64 .env
  del env.b64
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
Write-Output: D:\a\_temp\b171[2](https://github.com/StefanLobbenmeier/youtube-dl-gui/runs/7121445209?check_suite_focus=true#step:5:2)3ec-0f5e-40cd-a6e9-03771331ce46.ps1:2
Line |
   2 |  echo  > env.b64
     |  ~~~~~~~~~~~~~~~
     | Cannot process command because of one or more missing mandatory parameters: InputObject.

Error: Process completed with exit code 1.

So maybe we can also fix the CI build. Is there actually anyone interested continuing dev in an active fork since @jely2002 does not seem to respond anymore? I could offer mine, but I can only review PRs and not develop myself

StefanLobbenmeier · 2022-09-08T22:15:47Z

Inspired by that it is actually supposed to be able to build on windows I fixed the CI (removed anything related to Sentry) and made a new release here: https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/tag/v2.4.3

Feel free to link that in future issues 😄

claudiusraphael · 2022-09-12T03:50:59Z

Inspired by that it is actually supposed to be able to build on windows I fixed the CI (removed anything related to Sentry) and made a new release here: https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/tag/v2.4.3

Feel free to link that in future issues 😄

Just to confirm, tried the installer (for Windows: Open-Video-Downloader-Setup-2.4.3.exe ) and it works as expected under Windows 11 with latest Updates!

[EDIT] P.s.: Can also confirm AppImage (Open-Video-Downloader-2.4.3.AppImage ) under Fedora 35 Workstation + Ubuntu Desktop 22.04 and 20.04 and DMG (Open-Video-Downloader-2.4.3.dmg ) under Monterey with latest Updates (tested on MBP 13" Late 2015) , work as well!

claudiusraphael · 2022-09-12T04:03:43Z

@claudiusraphael no worries, this repo is anarchy now anyway 😄

Good to know that the windows build still works locally, the CI build has some strange powershell in it that fails:
Run echo  > env.b64
  echo  > env.b64
  certutil -decode env.b64 .env
  del env.b64
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
Write-Output: D:\a\_temp\b171[2](https://github.com/StefanLobbenmeier/youtube-dl-gui/runs/7121445209?check_suite_focus=true#step:5:2)3ec-0f5e-40cd-a6e9-03771331ce46.ps1:2
Line |
   2 |  echo  > env.b64
     |  ~~~~~~~~~~~~~~~
     | Cannot process command because of one or more missing mandatory parameters: InputObject.

Error: Process completed with exit code 1.
So maybe we can also fix the CI build. Is there actually anyone interested continuing dev in an active fork since @jely2002 does not seem to respond anymore? I could offer mine, but I can only review PRs and not develop myself

If you find a minute would you be so kind to post an excerpt/diff here of the changes made? I think it might be helpful to understand the details (for Learners), especially regarding PowerShell. Thx!

StefanLobbenmeier · 2022-09-12T05:57:31Z

If you find a minute would you be so kind to post an excerpt/diff here of the changes made? I think it might be helpful to understand the details (for Learners), especially regarding PowerShell. Thx!

no problem - you can find all my changes here: https://github.com/StefanLobbenmeier/youtube-dl-gui/commits/master

I had to remove sentry from the Linux build and the windows build here: StefanLobbenmeier@eba0da1

The reason those sentry related lines are in there are because this is a web service for error reporting, where I would need the secret of the owner to access his project. I could also create a new sentry project and use my own secret there, but I was not able to get that working. For now removing it is the easier solution.

jely2002 · 2022-09-12T06:17:39Z

Hi folks, will take a look at some open PR's in a couple hours. I need to test them before I can confidently release a new version and that takes time as it has been a while.

claudiusraphael · 2022-10-30T14:01:27Z

Hi folks, will take a look at some open PR's in a couple hours. I need to test them before I can confidently release a new version and that takes time as it has been a while.

I don't know what the actual status is, but if you need a tester for a new release on Windows 11 Pro, Windows 10 LTSC Enterprise, Macos Monterey, Fedora Workstation 36, Ubuntu 22.04, let me know.

If of help provide me detailed steps for debugging/creating reports.

To inform/ping me you can do so on discord: claudiusraphael#2172

odhiambo · 2022-12-01T14:53:07Z

Version 2.4.3 is running fine on my W11Pro.

StefanLobbenmeier · 2022-12-02T08:44:13Z

Just to avoid confusion @odhiambo you are talking about a release from https://github.com/StefanLobbenmeier/youtube-dl-gui/releases/? There is no 2.4.3 in https://github.com/jely2002/youtube-dl-gui/releases

odhiambo · 2022-12-02T09:22:28Z

Sorry for the confusion, but I was referring to the one created by @StefanLobbenmeier

MrFobwatch · 2023-05-25T02:42:31Z

So is there going to be active development on this repo anymore? I am using @StefanLobbenmeier 's fork now on my machine but was trying to figure out if I should put in a pull request to try and redirect the homebrew cask to the new repo as it is currently pulling the broken version from this repo.

Handle GitHub changes to fetch latest yt-dlp

5a61e7c

This was referenced Jun 9, 2022

"Please restart the app and disable antivirus" error #362

Open

"Error! Binaries missing/corrupted." #354

Open

olegshulyakov approved these changes Jun 18, 2022

View reviewed changes

ghost mentioned this pull request Jun 18, 2022

#243 Fix Error! Unhandled error (execa) #385

Open

4 tasks

StefanLobbenmeier mentioned this pull request Jun 29, 2022

Error: Binaries missing/corrupted #390

Open

StefanLobbenmeier mentioned this pull request Jun 29, 2022

Error! Unhandled error (execa). // Fedora Linux #389

Open

StefanLobbenmeier approved these changes Jun 29, 2022

View reviewed changes

This was referenced Sep 4, 2022

not working, error occur #415

Open

not working. showing error, binaries missing/corrupted #413

Open

keli approved these changes Sep 6, 2022

View reviewed changes

btsmartx approved these changes Sep 7, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle GitHub changes to fetch latest `yt-dlp` #375

Handle GitHub changes to fetch latest `yt-dlp` #375

richardsondev commented Jun 9, 2022 •

edited

olegshulyakov commented Jun 18, 2022

StefanLobbenmeier commented Jun 29, 2022 •

edited

StefanLobbenmeier commented Jun 29, 2022

keli left a comment

42Willow commented Sep 8, 2022 •

edited

StefanLobbenmeier commented Sep 8, 2022 •

edited

jrowberg commented Sep 8, 2022

claudiusraphael commented Sep 8, 2022 •

edited

StefanLobbenmeier commented Sep 8, 2022

StefanLobbenmeier commented Sep 8, 2022

claudiusraphael commented Sep 12, 2022 •

edited

claudiusraphael commented Sep 12, 2022

StefanLobbenmeier commented Sep 12, 2022

jely2002 commented Sep 12, 2022

claudiusraphael commented Oct 30, 2022

odhiambo commented Dec 1, 2022

StefanLobbenmeier commented Dec 2, 2022 •

edited

odhiambo commented Dec 2, 2022

MrFobwatch commented May 25, 2023

Handle GitHub changes to fetch latest yt-dlp #375

Are you sure you want to change the base?

Handle GitHub changes to fetch latest yt-dlp #375

Conversation

richardsondev commented Jun 9, 2022 • edited

Issue

Background

This PR

After screenshot

ytdlVersion file

Tests

olegshulyakov commented Jun 18, 2022

StefanLobbenmeier commented Jun 29, 2022 • edited

StefanLobbenmeier commented Jun 29, 2022

keli left a comment

Choose a reason for hiding this comment

42Willow commented Sep 8, 2022 • edited

StefanLobbenmeier commented Sep 8, 2022 • edited

jrowberg commented Sep 8, 2022

claudiusraphael commented Sep 8, 2022 • edited

StefanLobbenmeier commented Sep 8, 2022

StefanLobbenmeier commented Sep 8, 2022

claudiusraphael commented Sep 12, 2022 • edited

claudiusraphael commented Sep 12, 2022

StefanLobbenmeier commented Sep 12, 2022

jely2002 commented Sep 12, 2022

claudiusraphael commented Oct 30, 2022

odhiambo commented Dec 1, 2022

StefanLobbenmeier commented Dec 2, 2022 • edited

odhiambo commented Dec 2, 2022

MrFobwatch commented May 25, 2023

Handle GitHub changes to fetch latest `yt-dlp` #375

Handle GitHub changes to fetch latest `yt-dlp` #375

richardsondev commented Jun 9, 2022 •

edited

StefanLobbenmeier commented Jun 29, 2022 •

edited

42Willow commented Sep 8, 2022 •

edited

StefanLobbenmeier commented Sep 8, 2022 •

edited

claudiusraphael commented Sep 8, 2022 •

edited

claudiusraphael commented Sep 12, 2022 •

edited

StefanLobbenmeier commented Dec 2, 2022 •

edited