X (nee Twitter) downloads broken: JSON metadata not found

[qralston]

Checklist

• [ x ] I'm reporting a broken site support
• [ x ] I've verified that I'm running youtube-dl version 2021.12.17
• [ x ] I've checked that all provided URLs are alive and playable in a browser
• [ x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
• [ x ] I've searched the bugtracker for similar issues including closed ones

Verbose log

$ env PYTHONPATH=/tmp/youtube-dl python3 /tmp/youtube-dl/bin/youtube-dl --verbose --restrict-filenames 'https://twitter.com/KassyAkiva/status/1854605058488291648'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--restrict-filenames', 'https://twitter.com/KassyAkiva/status/1854605058488291648']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: c5098961b
[debug] Python 3.12.7 (CPython x86_64 64bit) - Linux-6.11.5-100.fc39.x86_64-x86_64-with-glibc2.38 - OpenSSL 3.1.4 24 Oct 2023 - glibc 2.38
[debug] exe versions: ffmpeg 6.1.2, ffprobe 6.1.2
[debug] Proxy map: {}
[twitter] 1854605058488291648: Downloading guest token
[twitter] 1854605058488291648: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://github.com/ytdl-org/youtube-dl/issues , using the appropriate issue template. Make sure you are using the latest version; see  https://github.com/ytdl-org/youtube-dl/#user-content-installation  on how to update. Be sure to call youtube-dl with the --verbose option and include the complete output.
  File "/tmp/youtube-dl/youtube_dl/extractor/common.py", line 679, in _request_webpage
    return self._downloader.urlopen(url_or_request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/youtube-dl/youtube_dl/YoutubeDL.py", line 2496, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 521, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 630, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 559, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 639, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Description

X (nee Twitter) appears to have made recent changes to thwart youtube-dl from downloading media, as attempting any twitter.com URL fails (even ones that worked in the past). This is true even when using the Git master branch, currently:

commit c5098961b04ce83f4615f2a846c84f803b072639 (HEAD -> master, origin/master, origin/HEAD)
Author: dirkf <[email protected]>
Date:   Tue Aug 6 20:59:09 2024 +0100

    [Youtube] Rework n function extraction pattern
            Now also succeeds with player b12cc44b

[dirkf]

See #32392, but now that X seems still to be a thing, a full update from the yt-dlp extractor would be in orderr.