Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Very slow download process on low end machine #2059

Open
2Friendly4You opened this issue Apr 2, 2024 · 2 comments
Open

Very slow download process on low end machine #2059

2Friendly4You opened this issue Apr 2, 2024 · 2 comments
Labels
Bug Unexpected problem or unintended behavior that needs to be fixed

Comments

@2Friendly4You
Copy link

2Friendly4You commented Apr 2, 2024 •
edited

System OS

Linux

Python Version

3.10 (CPython)

Install Source

pip / PyPi

Install version / commit hash

v4.2.5

Expected Behavior vs Actual Behavior

The entire download process takes about 350-400 seconds (about 6 minutes) for a single song (this does not change significantly with a different number of threads).
It is a low-end machine, but the CPU utilisation of Spotdl is only 5%, while the whole system uses about 10% of the CPU.
The RAM utilisation is 5 GB out of 8 GB, so it is not critical either.

Is there a problem/ performance issue/ bottleneck on my side or is spotdl not optimised for low-end machines?

Other (normal) programmes such as nginx and Python applications that use Flask, for example, do not run slowly.
The internet connection is fast enough, as it takes around 12 seconds to download a single song on another PC.

Steps to reproduce - Ensure to include actual links!

  1. Get a low end machine
  2. execute spotdl --log-level DEBUG --threads 8 "Test drive"
  3. Wait a long time

Traceback

webserver@webserver:~$ spotdl --log-level DEBUG --threads 8 "Test drive"
[20:39:10] DEBUG    MainThread - Downloader settings: {'audio_providers': ['youtube-music'], 'lyrics_providers':    downloader.py:131                    ['genius', 'azlyrics', 'musixmatch'], 'playlist_numbering': False, 'scan_for_songs': False,
                    'm3u': None, 'output': '{artists} - {title}.{output-ext}', 'overwrite': 'skip', 'search_query':
                    None, 'ffmpeg': 'ffmpeg', 'bitrate': None, 'ffmpeg_args': None, 'format': 'mp3', 'save_file':                                        None, 'filter_results': True, 'album_type': None, 'threads': 8, 'cookie_file': None,
                    'restrict': None, 'print_errors': False, 'sponsor_block': False, 'preload': False, 'archive':
                    None, 'load_config': True, 'log_level': 'DEBUG', 'simple_tui': False, 'fetch_albums': False,
                    'id3_separator': '/', 'ytm_data': False, 'add_unavailable': False, 'generate_lrc': False,
                    'force_update_metadata': False, 'only_verified_results': False, 'sync_without_deleting': False,
                    'max_filename_length': None, 'yt_dlp_args': None, 'detect_formats': None, 'save_errors': None,
                    'ignore_albums': None, 'proxy': None, 'skip_explicit': False, 'log_format': None, 'redownload':
                    False, 'skip_album_art': False, 'create_skip_file': False, 'respect_skip_file': False}
[20:39:10] DEBUG    MainThread - FFmpeg path: ffmpeg                                                                downloader.py:149
[20:39:10] DEBUG    MainThread - Found 0 known songs                                                                downloader.py:184
[20:39:11] DEBUG    MainThread - Archive: 0 urls                                                                    downloader.py:231
[20:39:11] DEBUG    MainThread - Downloader initialized                                                             downloader.py:233
[20:39:11] INFO     MainThread - Processing query: Test drive                                                           search.py:137
[20:39:17] DEBUG    MainThread - Found 1 songs in 0 lists                                                               search.py:339
[20:39:17] DEBUG    MainThread - Downloading 1 songs                                                                downloader.py:280
[20:39:38] DEBUG    asyncio_0 - Genius failed to find lyrics for John Powell - Test Drive - From How To Train Your  downloader.py:406
                    Dragon Music From The Motion Picture
[20:39:38] DEBUG    asyncio_0 - AzLyrics failed to find lyrics for John Powell - Test Drive - From How To Train     downloader.py:406
                    Your Dragon Music From The Motion Picture
[20:39:39] DEBUG    asyncio_0 - MusixMatch failed to find lyrics for John Powell - Test Drive - From How To Train   downloader.py:406
                    Your Dragon Music From The Motion Picture
[20:39:39] DEBUG    asyncio_0 - No lyrics found for John Powell - Test Drive - From How To Train Your Dragon Music  downloader.py:566
                    From The Motion Picture, lyrics providers: Genius, AzLyrics, MusixMatch
[20:39:39] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Searching for john powell - test drive - from how to train your  base.py:167
                    dragon music from the motion picture
[20:41:40] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Found 1 results for ISRC US3M51001211                            base.py:180
[20:41:40] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Filtered to 1 ISRC results                                       base.py:192
[20:41:40] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Best ISRC result is                                              base.py:201
                    https://music.youtube.com/watch?v=IpPIK4T068s with score 100.0
[20:41:40] DEBUG    asyncio_0 - Downloading John Powell - Test Drive - From How To Train Your Dragon Music From The downloader.py:654
                    Motion Picture using https://music.youtube.com/watch?v=IpPIK4T068s
[20:45:19] INFO     asyncio_0 - Downloaded "John Powell - Test Drive - From How To Train Your Dragon Music From The downloader.py:828
                    Motion Picture": https://music.youtube.com/watch?v=IpPIK4T068s
[20:45:19] DEBUG    MainThread - Took 367 seconds                                                                  entry_point.py:164

Other details

webserver@webserver:~$ sudo lshw -short
H/W path Device Class Description

                       system         System Product Name (SKU)

/0 bus B150M-C D3
/0/0 memory 64KiB BIOS
/0/41 memory 64KiB L1 cache
/0/42 memory 64KiB L1 cache
/0/43 memory 512KiB L2 cache
/0/44 memory 3MiB L3 cache
/0/45 processor Intel(R) Pentium(R) CPU G4400 @ 3.30GHz
/0/46 memory 8GiB System Memory
/0/46/0 memory [empty]
/0/46/1 memory 4GiB DIMM DDR3 Synchronous 1600 MHz (0,6 ns)
/0/46/2 memory [empty]
/0/46/3 memory 4GiB DIMM DDR3 Synchronous 1600 MHz (0,6 ns)
/0/100 bridge Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers
/0/100/1 bridge 6th-10th Gen Core Processor PCIe Controller (x16)
/0/100/1/0 /dev/fb0 display G72 [GeForce 7300 LE]
/0/100/14 bus 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller
/0/100/14/0 usb1 bus xHCI Host Controller
/0/100/14/1 usb2 bus xHCI Host Controller
/0/100/16 communication 100 Series/C230 Series Chipset Family MEI Controller #1
/0/100/17 scsi0 storage Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode]
/0/100/17/0 /dev/cdrom disk DVDRAM GH24NSD1
/0/100/17/1 /dev/sda disk 512GB INTENSO SSD
/0/100/17/1/1 /dev/sda1 volume 2096KiB BIOS Boot partition
/0/100/17/1/2 /dev/sda2 volume 4194MiB EXT4 volume
/0/100/17/1/3 /dev/sda3 volume 472GiB EFI partition
/0/100/1c bridge 100 Series/C230 Series Chipset Family PCI Express Root Port #5
/0/100/1d bridge 100 Series/C230 Series Chipset Family PCI Express Root Port #9
/0/100/1d.1 bridge 100 Series/C230 Series Chipset Family PCI Express Root Port #10
/0/100/1d.1/0 enp4s0 network RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
/0/100/1d.2 bridge 100 Series/C230 Series Chipset Family PCI Express Root Port #11
/0/100/1d.2/0 bridge ASM1083/1085 PCIe to PCI Bridge
/0/100/1f bridge B150 Chipset LPC/eSPI Controller
/0/100/1f/0 system PnP device PNP0c02
/0/100/1f/1 printer PnP device PNP0400
/0/100/1f/2 communication PnP device PNP0501
/0/100/1f/3 communication PnP device PNP0501
/0/100/1f/4 system PnP device PNP0c02
/0/100/1f/5 system PnP device PNP0c02
/0/100/1f/6 system PnP device PNP0b00
/0/100/1f/7 generic PnP device INT3f0d
/0/100/1f/8 system PnP device PNP0c02
/0/100/1f/9 system PnP device PNP0c02
/0/100/1f/a system PnP device PNP0c02
/0/100/1f/b system PnP device PNP0c02
/0/100/1f.2 memory Memory controller
/0/100/1f.3 card0 multimedia 100 Series/C230 Series Chipset Family HD Audio Controller
/0/100/1f.3/0 input4 input HDA Intel PCH Rear Mic
/0/100/1f.3/1 input5 input HDA Intel PCH Front Mic
/0/100/1f.3/2 input6 input HDA Intel PCH Line
/0/100/1f.3/3 input7 input HDA Intel PCH Line Out
/0/100/1f.3/4 input8 input HDA Intel PCH Front Headphone
/0/100/1f.4 bus 100 Series/C230 Series Chipset Family SMBus
/1 power To Be Filled By O.E.M.
/2 input0 input Sleep Button
/3 input1 input Power Button
/4 input2 input Power Button
/5 input3 input Eee PC WMI hotkeys
@2Friendly4You 2Friendly4You added the Bug Unexpected problem or unintended behavior that needs to be fixed label Apr 2, 2024
@xnetcat
Copy link
Member

xnetcat commented Apr 7, 2024

can you try downloading the file with yt-dlp?
can you try disabling audio conversion and/or manually converting m4a file to mp3 using ffmpeg?

@2Friendly4You
Copy link
Author

Trying with youtube-dlp:

webserver@webserver:~$ spotdl --log-level DEBUG --audio youtube --threads 8 "Test"
[22:06:28] DEBUG    MainThread - Downloader settings: {'audio_providers': ['youtube'], 'lyrics_providers': ['genius', 'azlyrics', 'musixmatch'], 'playlist_numbering': False, 'scan_for_songs': downloader.py:131
                    False, 'm3u': None, 'output': '{artists} - {title}.{output-ext}', 'overwrite': 'skip', 'search_query': None, 'ffmpeg': 'ffmpeg', 'bitrate': None, 'ffmpeg_args': None,
                    'format': 'mp3', 'save_file': None, 'filter_results': True, 'album_type': None, 'threads': 8, 'cookie_file': None, 'restrict': None, 'print_errors': False,
                    'sponsor_block': False, 'preload': False, 'archive': None, 'load_config': True, 'log_level': 'DEBUG', 'simple_tui': False, 'fetch_albums': False, 'id3_separator': '/',
                    'ytm_data': False, 'add_unavailable': False, 'generate_lrc': False, 'force_update_metadata': False, 'only_verified_results': False, 'sync_without_deleting': False,
                    'max_filename_length': None, 'yt_dlp_args': None, 'detect_formats': None, 'save_errors': None, 'ignore_albums': None, 'proxy': None, 'skip_explicit': False, 'log_format':
                    None, 'redownload': False, 'skip_album_art': False, 'create_skip_file': False, 'respect_skip_file': False}
[22:06:28] DEBUG    MainThread - FFmpeg path: ffmpeg                                                                                                                                            downloader.py:149
[22:06:28] DEBUG    MainThread - Found 0 known songs                                                                                                                                            downloader.py:184
[22:06:29] DEBUG    MainThread - Archive: 0 urls                                                                                                                                                downloader.py:231
[22:06:29] DEBUG    MainThread - Downloader initialized                                                                                                                                         downloader.py:233
[22:06:29] INFO     MainThread - Processing query: Test                                                                                                                                             search.py:137
[22:06:30] DEBUG    MainThread - Found 1 songs in 0 lists                                                                                                                                           search.py:339
[22:06:30] DEBUG    MainThread - Downloading 1 songs                                                                                                                                            downloader.py:280
[22:06:33] DEBUG    asyncio_0 - Found lyrics for Melanie Martinez - Test Me on Genius                                                                                                           downloader.py:400
[22:06:33] DEBUG    asyncio_0 - [45x3yuEpjmiqL4SFdT4srk] Searching for melanie martinez - test me                                                                                                     base.py:167
[22:06:39] DEBUG    asyncio_0 - [45x3yuEpjmiqL4SFdT4srk] Found 19 results for search query melanie martinez - test me with options {}                                                                 base.py:221
[22:06:39] DEBUG    asyncio_0 - [45x3yuEpjmiqL4SFdT4srk] Have to filter results: True                                                                                                                 base.py:244
[22:06:39] DEBUG    asyncio_0 - [45x3yuEpjmiqL4SFdT4srk] Filtered to 5 results                                                                                                                        base.py:256
[22:06:39] DEBUG    asyncio_0 - [45x3yuEpjmiqL4SFdT4srk] Best result is https://youtube.com/watch?v=suhVpVAp6a4 with score 100                                                                        base.py:264
[22:06:39] DEBUG    asyncio_0 - [45x3yuEpjmiqL4SFdT4srk] Returning best result https://youtube.com/watch?v=suhVpVAp6a4 with score 100                                                                 base.py:291
[22:06:39] DEBUG    asyncio_0 - Downloading Melanie Martinez - Test Me using https://youtube.com/watch?v=suhVpVAp6a4                                                                            downloader.py:654
[22:06:50] INFO     asyncio_0 - Downloaded "Melanie Martinez - Test Me": https://youtube.com/watch?v=suhVpVAp6a4                                                                                downloader.py:828
[22:06:50] DEBUG    MainThread - Took 20 seconds

Without conversion:

webserver@webserver:~$ spotdl --log-level DEBUG --format m4a --threads 8 "Test drive"
[22:22:30] DEBUG    MainThread - Downloader settings: {'audio_providers': ['youtube-music'], 'lyrics_providers': ['genius', 'azlyrics', 'musixmatch'], 'playlist_numbering': False,             downloader.py:131
                    'scan_for_songs': False, 'm3u': None, 'output': '{artists} - {title}.{output-ext}', 'overwrite': 'skip', 'search_query': None, 'ffmpeg': 'ffmpeg', 'bitrate': None,
                    'ffmpeg_args': None, 'format': 'm4a', 'save_file': None, 'filter_results': True, 'album_type': None, 'threads': 8, 'cookie_file': None, 'restrict': None, 'print_errors':
                    False, 'sponsor_block': False, 'preload': False, 'archive': None, 'load_config': True, 'log_level': 'DEBUG', 'simple_tui': False, 'fetch_albums': False, 'id3_separator':
                    '/', 'ytm_data': False, 'add_unavailable': False, 'generate_lrc': False, 'force_update_metadata': False, 'only_verified_results': False, 'sync_without_deleting': False,
                    'max_filename_length': None, 'yt_dlp_args': None, 'detect_formats': None, 'save_errors': None, 'ignore_albums': None, 'proxy': None, 'skip_explicit': False, 'log_format':
                    None, 'redownload': False, 'skip_album_art': False, 'create_skip_file': False, 'respect_skip_file': False}
[22:22:30] DEBUG    MainThread - FFmpeg path: ffmpeg                                                                                                                                            downloader.py:149
[22:22:30] DEBUG    MainThread - Found 0 known songs                                                                                                                                            downloader.py:184
[22:22:31] DEBUG    MainThread - Archive: 0 urls                                                                                                                                                downloader.py:231
[22:22:31] DEBUG    MainThread - Downloader initialized                                                                                                                                         downloader.py:233
[22:22:31] INFO     MainThread - Processing query: Test drive                                                                                                                                       search.py:137
[22:22:32] DEBUG    MainThread - Found 1 songs in 0 lists                                                                                                                                           search.py:339
[22:22:32] DEBUG    MainThread - Downloading 1 songs                                                                                                                                            downloader.py:280
[22:22:33] DEBUG    asyncio_0 - Genius failed to find lyrics for John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture                                         downloader.py:406
[22:22:33] DEBUG    asyncio_0 - AzLyrics failed to find lyrics for John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture                                       downloader.py:406
[22:22:34] DEBUG    asyncio_0 - MusixMatch failed to find lyrics for John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture                                     downloader.py:406
[22:22:34] DEBUG    asyncio_0 - No lyrics found for John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture, lyrics providers: Genius, AzLyrics, MusixMatch      downloader.py:566
[22:22:34] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Searching for john powell - test drive - from how to train your dragon music from the motion picture                                         base.py:167
[22:22:34] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Found 1 results for ISRC US3M51001211                                                                                                        base.py:180
[22:22:34] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Filtered to 1 ISRC results                                                                                                                   base.py:192
[22:22:34] DEBUG    asyncio_0 - [2QTDuJIGKUjR7E2Q6KupIh] Best ISRC result is https://music.youtube.com/watch?v=IpPIK4T068s with score 100.0                                                           base.py:201
[22:22:34] DEBUG    asyncio_0 - Downloading John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture using https://music.youtube.com/watch?v=IpPIK4T068s          downloader.py:654
[22:22:40] INFO     asyncio_0 - Downloaded "John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture": https://music.youtube.com/watch?v=IpPIK4T068s              downloader.py:828
[22:22:40] DEBUG    MainThread - Took 8 seconds                                                                                                                                                entry_point.py:164

Manually converting m4a to mp3 (takes around two seconds):

webserver@webserver:~$ ffmpeg -i 'John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture.m4a' -c:v copy -c:a libmp3lame -q:a 4 output.mp3
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x557b4b1826c0] stream 0, timescale not set
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'John Powell - Test Drive - From How To Train Your Dragon Music From The Motion Picture.m4a':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    title           : Test Drive - From How To Train Your Dragon Music From The Motion Picture
    artist          : John Powell
    album           : How To Train Your Dragon (Music From The Motion Picture)
    genre           : British Soundtrack
    track           : 11/25
    disc            : 1/1
    date            : 2010-01-01
    encoder         : Varese Sarabande
    WOAS            : https://open.spotify.com/track/2QTDuJIGKUjR7E2Q6KupIh
    rating          : 2
    album_artist    : John Powell
    comment         : https://music.youtube.com/watch?v=IpPIK4T068s
    copyright       : © 2010 DreamWorks Animation L.L.C., under exclusive license to Varese Sarabande, a division of Concord.
  Duration: 00:02:36.33, start: 0.000000, bitrate: 136 kb/s
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
  Stream #0:1: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 640x640 [SAR 118:118 DAR 1:1], 90k tbr, 90k tbn, 90k tbc (attached pic)
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
  Stream #0:0 -> #0:1 (aac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, mp3, to 'output.mp3':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    TIT2            : Test Drive - From How To Train Your Dragon Music From The Motion Picture
    TPE1            : John Powell
    TALB            : How To Train Your Dragon (Music From The Motion Picture)
    TCON            : British Soundtrack
    TRCK            : 11/25
    TPOS            : 1/1
    TDRC            : 2010-01-01
    TCOP            : © 2010 DreamWorks Animation L.L.C., under exclusive license to Varese Sarabande, a division of Concord.
    WOAS            : https://open.spotify.com/track/2QTDuJIGKUjR7E2Q6KupIh
    rating          : 2
    TPE2            : John Powell
    comment         : https://music.youtube.com/watch?v=IpPIK4T068s
    TSSE            : Lavf58.76.100
  Stream #0:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 640x640 [SAR 118:118 DAR 1:1], q=2-31, 90k tbr, 90k tbn, 90k tbc (attached pic)
  Stream #0:1(und): Audio: mp3, 44100 Hz, stereo, fltp (default)
    Metadata:
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 libmp3lame
frame=    1 fps=0.4 q=-1.0 Lsize=    2898kB time=00:02:36.30 bitrate= 151.9kbits/s speed=61.5x
video:135kB audio:2761kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.031319%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Unexpected problem or unintended behavior that needs to be fixed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xnetcat @2Friendly4You