Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update spotify-player extension — 'Your Library' command: fetch ALL liked tracks - needed for proper functionality #16335

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

inducingchaos
Copy link

@inducingchaos inducingchaos commented Jan 13, 2025

Problem

The 'Your Library' command needs your entire library dataset for accurate search results.

Spotify's API has limits:

This requires a balanced consideration of UX, performance, and API constraints. I implemented infinite fetching and caching to solve this.

Note: This PR only focuses on the "Liked Tracks" aspect of the 'Your Library' command, as well as the overlying architecture for other content types.

Future implementations needed for:

  • Podcasts
  • Albums
  • Artists
  • Playlists

Solution

Overview

  • Recursively fetch the entire library
  • Cross-instance caching with intelligent updates
  • API data minification to avoid memory issues
  • UX improvements to offset architectural challenges

Implementation Details

1. Data Fetching

  • Batched parallel requests
    • 5 parallel requests per batch
    • 50 tracks per request
    • Individual requests are too slow
    • Combined requests hit rate limits
  • Artificial delay between batches
    • 100ms to avoid rate limits
    • Exponential backoff (3 retries)

2. Caching

  • Persisted between Raycast runs using SWR-like strategy
  • Revalidation on every run
    • Uses total track count
    • Single track fetch from me/tracks
    • Instant search result updates
  • Manual refresh action
    • Re-fetches entire library
    • Default action for all list items
    • Accessible via cmd + r

3. Response Model + Memory Optimization

Minimized data structure:

interface MinimalTrack {
  id: string;
  name: string;
  artists: { name: string }[];
  album: {
    id: string;
    name: string;
    images: { url: string }[];
  };
  uri: string;
  duration_ms: number;
}

4. UX Improvements

  • Library loading/updating indicators
    • Progress percentage
    • Completion notifications

5. Miscellaneous

  • Enhanced action panel for songs
  • Standardized default actions
    • 'Enter' now plays instead of opening Spotify
  • Consistent Liked Tracks list item UI
    • Changed filtered results layout
    • Now shows: Title | Artists | Duration
  • Changed default behavior to close window after play actions

Known Issues

Memory Management

Data minimization limitations may require:

  1. Intra-fetch data-offloading to disk
  2. More sophisticated caching strategy
  3. 3rd party API for outsourcing operations

Current solution supports ~2,000 songs (verified)

  • Pre-allocated result array reduces memory pressure
  • Previous versions had memory heap overflow

Development Issues

  • Double-fetching observed
    • Possibly React dev mode behavior
    • Potential race condition:
    //  useYourLibrary.ts
    execute: options.execute !== false && !tracksLoading;
    //  useMySavedTracks.ts
    execute: options?.execute !== false;

UI/UX Challenges

  1. Progress/Loading State

    • Progress sometimes sticks at last percentage
    • May be dev-only issue during hot-reload
  2. Search Results

    • No pagination/limits due to Raycast list filtering
    • Categories complicate pagination
    • Potential solutions:
      • Custom filtering (reduces UX)
      • Programmatic pagination
      • Accept performance trade-off
  3. Performance

    • API-imposed fetch delays
    • Initial results flash
    • Search lag with large results
    • Cache hit latency
    • Progress indicator jitter

Error Handling

  • Not fully implemented
  • Current workaround: command re-run
  • User testing shows reliability

Cache Validation

Limitations:

  • Misses changes when total track count unchanged
  • Alternative approaches:
    1. 3rd party change observation
    2. Time-based invalidation
    3. Forward/backward validation
    4. Hybrid strategy

100% of the changes in this PR are AI-generated (except this one!). 🍾🤯 EDIT: not anymore!

Related

Closes #14663 and provides a half-solution for #7222.

Explore the full commit history here: https://github.com/inducingchaos/raycast-spotify-extension

Checklist

- fix(types): resolve type errors across components
- chore: deleted notes
- feat(ux): improve library fetch and window behavior
- docs: improve PR notes formatting and readability
- refactor(api): optimize getMySavedTracks with improved reliability and performance
- feat: add refresh action to empty library state
- fix: improve library fetch reliability and UX
- feat: add refresh action to all library sections
- docs: finalize PR notes for library implementation
- chore: remove debug console.logs
- fix(library): race conditions & documentation updates
- docs: update implementation notes and next steps
- feat(library): improve cache validation and background updates
- feat(ui): standardize track actions and display format
- docs: document search implementation decisions
- feat(ui): standardize track display format across all views
- docs: update progress notes with double fetching investigation
- refactor(library): generalize terminology for library-wide use
- feat(library): implement persistent caching with LocalStorage
- feat: improve loading UX and progress tracking
- feat: implement infinite fetch with search functionality
- fix: increase rate limit delay to avoid 429 errors
- refactor: unify track data structure across components
- perf: optimize track fetching with batched parallel requests
- perf: optimize hooks to prevent double fetching and re-renders
- perf: minimize track data structure to reduce memory usage
- test: disable track UI to isolate memory issue
- feat: add progress tracking for track fetching
- feat: add useMySavedTracks hook with caching
- chore: rename notes file, add todos
- chore: updated notes, debug logs
- build: initial + notes, infinite fetch
- Initial commit
@raycastbot raycastbot added extension fix / improvement Label for PRs with extension's fix improvements extension: spotify-player Issues related to the spotify-player extension labels Jan 13, 2025
@raycastbot
Copy link
Collaborator

Thank you for your first contribution! 🎉

🔔 @mattisssa @peduarte @sxn @dillionverma @andreaselia @stuart @tonka3000 @dancannon @pernielsentikaer @stevensd2m @erics118 @hjoelh @hobhouse @jatindotdev @the-devbear @rfaccio @badta5te @andyburris @thomaslombart @rhesamu @mpatel283 you might want to have a look.

You can use this guide to learn how to check out the Pull Request locally in order to test it.

You can expect an initial review within five business days.

@inducingchaos inducingchaos changed the title Update spotify-player extension Update spotify-player extension — 'Your Library': Fetch ALL Liked Tracks - Needed For Proper Functionality Jan 13, 2025
@inducingchaos inducingchaos changed the title Update spotify-player extension — 'Your Library': Fetch ALL Liked Tracks - Needed For Proper Functionality Update spotify-player extension — 'Your Library' command: fetch ALL liked tracks - needed for proper functionality Jan 13, 2025
@inducingchaos
Copy link
Author

If more context is needed for any of these points, check this first: https://github.com/inducingchaos/raycast-spotify-extension/blob/d5ea9efddd76196df947ae1435424768f2cb0a51/DO_NOT_EDIT_PR_NOTES.md

It is the human-written version of the PR description before I optimized it for readability with Claude.

This update could probably use a bit of refactoring, my intent with it was to create a direction and working proof of concept.

I am now using this version as my go-to for playing songs but there is definitely room for improvement. Do as you wish with the code - to me, it has been proven to be reliable and production-worthy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
extension fix / improvement Label for PRs with extension's fix improvements extension: spotify-player Issues related to the spotify-player extension
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Spotify Player] Only recent items showing in "Your Library" search results
2 participants