Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Realtime not returning runs when they are have large payloads or outputs #1451

Open
ericallam opened this issue Nov 1, 2024 · 2 comments
Open

Comments

@ericallam
Copy link
Member

No description provided.

@ericallam
Copy link
Member Author

This will be fixed alongside the realtime streams release next week.

@matt-aitken matt-aitken changed the title Realtime not returning runs when they are have payloads or outputs Realtime not returning runs when they are have large payloads or outputs Nov 8, 2024
ericallam added a commit that referenced this issue Nov 19, 2024
* WIP realtime streams

* Handle realtime with large payloads or outputs #1451

* feat: optimize Redis stream handling with batching

Add STREAM_ORIGIN to environment schema. Improve performance in
RealtimeStreams by using TextDecoderStream for simpler text
decoding and implementing batching of XADD commands for Redis
streams. Limit stream size using MAXLEN option. Update
environment variable repository with new variable type. Adjust
import statements for Redis key and value types.

* 🔧 chore: add dev dependencies for bundle analysis

* add metadata tests and a few more utilties

* Add stream tests and improve streaming

* Added AI tool tasks, descriptions to tasks

* Use the config file path to determine the workingDir, then the package.json path

* Remove stream test files

* useTaskTrigger react hook that allows triggering a task from the client

* Add streaming support for the realtime react hooks

* Add ability to stream results after useTaskTrigger

* Improve the stream throttling

* Use the runId as the ID key to bust the cache after triggering

* Upgrade to to the latest electric sql client and server

* Make realtime server backwards compat with 3.1.2 release

* Pass the runId into useRealtimeRun

* Fix scopes when specifiying reading all runs

* WIP @trigger.dev/rsc package

* Various fixes and accepted recommendations by CodeRabbit

* Regenerate pnpm lock file

* A couple tweaks to rsc and give up on rendering react in tasks for now

* Add changeset

* Remove triggerRequest from the useEffect deps

* Improve realtime & frontend authentication errors

* Fixed authorization tests

* Remove unnecessary log

* Add metadata.stream limits and improve the metadata streams structure

* Streams can now have up to 2500 entries

* Various coderabbit fixes

* additional react-hooks jsdocs
@cbracegirdle
Copy link

cbracegirdle commented Dec 3, 2024

Is this issue still present? I have runs missing from my realtime stream and this could be the reason...

Things were working fine until roughly the time that version 3.3.2 was released yesterday...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants