Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let the driver determine the appropriate payload storage format #116

Open
tonyarnold opened this issue Sep 5, 2022 · 3 comments
Open

Let the driver determine the appropriate payload storage format #116

tonyarnold opened this issue Sep 5, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@tonyarnold
Copy link

tonyarnold commented Sep 5, 2022

I'm frustrated that when I use the excellent Fluent Driver for Queues, the payload/JobData enforces storage as raw Data. This ends up meaning that for Postgres and MySQL, it's a column containing [UInt8], which is far from ideal.

I assume that this was done for the early drivers, which require storage this way, but it would be good to unpick this.

Offering the payload as Codable and expecting the driver to handle persistence would be a more flexible approach, and perhaps allow the use of things like Postgres' jsonb type for far easier inspection and debugging of in-flight jobs.

@jdmcd
Copy link
Member

jdmcd commented Sep 5, 2022

@tonyarnold thanks for the suggestion! On mobile so apologies for the quick response/possibly inaccurate information, but if I remember correctly we had to store the data inside of JobData as [UInt8] instead of Codable because we ran into problems with Codable not being allowed to be used as an existential, and some other problem when trying to use it as a generic. Sorry for the vagueness, when I'm back at a computer I'll see if I can dig in and remember exactly why we made that choice.

@tonyarnold
Copy link
Author

Surely that's the concern of the driver though, right — shouldn't Queues itself only be saying "here's what I need you to persist"?

@jdmcd
Copy link
Member

jdmcd commented Sep 6, 2022

@tonyarnold agreed, but I believe the only way we were able to coerce the type system into doing that was dropping down to bytes. Will have to carve out some time this week to take a look, in the meantime @gwynne I believe you helped me and @tanner0101 when we were writing that part of it, let me know if you recall any of this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants