-
Notifications
You must be signed in to change notification settings - Fork 787
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feedback on Multipart #7408
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi folks, and thanks for this great library.
Recently, I implemented a small webservice using Http4s and its
Multipart
support. I noticed the "we'd love the feedback" comment on the package object, so I figured I'd give some.As far as I can tell, the current decoder implementation for
Multipart
doesn't let you make any on-the-fly decisions that could cause parsing to abort. For example, if a part looks like a file upload but your endpoint isn't expecting any uploads; or if you're only expecting a file upload in a specific part. In these situations where your business logic will end up rejecting the request (responding with a a 40x), you end up wasting time if there are more parts with lots of bytes to receive before the decoder finishes.I also noticed the way
Part
s are represented is with aStream[F, Byte]
, which gives rise to a double-buffering problem when your business logic actually wants to dump a Part's content to a temp file. If the size of the file is large enough, the underlying decoder logic will buffer the content to a temp file of its own, and then the business logic would copy that content to its own temp file. Ideally, when receiving the headers for a Part, the business logic could decide to buffer that Part's body to a temp file, and expose aPath
to that temp file.There also doesn't seem to be a way to impose size limits on individual part bodies. I'd like to be able to "hang up" if for example someone tried to upload an overly-large file or an infinite stream of bytes.
I also noticed that
parseStreamedFile
andparseToPartsStreamedFile
are marked as deprecated, andthey recommend usingparseSupervisedFile
instead... butparseSupervisedFile
is package-private.In my efforts to work around these issues, I found the
MultipartParser.Event
type and theMultipartParser.parseEvents
pipe, which I could theoretically use to implement a new multipart decoder that addresses my above complaints, but those two things are currentlyprivate
in http4s 0.23.25.I've got a prototype implementation that uses
Part[IO] => DecodeResult[Resource[IO, *], A]
to represent on-the-fly decision making for Parts, and resource-allocation concerns for e.g. the buffer-to-temp-file case. It currently works in terms of anEvent
class that mirrors the one inMultipartParser
, but for my testing sandbox, I'm just hard-coding a sequence of Events. I've also created aMultipartReceiver[F, A]
trait which has an Applicative and a builder DSL.It's not quite PR-ready, but I figured I'd show some highlights and see if there's interest in incorporating it into http4s-core, or prompt the the
Events/parseEvents
code to become public so I could reuse them in a library I publish myself.The text was updated successfully, but these errors were encountered: