Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multipart support #852

Open
0xx400 opened this issue Oct 4, 2021 · 5 comments
Open

multipart support #852

0xx400 opened this issue Oct 4, 2021 · 5 comments

Comments

@0xx400
Copy link

0xx400 commented Oct 4, 2021

Hi, I see example link (Multipart Form) here https://crates.io/crates/tide/0.1.1 are broken.

Is there any workaround?

There was a fix, #47

But it seems to be reverted.

Thanks!

@yoshuawuyts
Copy link
Member

Hi, thanks for asking!

Unfortunately we still don't support multipart because we need to author a multipart parser, as we found that there were no implementations in the ecosystem which could provide the guarantees we needed from a parser.

You can track the progress of the issue here: http-rs/http-types#126

@kindly
Copy link

kindly commented Dec 11, 2021

Just in case someone needs this, here is an option.

Use multer, which is a multipart parser:

#cargo.toml
multer= "2"

Copy this buffered stream implementaion:

use async_std::io::{self, Read};
use async_std::stream::Stream;
use async_std::task::{Context, Poll};

use std::pin::Pin;

#[derive(Debug)]
pub struct BufferedBytesStream<T> {
    inner: T,
}

impl<T: Read + Unpin> Stream for BufferedBytesStream<T> {
    type Item = async_std::io::Result<Vec<u8>>;

    fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
        let mut buf = [0u8; 2048];

        let rd = Pin::new(&mut self.inner);

        match futures_core::ready!(rd.poll_read(cx, &mut buf)) {
            Ok(0) => Poll::Ready(None),
            Ok(n) => Poll::Ready(Some(Ok(buf[..n].to_vec()))),
            Err(ref e) if e.kind() == io::ErrorKind::Interrupted => Poll::Pending,
            Err(e) => Poll::Ready(Some(Err(e))),
        }
    }
}

Parse the request headers out of the tide Request to get out the multipart boundtry needed for parsing and use multer to parse the request. See multer docs:

use use multer::Multipart;

let multipart_boudry = "".to_string() 

if let Some(mime) = req.content_type() {
    let content_type = mime.essence().to_string();
    if content_type == "multipart/form-data" {
        if let Some(boundry) = mime.param("boundary") {
            multipart_boundry = boundry.to_string()
        }
    }
}

// use buffered stream on the request
let body_stream = BufferedBytesStream { inner: req };
let mut multipart = Multipart::new(body_stream, multipart_boundry.clone());

// download a file fromt the field "file" in the multipart request.

let download_file = "/some_path/";

while let Some(mut field) = multipart.next_field().await? {
    if field.name() != Some("file") {
        continue;
    }
    let mut output = File::create(&download_file).await?;

    while let Some(chunk) = field.chunk().await? {
        output.write_all(&chunk).await?;
    }
    output.flush().await?;
}

@kindly
Copy link

kindly commented Dec 11, 2021

I wonder if a BufferedByteStream could be added to asnyc-std like Bytes. https://docs.rs/async-std/latest/async_std/io/struct.Bytes.html#. With perhaps buffered_bytes on ReadExt https://docs.rs/async-std/latest/async_std/io/trait.ReadExt.html for a different stream implementation on readers.

I initially used a wrapped Bytes stream for this but it was much slower as it polls for every byte. The above BufferedBytesStream is a tweaked version of this.

The streams lines and split on BufRead work (except for keeping the delimiters) but are an issue as a file could be on one line, therefore most of the request would be put in memory.

@amatveiakin
Copy link

In the meantime, would it make sense to update Request.body_form documentation to mention that the form must be in “application/x-www-form-urlencoded” format rather than “multipart/form-data”?

It's really easy to accidentally get a form in multipart format if sending data from JS, because FormData is always encoded as “multipart/form-data”. And tide error message is quite unhelpful, so it can take some time to debug.

@yungcomputerchair
Copy link

I just ran into this while porting my Flask app to Tide. I agree that the docs should really have some indication of this restriction as it would have saved me a lot of debugging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants