How does HTTP file upload work?

When I submit a simple form like this with a file attached:

<form enctype="multipart/form-data" action="http://localhost:3000/upload?upload_progress_id=12344" method="POST">
<input type="hidden" name="MAX_FILE_SIZE" value="100000" />
Choose a file to upload: <input name="uploadedfile" type="file" /><br />
<input type="submit" value="Upload File" />
</form>

How does it send the file internally? Is the file sent as part of the HTTP body as data? In the headers of this request, I don't see anything related to the name of the file.

I just would like the know the internal workings of the HTTP when sending a file.


Let's take a look at what happens when you select a file and submit your form (I've truncated the headers for brevity):

POST /upload?upload_progress_id=12344 HTTP/1.1
Host: localhost:3000
Content-Length: 1325
Origin: http://localhost:3000
... other headers ...
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryePkpFF7tjBAqx29L

------WebKitFormBoundaryePkpFF7tjBAqx29L
Content-Disposition: form-data; name="MAX_FILE_SIZE"

100000
------WebKitFormBoundaryePkpFF7tjBAqx29L
Content-Disposition: form-data; name="uploadedfile"; filename="hello.o"
Content-Type: application/x-object

... contents of file goes here ...
------WebKitFormBoundaryePkpFF7tjBAqx29L--

Instead of URL encoding the form parameters, the form parameters (including the file data) are sent as sections in a multipart document in the body of the request.

In the example above, you can see the input MAX_FILE_SIZE with the value set in the form, as well as a section containing the file data. The file name is part of the Content-Disposition header.

The full details are here.


How does it send the file internally?

The format is called multipart/form-data , as asked at: What does enctype='multipart/form-data' mean?

I'm going to:

  • add some more HTML5 references
  • explain why he is right with a form submit example
  • HTML5 references

    There are three possibilities for enctype :

  • x-www-urlencoded
  • multipart/form-data (spec points to RFC2388)
  • text-plain . This is "not reliably interpretable by computer", so it should never be used in production, and we will not look further into it.
  • How to generate the examples

    Once you see an example of each method, it becomes obvious how they work, and when you should use each one.

    You can produce examples using:

  • nc -l or an ECHO server
  • an user agent like a browser or cURL
  • Save the form to a minimal .html file:

    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="utf-8"/>
      <title>upload</title>
    </head>
    <body>
      <form action="http://localhost:8000" method="post" enctype="multipart/form-data">
      <p><input type="text" name="text1" value="text default">
      <p><input type="text" name="text2" value="a&#x03C9;b">
      <p><input type="file" name="file1">
      <p><input type="file" name="file2">
      <p><input type="file" name="file3">
      <p><button type="submit">Submit</button>
    </form>
    </body>
    </html>
    

    We set the default text value to a&#x03C9;b , which means aωb because ω is U+03C9 , which are the bytes 61 CF 89 62 in UTF-8.

    Create files to upload:

    echo 'Content of a.txt.' > a.txt
    
    echo '<!DOCTYPE html><title>Content of a.html.</title>' > a.html
    
    # Binary file containing 4 bytes: 'a', 1, 2 and 'b'.
    printf 'axCFx89b' > binary
    

    Run our little echo server:

    while true; do printf '' | nc -l 8000 localhost; done
    

    Open the HTML on your browser, select the files and click on submit and check the terminal.

    nc prints the request received.

    Tested on: Ubuntu 14.04.3, nc BSD 1.105, Firefox 40.

    multipart/form-data

    Firefox sent:

    POST / HTTP/1.1
    [[ Less interesting headers ... ]]
    Content-Type: multipart/form-data; boundary=---------------------------735323031399963166993862150
    Content-Length: 834
    
    -----------------------------735323031399963166993862150
    Content-Disposition: form-data; name="text1"
    
    text default
    -----------------------------735323031399963166993862150
    Content-Disposition: form-data; name="text2"
    
    aωb
    -----------------------------735323031399963166993862150
    Content-Disposition: form-data; name="file1"; filename="a.txt"
    Content-Type: text/plain
    
    Content of a.txt.
    
    -----------------------------735323031399963166993862150
    Content-Disposition: form-data; name="file2"; filename="a.html"
    Content-Type: text/html
    
    <!DOCTYPE html><title>Content of a.html.</title>
    
    -----------------------------735323031399963166993862150
    Content-Disposition: form-data; name="file3"; filename="binary"
    Content-Type: application/octet-stream
    
    aωb
    -----------------------------735323031399963166993862150--
    

    For the binary file and text field, the bytes 61 CF 89 62 ( aωb in UTF-8) are sent literally. You could verify that with nc -l localhost 8000 | hd nc -l localhost 8000 | hd , which says that the bytes:

    61 CF 89 62
    

    were sent ( 61 == 'a' and 62 == 'b').

    Therefore it is clear that:

  • Content-Type: multipart/form-data; boundary=---------------------------9051914041544843365972754266 Content-Type: multipart/form-data; boundary=---------------------------9051914041544843365972754266 sets the content type to multipart/form-data and says that the fields are separated by the given boundary string.

  • every field gets some sub headers before its data: Content-Disposition: form-data; , the field name , the filename , followed by the data.

    The server reads the data until the next boundary string. The browser must choose a boundary that will not appear in any of the fields, so this is why the boundary may vary between requests.

    Because we have the unique boundary, no encoding of the data is necessary: binary data is sent as is.

    TODO: what is the optimal boundary size ( log(N) I bet), and name / running time of the algorithm that finds it? Asked at: https://cs.stackexchange.com/questions/39687/find-the-shortest-sequence-that-is-not-a-sub-sequence-of-a-set-of-sequences

  • Content-Type is automatically determined by the browser.

    How it is determined exactly was asked at: How is mime type of an uploaded file determined by browser?

  • application/x-www-form-urlencoded

    Now change the enctype to application/x-www-form-urlencoded , reload the browser, and resubmit.

    Firefox sent:

    POST / HTTP/1.1
    [[ Less interesting headers ... ]]
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 51
    
    text1=text+default&text2=a%CF%89b&file1=a.txt&file2=a.html&file3=binary
    

    Clearly the file data was not sent, only the basenames. So this cannot be used for files.

    As for the text field, we see that usual printable characters like a and b were sent in one byte, while non-printable ones like 0xCF and 0x89 took up 3 bytes each: %CF%89 !

    Comparison

    File uploads often contain lots of non-printable characters (eg images), while text forms almost never do.

    From the examples we have seen that:

  • multipart/form-data : adds a few bytes of boundary overhead to the message, and must spend some time calculating it, but sends each byte in one byte.

  • application/x-www-form-urlencoded : has a single byte boundary per field ( & ), but adds a linear overhead factor of 3x for every non-printable character.

  • Therefore, even if we could send files with application/x-www-form-urlencoded , we wouldn't want to, because it is so inefficient.

    But for printable characters found in text fields, it does not matter and generates less overhead, so we just use it.


    Send file as binary content (upload without form or FormData)

    In the given answers/examples the file is (most likely) uploaded with a HTML form or using the FormData API. The file is only a part of the data sent in the request, hence the multipart/form-data Content-Type header.

    If you want to send the file as the only content then you can directly add it as the request body and you set the Content-Type header to the MIME type of the file you are sending. The file name can be added in the Content-Disposition header. You can upload like this:

    var xmlHttpRequest = new XMLHttpRequest();
    
    var file = ...file handle...
    var fileName = ...file name...
    var target = ...target...
    var mimeType = ...mime type...
    
    xmlHttpRequest.open('POST', target, true);
    xmlHttpRequest.setRequestHeader('Content-Type', mimeType);
    xmlHttpRequest.setRequestHeader('Content-Disposition', 'attachment; filename="' + fileName + '"');
    xmlHttpRequest.send(file);
    

    If you don't (want to) use forms and you are only interested in uploading one single file this is the easiest way to include your file in the request.

    链接地址: http://www.djcxy.com/p/7092.html

    上一篇: 使用curl从HTTP POST获取响应头

    下一篇: HTTP文件上传如何工作?