Attachment (File Upload) API using HTML Form Based Uploads Deprecated

If you're currently using HTML Form Based Uploads and wish to continue (for now) then this document outlines how it works.

HTML form based uploads

This is a much more efficient way of sending files than using JSON based uploads, but not as efficient as using the Binary Upload method. The only real problem with it is that it doesn't conform to the API standard way of using JSON for everything, and can lead to some cludges when you're implementing this into your application.

It's based on this simple html form:

<form action="https://liquidfiles.example.com/attachments" enctype="multipart/form-data" method="post">
  &linput type="file" name="Filedata" filename="filename.ext">
<orm>

which would lead to the raw data being transmitted like this:

Content-type: multipart/form-data; boundary=AaB03x

--AaB03x
content-disposition: form-data; name="Filedata"; filename="filename.ext"
Content-Type: image/gif

... contents of filename.ext ...
--AaB03x--

While this may not look very different. It will enable us to send binary data as content, and the webserver can intercept this before the web application sees it and so on. Much more efficient.

This will also send the files separate from the message, and we'll just include references to the files when sending the message.

Example Request using curl

#!/bin/sh

# Some nice variables
api_key="Y9fdTmZdv0THButt5ZONIY"
server="https://liquidfiles.comapny.com"

# Uploading the actual file and get attachment id for each file
attachment_id=`curl -X POST --user "$api_key:x" -F Filedata=@bigfile.zip $server/attachments`

Please note in this example that curl syntax of @bigfile.zip means to load the data from the file bigfile.zip. If you enter '-F Filedata=bigfile.zip' without the @ it means send the string "bigfile.zip" as Filedata, that won't send the data from the file.

Example Response

8cvrfQ6zSQhLccJnKGzSV5

Please note with this response that it's not your normal JSON formatted type response. It's just the attachment id.

Example Request using curl with Pool ID

#!/bin/bash

# Some nice variables
api_key="Y9fdTmZdv0THButt5ZONIY"
server="https://liquidfiles.comapny.com"

# Uploading the actual file and get attachment id for each file
attachment_id=`curl -X POST --user "$api_key:x" -F Filedata=@bigfile.zip -F pool_id='partner-pool' $server/attachments`

Please note in this example that curl syntax of @bigfile.zip means to load the data from the file bigfile.zip. If you enter '-F Filedata=bigfile.zip' without the @ it means send the string "bigfile.zip" as Filedata, that won't send the data from the file.

Example Response

8cvrfQ6zSQhLccJnKGzSV5

Please note with this response that it's not your normal JSON formatted type response. It's just the attachment id.

Sending files in chunks

This will enable you to divide and send a large file uploaded in smaller pieces. This only works with the html form based upload and works by you splitting a large file in smaller pieces (chunks) and sending the chunks individually. When completed the server will rebuild the complete file. The benefits to this is that if one upload fails, the entire file doesn't have to be retransmitted. And some devices such as Microsoft ISA and TMG proxies struggle with files larger than 2Gb. Sending files in chunks will get around this and enable files of unlimited (well, limited by disk space) file size.

Request

Request URL: /attachments
Request VERB: POST
Parameters:
  Filedata:      # The html multiplart file data
  name           # String.  The file name. This is needed because we no longer can use the filename from the
                 #          html multipart file data.
  chunk          # Integer. The current piece between 0-(number of pieces -1)
  chunks         # Integer. The total number of pieces
Response:
  Attachment ID  # String.  A unique string representing the attachment id. This will be used to reference the
                 #          files in the message.

Example Request using curl

In this example, we're taking bigfile.zip, and splitting into two files: bigfile.zip.00 and bifile.zip.01 and sending them individually like this:

#!/bin/sh

# Some nice variables
api_key="Y9fdTmZdv0THButt5ZONIY"
server="https://liquidfiles.example.com"

curl -X POST --user "$api_key:x" -F Filedata=@bigfile.zip.00 -F name=bigfile.zip -F chunk=0 -F chunks=2 $server/attachments
attachment_id=`curl -X POST --user "$api_key:x" -F Filedata=@bigfile.zip.01 -F name=bigfile.zip -F chunk=1 -F chunks=2 $server/attachments`

In this example there are a few things to highlight:

• We will get the attachment id only when the last piece has been uploaded.
• The individual chunk sizes doesn't matter. If you're sending three chunks it can be two big ones and one small, three of equal size or one big, one medium and one small. It doesn't matter.
• You can send chunks in any order that you want. as long as you number the chunks correctly. You can for instance begin by sending the second chunk with chunk=1, followed by the first one with chunk=0. It's the chunk number that will order the pieces correctly on the server.
• The script above doesn't have any error handling. You need to make sure that you get a http response code 200 (success) after each chunk, and resend any chunks that fail accordingly. It's only when all pieces are uploaded that the server will rebuild the attachment and give you the attachment id.
• The "name" parameter needs to be unique for that user until the entire file has been uploaded. It's the only thing that we have to identify this file. If you try to send multiple files with the same name to the same user at the same time (the user with the api key: Y9fdTmZdv0THButt5ZONIY in this example) there will be a right mess on the server as it has no way of distinguishing between the two different files if the name is the same.

Checking what chunks have been uploaded

This API call requires LiquidFiles v2.5 or later.

When chunks are uploaded, they are stored for a week before being removed (if the file was not uploaded completely). Another thing to note is that a chunk (or a file for that matter) is interrupted during transit, the half uploaded chunk (or file) will be discarded. If you want to resume uploads, you can query the server to see what chunks are available and upload any missing chunks.

Also, a final thing to note is that chunks are unique per user and per filename. We don't have an Attachment ID (that would otherwise be unique) until the file has been completely uploaded. This means that if the same user starts uploading a new file using chunks with the same filename as the old one, and if you use this API call to check what's already uploaded you may well end up completing the old file with the remains of the new file.

Request

Request URL: /attachments/available_chunks
Request VERB: GET
Format: JSON
Parameters:
  name           # String.  The file name.
Response:
  chunks         # Array.   The response will be an array with chunk ID's and chunk sizes (in bytes).

Example Request using curl

#!/bin/sh

# Some nice variables
api_key="Y9fdTmZdv0THButt5ZONIY"
server="https://liquidfiles.company.com"

curl -X GET --user "$api_key:x" -H "Accept: application/json" -H "Content-Type: application/json" $server/attachments/available_chunks?name=bigfile.zip

{"chunks":[
  {
    "id":0,
    "size":104857600
  }, {
    "id":1,
    "size":104857600
  }]
}

In this case the first 2 chunks have been uploaded, both with 100Mb size. You can now continue with chunk 3 (chunk id 2, when starting from 0) starting at 200Mb.