I am trying to get multithreaded chunked upload working (the main idea is that we download a big file to our server by chunks (only keep them in memory until they are uploaded) and redirect them to dropbox via upload_chunk method from the dropbox api(Python)). The error messages after each failed chunk-upload are in the form [400] u'Submitted input out of alignment: got [181403648] expected [2097152]'. The first 1MB chunk is uploaded before the threaded mess as to get an upload_id to pass to all threads. It seems that dropbox expects the chunks to arrive in order. So is my task possible? Here is the code: http://pastebin.com/NSMEEPiS . This is my first attempt to write a multithreaded program so maybe there is huge misunderstanding about how everything should work.
2
votes
yes it does support it but you have to enable multithreading support in python see effbot.org/zone/thread-synchronization.htm
- Rachel Gallen
Your program looks kind-of-fine, but there are issues regarding termination, which is always messy with threads (you have to manually send a "done!" token to each thread via the queue, and have the thread quit correctly). Anyway, the answer to your question seem to be the one you suspected: dropbox might just refuse the upload to arrive in out-of-order pieces.
- Armin Rigo
Yes, they told me that this is the case in the dropbox forum.
- rzr
But then how does websites like urldroplet.com, zbigz.com do high speed dropbox uploads? No matter how good their Internet is, it should still take time like it does in my amazon server.
- scottydelta
