We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
ivanhigueram
7 years agoExplorer | Level 4
[Python V2] How to do batch upload?
Hi! I am trying to upload several files using the sessions and batch operators in the Dropbox SDK for Python. I'm trying to do something like this: dbx = dropbox.Dropbox(<API KEY>)
commit_info...
9krausec
5 years agoExplorer | Level 4
Greg-DB - Thank you for following this and providing input. I think I'm close to a solution and will continue tinkering. In the meantime, please review my method below and provide any further suggestions.
For output, I am getting an "UploadSessionFinishBatchLaunch" return at the end with an async_job_id. However, nothing is uploaded to dropbox after completion. Thank you.
from os import listdir
from os.path import isfile, join
import dropbox
def batchUpload(srcFiles, dstDir, dbx=None):
if not dbx: dbx = DropboxTeam(DROPBOX_TOKEN).as_admin(ADMIN_ID)
chunkSize = 2 * 1024 * 1024
uploadEntryList = []
for src in srcFiles:
baseName = os.path.basename(src)
dstPath = dstDir + "/" + baseName
fileSize = os.path.getsize(src)
print("{0} ==> {1}".format(src, dstPath))
with open(src, "rb") as fp:
if fileSize <= chunkSize: # file smaller than chunkSize, no files_upload_session_append_v2
uploadSessionStartResult = dbx.files_upload_session_start(fp.read(), dstPath, close=True)
cursor = dropbox.files.UploadSessionCursor(session_id=uploadSessionStartResult.session_id,
offset=fp.tell())
commit = dropbox.files.CommitInfo(path=dstPath)
else: # file larger than chunkSize. Use files_upload_session_append_v2
uploadSessionStartResult = dbx.files_upload_session_start(fp.read(chunkSize))
cursor = dropbox.files.UploadSessionCursor(session_id=uploadSessionStartResult.session_id,
offset=fp.tell())
commit = dropbox.files.CommitInfo(path=dstPath)
while fp.tell() <= fileSize:
if ((fileSize - fp.tell()) <= chunkSize): # file chunk append complete
print("COMPLETE CHUNKS")
uploadEntryList.append(files.UploadSessionFinishArg(cursor=cursor, commit=commit))
break
else:
print(fp.tell())
dbx.files_upload_session_append_v2(fp.read(chunkSize),
cursor)
cursor.offset = fp.tell()
uploadEntryList.append(files.UploadSessionFinishArg(cursor=cursor, commit=commit))
print( dbx.files_upload_session_finish_batch(uploadEntryList) )
srcDir = "C:/Users/alpha/Documents/scratch/LIGHTS - Copy"
srcFiles = [join(srcDir, f).replace('\\', '/') for f in listdir(srcDir) if isfile(join(srcDir, f))]
dstDir = "/scratch/poopOne"
dbx = DropboxTeam(DROPBOX_TOKEN).as_admin(ADMIN_ID)
batchUpload(srcFiles, dstDir, dbx)Greg-DB
Dropbox Community Moderator
5 years ago9krausec You'll need to poll files_upload_session_finish_batch_check to check on the result of the job. If you're not seeing anything uploaded, something must have failed, and the result from files_upload_session_finish_batch_check should tell you what it was.
- 9krausec5 years agoExplorer | Level 4
Greg - Thank you again. No errors were returned. Just states that status is "in-progress"
UploadSessionFinishBatchJobStatus('in_progress', None)
I'll tell ya what - You follow me through here to the end and I bet this is going to be an excellent reference for everyone else using Python. I'm actually surprised I wasn't able to find a working example in this forum, or in the examples on git.
In the meantime I'll keep at it.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.
If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!