We’re Still Here to Help (Even Over the Holidays!) - find out more here.
Forum Discussion
[Python V2] How to do batch upload?
Hi! I am trying to upload several files using the sessions and batch operators in the Dropbox SDK for Python. I'm trying to do something like this: dbx = dropbox.Dropbox(<API KEY>)
commit_info...
Hi Greg!
Again thanks for replying back so promptly. I have some follow up questions. If I am looping through different Python objects, not files, that's why I'm first converting the pd.DataFrame to a string, and then pointing it to the dbx.files_upload_session_start() function.
Since my file is complete, and I am not passing a files in a context manager or a StringIO, I did not specify any offset in the cursor. Now that I'm trying to run the loop, I received the following error:
Traceback (most recent call last):
File "/Users/ivan/.pyenv/versions/weather_data/lib/python3.6/site-packages/dropbox/stone_serializers.py", line 337, in encode_struct
field_value = getattr(value, field_name)
File "/Users/ivan/.pyenv/versions/weather_data/lib/python3.6/site-packages/dropbox/files.py", line 10278, in offset
raise AttributeError("missing required field 'offset'")
AttributeError: missing required field 'offset'
What are the real advantages of using the batch operations to upload files? Seems a convulted use case for objects in memory, rather than files.
Thanks again for your help.
Greg-DB
Dropbox Community Moderator
Dropbox Community ModeratorRegardless of where the data is coming from, the `UploadSessionCursor` object does require an `offset`, in order "to make sure upload data isn’t lost or duplicated in the event of a network error". It sounds like in your case the `offset` value would be the length of the string.
The main advantage of using `files_upload_session_finish_batch` is to minimize the number of "locks" needed when uploading multiple files. The Data Ingress Guide covers this in more detail. This applies to uploading from memory or files.
The main advantage of using "upload sessions" to begin with is to enable apps to upload large files. If you're just uploading small files, you can certainly do so just using `files_upload`, but you'd need to do so serially to avoid lock contention. Based on the code you provided, you're already uploading serially though, so it may not make much of a difference if you wish to switch to `files_upload` for simplicity.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
