Need to see if your shared folder is taking up space on your dropbox 👨💻? Find out how to check here.
Forum Discussion
MatthiD
2 years agoNew member | Level 2
Upload multiple files in session
Backstory I have a bunch of small PDFs (18 kb each). filesUpload worked until I understood that a `429` is quite usual. Researched and found the batch endpoints. I wanted to use `/upload_session...
MatthiD
2 years agoNew member | Level 2
Hey DB-Des ,
thank you so much for the detailed answer.
I already saw the examples and explanations, but could not adapt it to my specific use case.
Meanwhile I solved the issue.
Two things I recognized/learned:
- I did not understand the cursor concept at first. I thought, it should point to the beginning of the file part, but after sending data the cursor should point to the end of the file part (file length) in the finish method.
Maybe a graphic or animation would be helpful to explain this. - I am still unsure about the `filesUploadSessionStartBatch()` and `filesUploadSessionFinishBatchV2()`. According to the documentation, these two do not send any data in combination, right? So we would need either the `filesUploadSessionAppendV2` or the “missing” `filesUploadSessionAppendBatchV2`. Since the whole thing was to reduce requests due to a 429 of the looped uploadFile method, I don’t understand why I should go with the single append method and call it multiple times again. Performance guide says:
For each file in a batch, in parallel, call /files/upload_session/append_v2 as needed to upload the full contents of the file over multiple requests.
Why? Does it mean that the buffer prevents the 429s (regardless to the amount of requests to the buffer)? Maybe a graphic would help here as well to understand the steps and the infrastructure.
I solved everything for my case, but wanted to drop these thoughts for people having the same problem.
Additionally my code that worked for me:
async function filesUploadSessionAppendBatch(
content: ArrayBufferLike,
entries: AppendBatchEntry[],
) {
// this gets a token or a new one if it is expired
const accessToken = await dropboxMaker.getToken();
const response = await fetch(
"https://content.dropboxapi.com/2/files/upload_session/append_batch",
{
method: "POST",
headers: {
Authorization: `Bearer ${accessToken}`,
"Dropbox-API-Arg": JSON.stringify({
entries,
}).replace(/[\u007f-\uffff]/g, getSafeUnicode),
"Content-Type": "application/octet-stream",
},
body: content,
},
);
return response.json();
}
export async function uploadFiles(files: UploadFileData[]) {
const num_sessions = files.length;
try {
if (num_sessions < 1) {
return {};
}
const dbx = await dropboxMaker.client();
if (num_sessions === 1) {
// upload one file
await dbx.filesUpload({
...files[0],
});
} else {
// upload multiple files
// merges all files in one ArrayBuffer (to make one single request....will still be far away from the upload limit)
const allContent = concatArrayBuffers(
...files.map(({ contents }) => contents),
);
const startResponse = await dbx.filesUploadSessionStartBatch({
num_sessions,
});
const { session_ids } = startResponse.result;
const batchData: Array<FullBatchEntry> = files.reduce(
(
acc: { offset: number; entries: Array<FullBatchEntry> },
cur,
index,
) => {
acc.entries.push({
close: true,
cursor: {
session_id: session_ids[index],
offset: 0,
// always start with 0 for a new session. Don’t upcount.
},
commit: {
autorename: true,
/** @ts-expect-error Wrong type used (see CommitInfo) */
mode: "add",
mute: false,
path: cur.path,
},
length: cur.contents.byteLength,
});
// not needed anymore
// acc.offset += cur.contents.byteLength;
return acc;
},
{
offset: 0,
entries: [],
},
).entries;
await filesUploadSessionAppendBatch(
allContent,
batchData.map(({ cursor, length, close }) => ({
cursor,
length,
close,
})),
);
await dbx.filesUploadSessionFinishBatchV2({
entries: batchData.map(({ commit, cursor, length }) => ({
commit,
cursor: {
...cursor,
// point to the end of the upload file part...so length
offset: length,
},
})),
});
}
} catch (error) {
console.log(error);
}
}Thanks again for your help 💙
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.
If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!