Discuss Dropbox Developer & API
Hi,
I am now doing an online experiment via URL and trying to collect data using dropbox javascript SDK.
At the end of the experiment, the success message was always prompted. But the data was not always uploaded to my dropbox.
About 1/5 data was missing on the way to my dropbox.
I have no idea why the data is missing and need help with possible reasons.
Here is my code. Each file is about 30kb, so the file size shouldn't be a problem.
I don't know if this is the right way to place the question. Is any extra information needed?
require("dotenv").config();
var Dropbox = require("dropbox").Dropbox;
const fetch = require("node-fetch");
const dbx = new Dropbox({
accessToken: #############,
refreshToken: #############,
clientId: #############,
clientSecret: #############,
fetch
});
var saveDropbox = function(content, filename){
dbx.filesUpload({
path: "/" + filename,
contents: content
})
};
app.post("/experiment-data/:filename", function(request, response){
// convert json to csv
DATA_CSV = json2csv(request.body);
var filename = request.params.filename;
savedropbox(DATA_CSV, filename);
response.end();
});
function(){
var sel_data = jsPsych.data.get();
$.ajax({
type: "POST",
url: "/experiment-data/" + filename,
data: JSON.stringify(sel_data.values()),
contentType: "application/json"
}).done(function(){
alert("You have compeleted the experiment and your data has been
saved! ");
}).fail(function(){
alert("A problem occurs while writing data to Dropbox. "
+ "Data will be saved to your computer. ";
var csv = sel_data.csv();
downloadCSV(csv, fname);
});
@fangzefunny wrote:...
At the end of the experiment, the success message was always prompted. But the data was not always uploaded to my dropbox.
...
... var saveDropbox = function(content, filename){ dbx.filesUpload({ path: "/" + filename, contents: content }) }; ......
Hi @fangzefunny,
In your 'saveDropbox' function, what are you doing with Dropbox 'filesUpload' method result? On success there should be received a file metadata to the file just uploaded and on error - the corresponding error information. As far as I can see, you are ignoring this information. In this context, you function always succeeds regardless if actual upload succeeds or fails. The information of the error pointing the reason of an upload fail is lost too. That's why you never get any error message and your 'success message' always appears. 😉 Be more careful.
Hope this gives direction.
@fangzefunny wrote:...
At the end of the experiment, the success message was always prompted. But the data was not always uploaded to my dropbox.
...
... var saveDropbox = function(content, filename){ dbx.filesUpload({ path: "/" + filename, contents: content }) }; ......
Hi @fangzefunny,
In your 'saveDropbox' function, what are you doing with Dropbox 'filesUpload' method result? On success there should be received a file metadata to the file just uploaded and on error - the corresponding error information. As far as I can see, you are ignoring this information. In this context, you function always succeeds regardless if actual upload succeeds or fails. The information of the error pointing the reason of an upload fail is lost too. That's why you never get any error message and your 'success message' always appears. 😉 Be more careful.
Hope this gives direction.
@fangzefunny Здравко is correct; you should check the response or error for the call, which you are currently ignoring. You can find an example of handling the response and error for an upload call with the Dropbox API v2 JavaScript ....
Thanks! I found the reason today.
Sometimes, people make too many mistakes, making the (adaptive) experiment too long, and that causes the data file being too large. The large file cannot pass through the "post" operation, causing the problem of 413.
I was not able to see the error message before. I run it online, so I cannot see others' error messages. When I did it locally, I responded well, so the issue did not occur.
Anyway, thanks! It is not a problem with Dropbox!
@fangzefunny wrote:... The large file cannot pass through the "post" operation, causing the problem of 413. ...
Hi @fangzefunny,
For large files consider using upload sessions instead, not single upload. Every upload transaction is guaranteed to pass for no more than 150MB. In a session you can split the file content on pieces 150MB each, at most. In such a way you can upload significantly larger file. 😉
Good luck.
Got it! Thanks for great suggestions!
That's correct; the same example also shows how to use upload sessions for large files.
Hi there!
If you need more help you can view your support options (expected response time for a ticket is 24 hours), or contact us on X or Facebook.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!