upload API only works for very small files

[ags65]

Hello,

I try this command from my raspberry pi 2 with raspbian buster:

curl -v -X POST https://content.dropboxapi.com/2/files/upload --header "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" --header "Dropbox-API-Arg: {\"path\": \"/README.md\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" --header "Content-Type: application/octet-stream" --data-binary @/home/pi/Dropbox-Uploader/README.md

and I get the error 52 Empty reply from server:

> POST /2/files/upload HTTP/1.1
> Host: content.dropboxapi.com
> User-Agent: curl/7.64.0
> Accept: */*
> Authorization: Bearer xxxxxxxxx
> Dropbox-API-Arg: {"path": "/README.md","mode": "add","autorename": true,"mute": false,"strict_conflict": false}
> Content-Type: application/octet-stream
> Content-Length: 8900
> Expect: 100-continue
>
* Expire in 1000 ms for 0 (transfer 0x1679880)
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Empty reply from server
* Connection #0 to host content.dropboxapi.com left intact
curl: (52) Empty reply from server

but the same command with a very small file works:

> POST /2/files/upload HTTP/1.1
> Host: content.dropboxapi.com
> User-Agent: curl/7.64.0
> Accept: */*
> Authorization: Bearer xxxxxxxxxxxx
> Dropbox-API-Arg: {"path": "/borrar.txt","mode": "add","autorename": true,"mute": false,"strict_conflict": false}
> Content-Type: application/octet-stream
> Content-Length: 5
>
* upload completely sent off: 5 out of 5 bytes
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sun, 01 Mar 2020 18:54:34 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< cache-control: no-cache
< pragma: no-cache
< X-Server-Response-Time: 823
< X-Dropbox-Request-Id: 6a837c20651b6d6f2f6498f45e297b3f
< X-Robots-Tag: noindex, nofollow, noimageindex
<

Thanks for any clue you can give me to help me.

[ags65]

interesting...

the cURL command works in httpbin.org and the data seems to be OK:

Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 34.230.193.231:443...
* TCP_NODELAY set
* Connected to httpbin.org (34.230.193.231) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=httpbin.org
* start date: Jan 18 00:00:00 2020 GMT
* expire date: Feb 18 12:00:00 2021 GMT
* subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
> POST /post HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.65.3
> Accept: */*
> Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Dropbox-API-Arg: {"path": "/Cromosoma.cc","mode": "add","autorename": true,"mute": false,"strict_conflict": false}
> Content-Type: application/octet-stream
> Content-Length: 2863
>
* upload completely sent off: 2863 out of 2863 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 06 Mar 2020 16:28:55 GMT
< Content-Type: application/json
< Content-Length: 4463
< Connection: keep-alive
< Server: gunicorn/19.9.0
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{
"args": {},
"data": "data&colon;application/octet-stream;base64,LyoqIEBma............g==",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Authorization": "Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"Content-Length": "2863",
"Content-Type": "application/octet-stream",
"Dropbox-Api-Arg": "{\"path\": \"/Cromosoma.cc\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}",
"Host": "httpbin.org",
"User-Agent": "curl/7.65.3",
"X-Amzn-Trace-Id": "Root=1-5e627a47-49885dec98b9e392fb837b9a"
},
"json": null,
"origin": "83.47.100.114",
"url": "https://httpbin.org/post"
}
* Connection #0 to host httpbin.org left intact

[Здравко]

Wow... If content of 'data' field match exactly to the posted file... hands up... I haven't more ideas. Isn't there something missing at the end? The content there is JSON-ASCII encoded! Decode it before compare if need (or opposite). For example, encode the file you passed using:

python3 -c "import json;import sys;f=open(sys.argv[1],'r');print(json.dumps(f.read()));f.close()" /home/pi/Dropbox-Uploader/Cromosoma.cc

And compare the result to 'data' field content above.

[ags65]

Yes, the content of "data" field match to the file:

https://httpbin.org:

 "data": "/** @file Cromosoma.cc\n    @brie...... << crom[i];\n    }\n    cout << endl;\n}\n",

print(json.dumps):

"/** @file Cromosoma.cc\n    @brie...... << crom[i];\n    }\n    cout << endl;\n}\n"

no missing characters at the end.

[Здравко]

I'm up to here. Probably @Greg-DB can propose some additional checks.

[Greg-DB]

Unfortunately I don't know what the issue may be here. We'll try to look into it, but I can't make any promises as it seems to be client-specific, based on the observation that it doesn't occur on your other computer.

[ags65]

Thank you Greg, I understand.

I will communicate if I make new advances.

[Greg-DB · Accepted solution]

@ags65 We looked into this, and it seems like you may be running in to this issue:

https://www.raspberrypi.org/forums/viewtopic.php?t=223026

As noted there, you can try:

To stop sending DFs, you can change net.ipv4.ip_no_pmtu_disc value to 1 in sysctl.
I would also try reducing the MTU on the interface in order to advertise a smaller MSS to the TCP peer.

[ags65]

Amazing, now it works !!!

The parameter ip_no_pmtu_disc does not seem to influence but I have lowered the value of the MTU from 1500 to 1200 and now the uploads work without problems

Thank you very much to both of you.