cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Do you have an idea for Dropbox Paper? Share it with us here!

Possible bug corrupting file names with certain unicode code points

ncw
Helpful | Level 5

An rclone user has discovered that sometimes file names appear to get corrupted.

See this rclone issue for full details.

I managed to reproduce this and capture an HTTP transaction.

First here is uploading a 0 length file named "Русский.txt"

 

2019/10/10 12:40:16 DEBUG : HTTP REQUEST (req 0xc000463900)
2019/10/10 12:40:16 DEBUG : POST /2/files/upload HTTP/1.1
Host: content.dropboxapi.com
User-Agent: rclone/v1.49.5
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: application/octet-stream
Dropbox-Api-Arg: {"path":"/tdir/Русский.txt","mode":{".tag":"overwrite"},"autorename":false,"client_modified":"2000-01-01T00:00:00Z","mute":false,"strict_conflict":false}
Accept-Encoding: gzip

0

And here is the response

2019/10/10 12:40:17 DEBUG : HTTP RESPONSE (req 0xc000463900)
2019/10/10 12:40:17 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json
Date: Thu, 10 Oct 2019 11:40:17 GMT
Pragma: no-cache
Server: nginx
Vary: Accept-Encoding
X-Dropbox-Request-Id: 3a11434e22b08d87588de53cb34c047b
X-Robots-Tag: noindex, nofollow, noimageindex
X-Server-Response-Time: 553

257
{"name": "\u00d0\u00a0\u00d1\u0083\u00d1\u0081\u00d1\u0081\u00d0\u00ba\u00d0\u00b8\u00d0\u00b9.txt", "path_lower": "/tdir/\u00f0\u00a0\u00f1\u0083\u00f1\u0081\u00f1\u0081\u00f0\u00ba\u00f0\u00b8\u00f0\u00b9.txt", "path_display": "/tdir/\u00d0\u00a0\u00d1\u0083\u00d1\u0081\u00d1\u0081\u00d0\u00ba\u00d0\u00b8\u00d0\u00b9.txt", "id": "id:qqS0afUHwS0AAAAAAAPHng", "client_modified": "2000-01-01T00:00:00Z", "server_modified": "2019-10-10T11:40:17Z", "rev": "5948cdd90ee8c082af73a", "size": 0, "is_downloadable": true, "content_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}
0

Note that the name in the response appears to be corrupted.

If you decode this name you get

>>> print("\u00d0\u00a0\u00d1\u0083\u00d1\u0081\u00d1\u0081\u00d0\u00ba\u00d0\u00b8\u00d0\u00b9.txt")
Ð ÑÑÑкий.txt

Which is now the name of the file in listings instead of "Русский.txt".

 

Since this capture is right at the HTTP level, I don't think it can be my code or the dropbox Go SDK, I think this must be a bug in the dropbox server code.

1 Accepted solution

Accepted Solutions

Re: Possible bug corrupting file names with certain unicode code points

Dropboxer

Thanks for the report! Sending non-ASCII characters in HTTP headers is not officially supported, so please make sure you're encoding any non-ASCII characters in headers as documented here:

https://www.dropbox.com/developers/reference/json-encoding

There was a change on our server stack yesterday that affected how we handled HTTP headers without proper encoding. That resulted in malformed file paths/names like the user reported in the GitHub issue. We've reverted that change, so that should be working again, but please make sure your headers get encoded properly.

View solution in original post

Reply
Loading...
6 Replies

Re: Possible bug corrupting file names with certain unicode code points

Star Collaborator | Level 19

Hi @ncw,

Take in account that http header content have to be ASCII-based and if need (as is in particular case) names (and not only) have to be JSON-encoded. JSON encoding is missing in your case. winking face

Hope this helps.

Reply
Loading...

Re: Possible bug corrupting file names with certain unicode code points

Dropboxer

Thanks for the report! Sending non-ASCII characters in HTTP headers is not officially supported, so please make sure you're encoding any non-ASCII characters in headers as documented here:

https://www.dropbox.com/developers/reference/json-encoding

There was a change on our server stack yesterday that affected how we handled HTTP headers without proper encoding. That resulted in malformed file paths/names like the user reported in the GitHub issue. We've reverted that change, so that should be working again, but please make sure your headers get encoded properly.

View solution in original post

Reply
Loading...

Re: Possible bug corrupting file names with certain unicode code points

ncw
Helpful | Level 5

Thanks for that.

It must be a bug in dropbox Go SDK  that the headers aren't being properly encoded

I'll report a bug there in a moment!

Reply
Loading...

Re: Possible bug corrupting file names with certain unicode code points

ncw
Helpful | Level 5

I made an issue in the Dropbox Go SDK repo which is where the problem needs to be fixed.

Reply
Loading...

Re: Possible bug corrupting file names with certain unicode code points

Explorer | Level 4

"We've reverted that change" - people with the Synology NAS are still having this problem.

Reported here: https://community.synology.com/enu/forum/1/post/129064

Reply
Loading...

Re: Possible bug corrupting file names with certain unicode code points

Dropboxer

@Hexrby Thanks for the information. If you're still seeing any broken functionality, please let the third party app developer know so that they can contact us with the relevant technical details so we can investigate.

Reply
Loading...
API Support

Have a question? Our Dropbox Community is here to help you find an answer!
You won’t have to wait too long for a reply, but why not learn more about the Community while you’re here.

Not the right discussion for you? Find out what else our Community has to say about API Support.

Or, search our Help Center for even more.

Who's talking
Top contributors to this post