Need to see if your shared folder is taking up space on your dropbox 👨‍💻? Find out how to check here.

Forum Discussion

Sadsyndrome's avatar
Sadsyndrome
Explorer | Level 3
4 years ago

Python Dropbox API slow performance

Hello

Im not a professional programmer..I recently built an app on python running on raspberry pi for monitoring energy usage at home.

it all works fine, I was using Dropbox app to save photo images temporarily. Images are approx 1mb saved every 30 mins and deleted once I have finished with them.

 

this worked fine for a few hours until my access token expired. After getting a refresh token to resolve that issue it was all working again. However the next day performance dropped for the simple upload of these image files and even a test text file. The upload takes in region of 3 .5 minutes regardless of size of upload or even reading the directory listing.

I assumed this was th refresh token process so downloaded a new access token, but even using that without refresh token the Dropbox ap still takes 3.5 mins for any action??

 

Any ideas what may cause this? As I said it was working fine until original access token expired and also for a period after I changed it for refresh token? Would it be rate limiting or something like that?

 

 

4 Replies

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Community Moderator rankDropbox Community Moderator
    4 years ago

    The access token/refresh token shouldn't have any impact on that. The Dropbox API does have a rate limiting system, but it works by responding with a rate limit error response to any call it needs to reject, not by slowing down the connection speed.

     

    Your connection speed to Dropbox depends on the routing you get between your ISP and our servers, and may be slower than your ISP's rated speeds. Sometimes resetting or retrying your connection gets you a different route and better speeds, but that is outside of our control. Some ISPs also throttle sustained connections so if you see an initial high connection speed followed by lower speeds, that could be the reason. Otherwise, there may be other local factors impacting the processing speed of your code, as well as the speed of your network connection. We can't offer help with your device or network connection though, so you may need to contact your computer and/or network admin, if any, for help with that.

     

    To debug this you may want to step through with a debugger, and/or add more logging. If you do find an issue with a specific Dropbox API call, feel free to show the details here so we can take a look. (Be sure to redact any access token, refresh token, or app secret though.)

  • Sadsyndrome's avatar
    Sadsyndrome
    Explorer | Level 3
    4 years ago

    Thx. I don’t think it’s a network error. I’m transferring data to google sheets with no issue. The upload file process takes 3.5 mins for a single text file 201 bytes. The time does seem to be pretty consistent at approx 3.5 mins regardless of file size. It worked perfectly yesterday. Even getting file metadata takes over 2mins? Is there anywhere I can look at the Dropbox end ? I don’t have an IDE I just use a text editor to knock up my python scripts and not sure where to start with debugging to be honest. I guess I could take a look at network monitor if I remember how! I’ve been out of the IT game for 4 years now and memory is fading:)

     

    here is script and  output. Sorry couldn’t see how to attach these so I put them in this message. It works (slowly) so I assume keys and python syntax ok 

    ##

    import dropbox
    from datetime import datetime
    mylocpath ='/home/mark/FTP/myfiles/'
    locfname='gasmeter.txt'
    dboxpath='/meterreadings/'
    file_from = mylocpath +locfname
    file_to = dboxpath+locfname

    dbx = dropbox.Dropbox(
    app_key = "mykey",
    app_secret = "mysecret",
    oauth2_refresh_token = "myrefreshtokeen"
    )
    f = open(file_from, 'rb')

    now = datetime.now()
    current_time = now.strftime("%H:%M:%S")
    print ("start dbx call Time =", current_time)

    dbx.files_upload(f.read(), file_to)

    now = datetime.now()
    current_time = now.strftime("%H:%M:%S")
    print ("end dbx call =", current_time)
    print ("done")

    ##

    mark@raspberrypi:~/pwrlog/test $ python dbupl.py
    start dbx call Time = 20:10:10
    end dbx call = 20:13:32
    done
    mark@raspberrypi:~/pwrlog/test $

     

     

     

    e = 19:25:12
    done

     

     

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Community Moderator rankDropbox Community Moderator
    4 years ago

    The route you get to the Dropbox servers may be different than the route to get to the servers for other services, so a comparison with other services may not be applicable.

     

    Anyway, Dropbox does not offer the ability to debug this from the Dropbox servers. You'd need to do any debugging on your side.

     

    Looking at the code itself, I don't see any issues. I just tried this script myself, plugging in my path and key/secret/refresh token values, and the script ran in 2 seconds to successfully upload a 1 MB file.

     

    I see you're using the Python SDK, so the following code would enable additional debugging output which might be useful:

    try:
        import http.client as http_client
    except ImportError:
        # Python 2
        import httplib as http_client
    http_client.HTTPConnection.debuglevel = 1
    
    import logging
    logging.basicConfig(level=logging.DEBUG)
    logging.getLogger().setLevel(logging.DEBUG)
    requests_log = logging.getLogger("requests.packages.urllib3")
    requests_log.setLevel(logging.DEBUG)
    requests_log.propagate = True
    
    import urllib3
    urllib3.add_stderr_logger()

     

  • Sadsyndrome's avatar
    Sadsyndrome
    Explorer | Level 3
    4 years ago

    Hi. Just a quick update in case it is useful.

    I reinstalled dropbox using pip install. 3.5 mins download persisted.

    rebooted rpi still no joy

    tested over a different network ( my smartphone hotspot) same problem on that network

    I ran tcpdump and saw the host name api.dropboxapi.com in amongst lots of traffic

    I pinged that from rpi but  dns response was ipv6 and no traffic flowed?

    I disabled ipv6 on rpi and problem is fixed???

    not sure why it worked yesterday?

    unless my changes have coincided with  something else?

    my api shows 1000 hits a day on analytics, I don’t understand that either???? I love technology:)

     

    thx for your prompt responses

     

     

     

     

     

About Dropbox API Support & Feedback

Node avatar for Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.

The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!