cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Want to learn some quick and useful tips to make your day easier? Check out how Calvin uses Replay to get feedback from other teams at Dropbox here.

Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Re: Dropbox API not returning all files

Dropbox API not returning all files

pyraxic
Helpful | Level 6
Go to solution

I am using Django and trying to get the list of all files. My dropbox contains 4149 files but it only returns 3998. Upon checking it seems that the most recent files uploaded to Dropbox isn't showing up in the list. Here is my code.

 

    dbx = dropbox.Dropbox(oauth_result.access_token)
    metadata = dbx.files_list_folder("", recursive=True)
    flist = []
    if metadata.has_more == True:
        m1 = metadata.entries
        cur = metadata.cursor
        for i in m1:
            if isinstance(i, dropbox.files.FileMetadata):
                flist.append([i.name])
        m2 = dbx.files_list_folder_continue(cur)
        while m2.has_more == True:
            for i in m2.entries:
                if isinstance(i, dropbox.files.FileMetadata):
                    flist.append([i.name])
            cur = m2.cursor
            m2 = dbx.files_list_folder_continue(cur)
1 Accepted Solution

Accepted Solutions

Greg-DB
Dropbox Staff
Go to solution

I tried running the code you posted and was able to reproduce the issue of it dropping the last page. Here's a fixed version:

 

dbx = dropbox.Dropbox(oauth_result.access_token)
result = dbx.files_list_folder("", recursive=True)
file_list = []

def process_entries(entries):
    for entry in entries:
        if isinstance(entry, dropbox.files.FileMetadata):
            file_list.append([entry.name])

process_entries(result.entries)

while result.has_more:
    result = dbx.files_list_folder_continue(result.cursor)

    process_entries(result.entries)

print(len(file_list))

View solution in original post

9 Replies 9

Greg-DB
Dropbox Staff
Go to solution
For reference, where/how are you getting the 4149 number from?

Does it include folders by any chance? In your code, you're only picking out files. Folders would be `FolderMetadata`.

Also, note that it looks like your code wouldn't list any files if the account happens to only have one page worth of entries. I.e., it is possible for the first response's `has_more` to be `False`, in which case you wouldn't get to your `for` loop.

pyraxic
Helpful | Level 6
Go to solution

I am getting 4149 from dropbox web gui. I only have one folder called "/Camera". I selected all files and it shows that there are 4149 files in that folder. Even if I try to use the code

 metadata = dbx.files_list_folder("/Camera", recursive=True)

 It still shows 3998 files.

Greg-DB
Dropbox Staff
Go to solution
Thanks! I just took another look at your code, and it looks like it actually won't process the last page of results. That is, the last call to `files_list_folder_continue` will cause `m2.has_more` to be `False`, which will terminate your `while` loop before that final `m2.entries` is processed. You'll just need to restructure the code to avoid that. (E.g., only put `files_list_folder_continue` at the top of `while` loop, or something to that effect.)

pyraxic
Helpful | Level 6
Go to solution

Hi Greg,

 

files_list_folders_continue() is already outside of the while loop.

 

m2 = dbx.files_list_folder_continue(cur)
        while m2.has_more == True:
            for i in m2.entries:
                if isinstance(i, dropbox.files.FileMetadata):
                    flist.append(i.name)
            cur = m2.cursor
            m2 = dbx.files_list_folder_continue(cur)

Greg-DB
Dropbox Staff
Go to solution
I'm referring to the second call to files_list_folder_continue at the end of the while loop.

pyraxic
Helpful | Level 6
Go to solution

Even putting it outside of while loop didn't make any difference.

 

        m2 = dbx.files_list_folder_continue(cur)
        while m2.has_more == True:
            for i in m2.entries:
                if isinstance(i, dropbox.files.FileMetadata):
                    flist.append(i.name)
            cur = m2.cursor
        m2 = dbx.files_list_folder_continue(cur)
    return render(request, 'accounts/dropbox.html', {'flist': flist})

pyraxic
Helpful | Level 6
Go to solution

I even changed that to outside of loop but same results. It doesn't seem to be getting to the while loop at all.

Greg-DB
Dropbox Staff
Go to solution

I tried running the code you posted and was able to reproduce the issue of it dropping the last page. Here's a fixed version:

 

dbx = dropbox.Dropbox(oauth_result.access_token)
result = dbx.files_list_folder("", recursive=True)
file_list = []

def process_entries(entries):
    for entry in entries:
        if isinstance(entry, dropbox.files.FileMetadata):
            file_list.append([entry.name])

process_entries(result.entries)

while result.has_more:
    result = dbx.files_list_folder_continue(result.cursor)

    process_entries(result.entries)

print(len(file_list))

pyraxic
Helpful | Level 6
Go to solution

Thank you!! It works.

Need more support?