listFolder with includeMediaInfo not listing all available files

[ramonpoca]

We have a problem while integrating the v2 iOS API on our photography App.

If we call listFolder for the root folder with includeMediaInfo set to @yes, fewer results are returned (e.g. recently batch uploaded photos are not included in the response).

If we set includeMediaInfo:@NO, all files are listed.

[Greg-DB]

If you're using /files/get_metadata with include_media_info=true, you should get 'pending' media information. I just tried it, and it is working for me. Can you share the HTTP request and response for an API call like that that isn't working for you? Thanks in advance! 

Also, the API is designed to show pending media info from /files/get_metadata but not /files/list_folder. The former can be accessed on demand, but the latter is only supposed to return one entry per file, so they are deferred until all of the requested (media) information is available. I'll send this along as a feedback though, to see if we can do this in a different way in the future.

In any case, the delayed files/folders appear to be due to an issue on our side. We're looking into it, and will follow up here one we have an update.

[De L.1]

@Greg-DB thanks for your response and for looking into this issue. I really hope that you can figure out soon what stopped the media information processing!

You also explained nicely why /files/list_folder is responding differently than /files/get_metadata. However, it doesn't really make sense to us. We think that /files/list_folder should always return all files and folders on the Dropbox even if the media information isn't yet processed. The response definitely shouldn't be deferred until everything is known (we actually haven't seen any cloud API doing this).  In our humble opinion this isn't a 'Feature request' but just an essential part of this kind of API. Let's give an example:

Lets say that we have a Video App where new videos are uploaded for our clients. Our clients use the App to see and use those files. When we upload new media files, the client is informed that new files are added. However, when the clients App wants to see the new files via a /files/list_folder/continue call with a curser obtained from /files/list_folder/get_latest_cursor with include_media_info = true, it doesn't return any of the newly added files. Now the client has to wait (even "on the scale of hours") till all media information is processed before it can see any of the new files via this call.

We are not sure how we are supposed to solve this situation. The App is able to cope with files having the media information pending perfectly well.  We just try to get this information later on for files missing this information, and in the mean time we deal with the dimensions and duration in another way. But the app has to receive this pending tag to know that there is something there. As far as I can see, the only solution we currently have, is to use a /files/list_folder/get_latest_cursor call with include_media_info = false, and do a seperate /files/get_metadata call on each and every media file returned by /files/list_folder/continue to see the status of the media information. 

What is your view on this?

[Janne L.]

Hi!

I'm having similar issues. For example I have few files in 'Camera Uploads' , oldest pics and videos are ~ 3 days old. Here are some Python examples.

Media info not included:

>>> result = dbx.files_list_folder('/Camera Uploads')

>>> print len(result.entries)

12

Media info included:

>>> result = dbx.files_list_folder('/Camera Uploads', include_media_info=True)
>>> print len(result.entries)
0

If I target one file at a time:

>>> dbx.files_get_metadata('/Camera Uploads/2017-06-09 17.25.40.jpg', include_media_info=True)

..

media_info=MediaInfo(u'pending', None)

..

The media info is still pending after three days? It seems that there some issues with your media metadata system? API v1 was much more faster in this case, almost instant.

[ramonpoca]

We've implemented a fix by double-querying listFolder first with no mediaInfo and next with mediaInfo and "merging" the mediaInfo obtained on the second request, if any.

This is obviously not optimal neither for us nor for the API server.

In related news, a JPEG file I uploaded on friday via batch upload still has no metadata. Looks like a problem in the metadata processing backend?

[cdemiris99]

Thanks for responding.

We are using /files/list_folder, so that explains why I'm not seeing them.  It hasn't been a real issue in the past as long as files were being processed in a reasonable amount of time (and hopefully once this is fixed it we'll be ok for now).  

However, it seems inefficient if I now have to do list_folder without the metadata then have to query individually for each file to get metadata (or merge two separate list_folder calls like another user mentioned), so I really do hope it can be changed in the future to have list_folder simply provide the same list regardless of the include_media_info flag, and simply provide media info if available.

[Greg-DB]

It looks like we should be able to get this fixed today. I'll follow up here once I have an update.

And thanks for your feedback on the difference between get_metadata and list_folder. I'm sending it along to the team.

Your solution of using get_metadata in addition to list_folder with include_media_info=false would work. You can also keep two different list_folder cursors, one with include_media_info=false and include_media_info=true and track them both if you'd prefer.

[Greg-DB]

The team worked on a fix, but unfortunately it looks like it will take a little longer to get deployed than originally anticipated. The fix should be live tomorrow.

[Greg-DB]

This is fixed now. Calls to list_folder/continue will now return the latest entries. Please let us know if you're still seeing any issues.

[cdemiris99]

Working great for me in first tests. Thank you very much!

[BranchSolutions]

Sorry, but for me the API isn't working.

I've uploaded a new file 1 hour ago and is still missing from file list (it's the only file present in the folder).