Your workflow is unique 👨‍💻 -  tell us how you use Dropbox here.

Forum Discussion

SimoneBuzzi's avatar
SimoneBuzzi
Explorer | Level 4
8 years ago

Out of memory with java sdk during uploading (unlimited upload buffer)

Hi,   I'm tring to use the java sdk to move file from pc to dropbox space. I've written a method that upload file without problem while they are sufficently small (50MB), but generate a java.la...
API
Greg-DB's avatar
Greg-DB
Icon for Dropbox Community Moderator rankDropbox Community Moderator
8 years ago
Thanks for the report! I don't believe the buffer is configurable, but we'll be happy to look into this issue.

First, what version of the SDK are you using? If you're not already using the latest version (currently v3.0.3) please update to that and try again.

If you're still seeing this issue on the latest version, please let me know, and share the code to reproduce the issue. Thanks in advance!
SimoneBuzzi's avatar
SimoneBuzzi
Explorer | Level 4
8 years ago

Hi Greg,

 

you can find a sample project that show the same problem here

https://www.dropbox.com/s/54b86x1hgxpup99/dropboxProblem.zip?dl=0

 

The error generated is 

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2271)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:78)
at com.dropbox.core.util.IOUtil.copyStreamToStream(IOUtil.java:59)
at com.dropbox.core.util.IOUtil.copyStreamToStream(IOUtil.java:68)
at com.dropbox.core.util.IOUtil.copyStreamToStream(IOUtil.java:43)
at com.dropbox.core.http.HttpRequestor$Uploader.upload(HttpRequestor.java:98)
at com.dropbox.core.DbxUploader.uploadAndFinish(DbxUploader.java:95)
at com.dropbox.core.v2.DbxUploadStyleBuilder.uploadAndFinish(DbxUploadStyleBuilder.java:92)
at it.xxx.cloudLib.CopyToDropBox.upload(CopyToDropBox.java:45)
at it.xxx.cloudLib.CopyFile.main(CopyFile.java:17)

 

I'm using the last SDK version as you can find in the pom.

During my test I have experienced the same issue using one of 2.x.x version

 

I'm experiencing the problem using

Linux XXX 3.2.0-126-generic #169-Ubuntu SMP Fri Mar 31 14:15:21 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

 

Java

java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)

I'm compiling for 1.7 version

 

I obtain the error trying to uploading a file of 1.1 GB, I know that the maximum size for a file is 5GB, do I hope I can use the SDK till this line

 

Thanks for your help

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Community Moderator rankDropbox Community Moderator
    8 years ago
    Thanks! That's helpful. We'll look into it.
    • Greg-DB's avatar
      Greg-DB
      Icon for Dropbox Community Moderator rankDropbox Community Moderator
      8 years ago

      I just took a look over your last reply and the supplied code, and I notice you mentioned you're trying to upload a 1.1 GB file, but you're using UploadBuilder. For large files, that is, anything bigger than 150 MB, you need to use "upload sessions", per the documentation for uploadBuilder. (You may even want to use upload sessions for files smaller than 150 MB, depending on the scenario.)

       

      There's an example of using upload sessions here. For reference, using this method, there isn't an overall file size limit. You can upload files of any size, including larger than 5 GB. (Note that this only applies to the Dropbox side. Your local filesystem may have its own limit.)

       

      • SimoneBuzzi's avatar
        SimoneBuzzi
        Explorer | Level 4
        8 years ago
        Thank you for your help.

        I've created a method that analyzes the file and decides which method to use.
        I think that create a single method that implement this logic could be usefull also for other.
        Do you see any limitation to this approach?

About Dropbox API Support and Feedback

Node avatar for Dropbox API Support and Feedback
Get help with the Dropbox API from fellow developers and experts.

The Dropbox Community team is active from Monday to Friday. We try to respond to you as soon as we can, usually within 2 hours.

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X, Facebook or Instagram.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!