script to transfer files from remote server directly to Dropbox

[daveh0]

Hi - I have a script currently running on an Amazon Lightsail instance that copies files from 1 remote server ("remote Server 1") to a different remote server ("Remote Server 2") using a simple SCP command. Beyond accessing the files from Remote Server 1 and copying them to a specified location on Remote server 2, I do not have access to either of the remote servers... which is why I'm running the script from the AWS instance. Also, the request to Remote Server 1 must also be coming from a white-listed, dedicated IP otherwise I would just run this script from my home computer.

Requirements have changed and now Remote server 2 needs to be a Dropbox location. I can see in the API documentation that I can copy LOCAL files directly to Dropbox, but I'm wondering if that PATH parameter can specify an SSH connection to another remote server??

So instead of:

curl -X POST https://content.dropboxapi.com/2/files/upload \
    --header "Authorization: Bearer " \
    --header "Dropbox-API-Arg: {\"path\": \"/Homework/math/Matrices.txt\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" \
    --header "Content-Type: application/octet-stream" \
    --data-binary @local_file.txt

it would be something like

curl -X POST https://content.dropboxapi.com/2/files/upload \
    --header "Authorization: Bearer " \
    --header "Dropbox-API-Arg: {\"path\": \"me@Remote-server_1:/Homework/math/Matrices.txt\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" \
    --header "Content-Type: application/octet-stream" \
    --data-binary @local_file.txt

Does anyone know if something like this is possible or would I have to copy the files from Remote Server 1 to the ASW instance and then use the API to move them from the AWS instance to Remote Server 2?

[Greg-DB · Accepted solution]

The Dropbox API itself doesn't offer a way to get files directly from SSH connections like this, but to clarify, note that the "path" parameter you're referring to is how you specify where in the connected Dropbox account you want to put the uploaded file. It is not how you specify where the actual file data to upload comes from.

You send the file data you want to upload to the Dropbox API /2/files/upload endpoint in the HTTPS request body. In this curl example, that's done via curl's '--data-binary' option. In this sample, you're telling curl to upload the data from the local file with the name "local_file.txt" from the current directory.

You might be able to connect scp or ssh to curl and pipe the data directly through to the Dropbox API endpoint, though I don't have a sample of how that might work. You'd need to refer to the ssh, scp, and/or curl documentation for more information.

[daveh0]

Gotcha. Thanks for the correction regarding Path parameter - temporary moment of absent-minded-ness - I'm glad you got the point anyway. Working on my script now - I'll post the solution when it hits production.