Bash Upload Script

[ThatBritishOne]

Hello,

I'm trying to create a script to for each over a folder, and upload to the API but I cannot figure out where I'm going wrong.

This is the script I have so far, any help would be appreciated 🙂

Spoiler

#!/bin/bash
UPLOAD_PATH="Backups"

if [[ -f '/root/dropbox.cfg' ]]; then
    echo 'It appears you have set an API Key for Dropbox already.'
    echo 'Using your configured API Key to proceed...'
    source /root/dropbox.cfg
else
    echo 'No existing API Key for Dropbox was found...'
    read -p 'Please provide an API Key to proceed ' api_key
    if [[ -z "$api_key" ]]; then
        echo "No key was provided. Exiting..."
        exit
    fi
    echo "DROPBOX_API_KEY=\"$api_key\"" >> /root/dropbox.cfg
    DROPBOX_API_KEY=$api_key
fi

read -p 'Please provide the path to the folder you wish to upload ' dest_path
if [[ -z "$dest_path" ]]; then
    echo "No path was provided. Exiting..."
    exit
else
    # Check if upload session exists
    if [[ -f "$dest_path/uploadsession.cfg" ]]; then
        echo "File exists using this upload session."
        source "$dest_path/uploadsession.cfg"
    # If it doesn't exist... create a session to be used
    else
        cd $dest_path
        touch uploadsession.cfg
        FIRSTFILE="$dest_path/uploadsession.cfg"
        curl -X POST https://content.dropboxapi.com/2/files/upload_session/start \
            --header "Authorization: Bearer $DROPBOX_API_KEY" \
            --header "Dropbox-API-Arg: {\"close\":false}" \
            --header "Content-Type: application/octet-stream" \
            --data-binary @${FIRSTFILE} >> /root/output.txt

        result=$(grep -oP '(?<="session_id": ")[^"]*' /root/output.txt)

        SESSION_ID=""
        IFS=':'
        for x in $result
        do 
            if [[ $x != "pid_upload_session" ]]; then
                SESSION_ID=$x
            fi
        done
        unset IFS

        echo "SESSION_ID=\"$SESSION_ID\"" >> "$dest_path/uploadsession.cfg"
        echo "Session ID = $SESSION_ID"
    fi

    # For each through each directory and folder, append to session
    for file in $dest_path/* $dest_path/**/*; do
        echo ${SESSION_ID};
        echo $file;
        curl -X POST https://content.dropboxapi.com/2/files/upload_session/append_v2 \
            --header "Authorization: Bearer $DROPBOX_API_KEY" \
            --header "Dropbox-API-Arg: {\"close\":false,\"cursor\":{\"offset\":0,\"session_id\":\"${SESSION_ID}\"}}" \
            --header "Content-Type: application/octet-stream" \
            --data-binary @${file} >> /root/output.txt
    done;

    # Finish Session
    curl -X POST https://api.dropboxapi.com/2/files/upload_session/finish_batch_v2 \
        --header "Authorization: Bearer $DROPBOX_API_KEY" \
        --header "Content-Type: application/json" \
        --data "{\"entries\":[{\"commit\":{\"autorename\":true,\"mode\":\"add\",\"mute\":false,\"path\":\"/${UPLOAD_PATH}\",\"strict_conflict\":false},\"cursor\":{\"offset\":0,\"session_id\":\"${SESSION_ID}\"}}]}"
fi<br />

[Здравко]

Hi @ThatBritishOne,

Seems you cannot understand the idea of different types of upload strategies. You can use a single call upload (seems most suitable for you) and using /2/files/upload you can upload all files, one by one. If some file is bigger than 150MB, then you have to use upload session. Upload session (/2/files/upload_session/start) is NOT for upload multiple files, but upload one file in pieces, every one piece 150MB at most. Be careful here. 🙂 Seems you are mixing the idea here! Other option is upload multiple files in batch (/2/files/upload_session/start_batch) that let you finish multiple files at once and decrease chance for call conflicts (possible on high load - multiple applications try access/change the same account content simultaneously). Hope this gives direction at least. Something you might be source of mistake is your 'DROPBOX_API_KEY'. What this actually means???!!! The name suppose application key, but you are using it as an access token! Take care. Take in mind that access token need refresh and you didn't implement any handling with refresh token. 😉 You may consider such option.

Good luck.

[ThatBritishOne]

Hello,

Thanks for the help, I managed to update the script so it for eaches over every file and uploads it.

The problem I'm having now is keeping the folder structure, as it uploads it as a file rather than creates a new folder each time.

Is there a way I can preserve the path to upload and create the folders automatically?

[Здравко]

---

@ThatBritishOne wrote:

...

Is there a way I can preserve the path to upload and create the folders automatically?

---

The API creates folder structure automatically - no folder(s) containing file need creation explicitly (only empty folders, if any, need explicit creation). Check what 'path' have you set for each entry (need to match actual relative path and the Dropbox path format rule).

[Greg-DB]

@ThatBritishOne Здравко is correct, you can control the folder hierarchy where you upload a file by using the "path" value. For example, if you want to upload a file named "newbackup" into a folder named "10-24-2022" which is inside a folder named "Backups", the "path" value should be "/Backups/10-24-2022/newbackup".