I recommend trying out the Embedder to see if it works for your use case. That would be the safest way to do this. That works by supplying it a shared link to the folder you want to display.
Otherwise, you would need to use the API, which would require a refresh token for this scenario. Like the now-deprecated long-lived access tokens, a refresh token allows long-term access without explicit re-authorization. With the refresh token, there is just the additional programmatic step of getting a new short-lived access token whenever needed. This isn't really recommended though since /2/files/list_folder[/continue] require an access token and refresh/access tokens aren't meant to be distributed to other users.
For our usecase we want to include the files in our own UI and also access subfolders of the shared folder based on some custom logic.
The folder structure in dropbox may look like this:
then based on some logic we want to display the contents of e.g. /b/bb and /a/aaa.
I'm afraid that is not possible using the Embedder.
You write that making refresh tokens publicly available is not the way they meant to be used. Do you see any security issues in this regard? I mean the information is publicly available anyways. A user could use the token to query the API on his behalf but all he would see is the public information.
If everything in the connected account accessible to the app is public anyway, that would allay concerns about data leakage at least. There may still be issues with exposing the ability to perform other API calls, such as uploading files, but that could be minimized by restricting the refresh token (and consequently the access tokens) to the 'files.metadata.read' scope only (which is what /2/files/list_folder[/continue] require).
Note though that even then anyone with the token could still revoke it for everyone by calling /2/auth/token/revoke.
Additionally, by having multiple people using a refresh token for the same account, you'd be more likely to run in to the rate limiting system.