OTA update fails in CC3200

@Здравко Thank you so much for recreating the issue.

Greg-DB Please find my new packet capture in the google drive link. Please let me know if it works

https://drive.google.com/file/d/1v4dUI2lMIRJ-1Qg-u2Phl10IhSpVuCJr/view?usp=sharing

Regards,

Sundar

sundares80 Thank you! We'll look into it.

sundares80 The team is asking if you could also share some sample IP addresses and corresponding timestamps for failed requests so they can try to correlate this. Thanks in advance!

Greg-DB, There is an easy way to reproduce the same (ok maybe not the same, but almost the same) on a fetching from dropbox.com. Your main site is affected too.

First determine the host IP, next prepare traffic dump and at the end just fetch it. Follow the next sequence:

host www.dropbox.com
sudo tcpdump -v host IP_FROM_PREVIOUS_COMMAND -w dropbox.pcap &
DUMPID=$!
curl -v https://www.dropbox.com/ 2> dropbox.http > /dev/null
kill $DUMPID

 In advance close all browser tabs that may affect the records (i.e. with open Dropbox sites) or other tools.

Now you should have complete encrypted and unencrypted records to the same traffic. 🙂

dropbox.pcap - encrypted,

dropbox.http - unencrypted.

You can do the same while calling some API.

Hope this helps.

Здравко Thanks for sharing this! I'll share this with the team.

Also, can you clarify why the Encrypted Alert in this case is problematic? From my understanding, TLS Alerts like this can be expected in standard use of the TLS protocol. For example, they can be used to close the connection after successfully sending data. It would be helpful if you could elaborate on why the alert would be wrong in this case and how you're checking that. Thanks!

Hi Greg,

As per TI, this extra TLS alert stopping the client code execution. Could you please remove this.

Regards,

Sundar

---

Greg-DB wrote:

..., TLS Alerts like this can be expected in standard use of the TLS protocol. For example, they can be used to close the connection after successfully sending data. ...

---

Definitely NO!

Such a message (code 21) means incorrect encryption! Yes, it's known that many servers misbehave and most SSL/TLS libraries are ignoring this error and treat such as a break (as you said - even more, it's something mandatory with such a code), but this is definitely not something that should be expected. This is rather known bug in many implementation, but definitely not something that should be treated as "expected". The connection should be canceled exactly because the state is unexpected! By the way, most present day web servers (and not only) have corrected this issue; servers like yours are rather exceptions at present.

---

sundares80 wrote:

...

As per TI, this extra TLS alert stopping the client code execution. ...

---

😁 Hi sundares80,

This is pure try to excuse their own inability to handle such cases. Such cases are something that should be expected in the code and handled appropriately! Usually falling in any unexpected state (including the one discussed here, but not only) should lead to connection break and recreation, definitely not confusion! 😉

Hi Здравко,

We are in a critical situation. We have installed 100K IOT devices throughout the US. We cannot access the device now. The only thing we can do is perform an OTA update, which is currently not working due to your changes. Please remove this TLS alert message for 6–8 months. During that period, we will do an over-the-air update and fix this issue from our side.

Regards,

Sundar

Здравко Can you clarify where/how you're seeing code 21 exactly and what type that is? I initially thought I had reproduced an AlertDescription 21, but I believe what I saw was actually just a record ContentType 21, which just means "alert", not a specific alert. In that case, it wouldn't necessarily indicate incorrect encryption or not. The actual alert content was still encrypted.

I was able to configure curl with SSLKEYLOGFILE and point Wireshark at the file so that it could decrypt the alerts though, in which case when I performed a /2/files/get_temporary_link Dropbox API call, I only saw TLS records with Content Type 21 and Alert Description 0, which is just Close Notify. I did not observe any Alert Description 21.

Hi Greg-DB,

Yes, I may got confused about the exact code. I tried to reproduce it right now - unsuccessful.

There is other strange situation though. Dropbox server refuses to renegotiate! Despite this activity support is optional, it's highly recommended (according to RFC 5246). About Client Hello:

---

When this message will be sent:

   When a client first connects to a server, it is required to send

   the ClientHello as its first message. The client can also send a

   ClientHello in response to a HelloRequest or on its own initiative

   in order to renegotiate the security parameters in an existing

   connection.

---

About "TLS protocol issues":

---

While renegotiation is an optional feature, supporting it is
highly recommended.

---

When some client tries renegotiation, Dropbox response is:

Received Record
Header:
  Version = TLS 1.2 (0x303)
  Content Type = Alert (21)
  Length = 18
    Level=fatal(2), description=no renegotiation(100)

 Hm...🤔 According documentation, such a message should be warning, not fatal!  Again, RFC 5246 states:

---

no_renegotiation

   Sent by the client in response to a hello request or by the server

   in response to a client hello after initial handshaking. Either

   of these would normally lead to renegotiation; when that is not

   appropriate, the recipient should respond with this alert. At

   that point, the original requester can decide whether to proceed

   with the connection. One case where this would be appropriate is

   where a server has spawned a process to satisfy a request; the

   process might receive security parameters (key length,

   authentication, etc.) at startup, and it might be difficult to

   communicate changes to these parameters after that point. This

  message is always a warning.

---

This is a step aside from OP, but I just discovered it and that's why I'm mentioning. Such a message doesn't let a chance to client to decide whether to proceed further (without renegotiation) or not! This is not an issue in particular situation since there is no preceding handshake in the shared pcap, so that alert cannot be such a type (despite no way to be certain since encrypted).

There is another thing that may be interesting in context of OP. As can be seen (if we assume the alert is "close notify"), it comes from server side, not from client side (as it usually does). Such thing is possible only when in the request is header "Connection: close" or I was not able to reproduce it in other situation at least. So to be possible confirmation or rejection of this supposing (inappropriate header), sundares80  should send plain request to pseudo-server (like netcat, for instance) and see if there is some unintentional header. I suppose wouldn't be possible SSLKEYLOGFILE usage since most probably the application doesn't support it.

Здравко Thanks for writing that up! I'll pass it along to the team.

---

sundares80 wrote:

... The only thing we can do is perform an OTA update, which is currently not working due to your changes. ...

---

Hi sundares80,

I don't know what do you mean with "your changes", but I'm a regular Dropbox user, like you, and complain about different imperfections (very often bugs), the same like you do. I can "remove" something as much as you can - neither more nor less. Following this context line, did you check your code for bugs? 🧐 I'm already pretty sure there are such that lead to the result you're observing. You can model the behavior of your device (and your code accordingly) with following curl command:

TOKEN="PUT YOUR ACCESS TOKEN HERE"
NL=$'\r\n'
curl -v https://api.dropboxapi.com/2/users/get_current_account -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -H 'content-length: 4' --http1.1 -d "null$NL" -: https://api.dropboxapi.com/2/users/get_current_account -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -H 'content-length: 4' --http1.1 -d "null$NL" > /dev/null

The command curl is more stable and continue work, but the same "noise" in the communication may be observed. Comment this with your TI support if you want. Or rather if they want to do such a commenting with you... 😁 Would be pretty inconvenient to them!

Good luck.

PS: sundares80 don't use their buggy libraries, but rather some third party library or reimplement the HTTP communication yourself to passaround current bugs.

ЗдравкоThank you for your analysis. Yes, we agree. We have to redesign our HTTP client library. but I cannot access the physical IOT device since it is installed in multiple locations in the US and Europe. The only way I can update the client code is by doing an OTA update.

Greg-DB/Здравко

As per the packet capture, our IOT device support below chipers.

When I scan drop box server with below link, I see only warning message (Weak cipher) for one cipher which is used by the device. If you help to change it to green, I belive this issue is going to resolve.

https://www.ssllabs.com/ssltest/analyze.html?d=api.dropbox.com&s=162.125.8.19

Regards,

Sundar 

sundares80, I don't want disappoint you, but the cipher in use has nothing to do with your issue. Even more: Any server cannot select something client hasn't provided! If you count that as some possible other issue - it's your responsibility.

sundares80 While that SSL Labs SSL Server Test tool is not made by Dropbox and I am not an authority on it, my understanding is: The cipher being marked orange instead of green only indicates that the cipher is considered "weak", and does not indicate anything about the the server's use of the cipher. The inclusion of the cipher in that list at all indicates that the server supports that cipher. That "weak" is a description of the cipher itself, not the server configuration.

That being the case, it's not possible for us to change that cipher from orange to green. We could only remove it from that list for our servers entirely.

Actually there is a way to be solved discussed issue. The existence of empty lines is not something defined in HTTP standard, but also it's not something denied (not explicitly at least). 😉 Many present day servers also ignore such extra blank symbols, if any, at the request end. I don't know if Dropbox may decide to do the same, but if so that wouldn't be an issue anymore. (No more Bad request response - provoked by this empty line - directly following OK response)

Good luck.

Здравко I can reproduce the 400 Bad Request using your code snippet, but can you clarify how you determined that this "Bad Request" issue is the cause of the problem originally reported in this thread? I don't see "Bad Request" or "400" mentioned in this thread before. The original poster seems to have reported the issue as being in the TCP/IP or TLS layer instead. Thanks!

Greg-DB, as you mentioned too (and now I'm pretty sure you're right) the alert is "Close Notify"! A look on the sources shows that there is no way TI client to avoid adding one more empty line at the end... modeling only confirms it. That's it.

Experimenting with other sites shows that most sites ignore additional blanks after requests.

Of course this reflects TLS traffic since TLS wraps HTTP. 🤷

Add:

---

Greg-DB wrote:

... I don't see "Bad Request" or "400" mentioned in this thread before. ...

---

I'm a bit confused of above. 🤔 Did you expect to see it? If you have seen something similar somewhere it had definitely not been encrypted. 😉 Let's recall what we are talking about is encrypted piece of data! That's why it's difficult to discover such a thing on first look.

Thanks!

sundares80 Would you be able to modify a device you have access to in order to have it not send the additional new line as a test? If you can verify that fixes the issue, I can ask the team to look into changing the server to allow that (though please note I can't make any promises on that).

---

Sundaresan Thangeswaran1:

Hi Kobi,

Dropbox says they fixed "new line" issue. Could you please check if that issue resolves our OTA issue without code changes on the device. https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/OTA-update-fails-in-CC3200/m-p/771280#M33736

Regards,

Sundar

---

---

Kobi Leibovitch TI__Guru:

I'll be able to check next week. Is it regarding a "new line" within the HTTP headers or at the end of the payload?

Have you checked that you can now complete the update?

---

---

Sundaresan Thangeswaran1:

Hi Kobi,

I don't know where he wants to add a line.

They have not updated the changes in production yet. Anyway, I tested today, and it is not working.

Below is their answer.

"Would you be able to modify a device you have access to in order to have it not send the additional new line as a test? If you can verify that fixes the issue, I can ask the team to look into changing the server to allow that (though please note I can't make any promises on that)"

Please let us know if this fix our issue.

Regards,

Sundar

---

---

Kobi Leibovitch TI__Guru:

so need to better understand where they expect the extra new line.

again, i'll be able to check this only next week (but you can try yourself as the SDK contains all the sources and maefiles.

---

OMG 🤣😂, What kind of comedy!!! 😁 Some comedy actors may left without engagement. Here it's not a play, it's something native.

sundares80 To clarify, we have not made any changes on the Dropbox servers to address this. Здравко helpfully identified a potential cause of the problem, which is that your devices may be sending an unexpected new line at the end of the request body payload. In order to validate if that is the cause of the issue, and if addressing that would resolve the issue for you, I asked you to change one of your devices to make it not send that new line and see if it then works. If so, that would validate that as the cause of the issue so that I could then request that the team address that server-side.

Hi Greg,

Thank you for the clarification.

I have posted your message in the TI forum

I have asked to remove the extra line and test the issue.

Regards,

Sundar

Hi Greg,

TI confirms that removing the new line resolves the issue. 

https://e2e.ti.com/support/wireless-connectivity/wi-fi-group/wifi/f/wi-fi-forum/1337191/cc3200-ota-error--6-for-dropbox/5203134#5203134

Once you make those changes in production, our devices will do an OTA update without any code modification on the client side. Please correct me if I am wrong.

Regards,

Sundar