Start a new topic

.TFT Update, OTA (WiFi ESP8266) from remote Server.

Hello Everyone,


Further to efforts earlier this year in,


Firmware Update without Nextion Editor? http://support.iteadstudio.com/support/discussions/topics/1000062718/page/1?url_locale=


I had an idea to move this up a level or two, and try to implement an OTA solution that essentially could be fully automated. Freeing the end user from any business with device updates.


Today I can announce achieving the first leg of that goal. I've modified a sketch by Markus Sattler using his ESP8266HTTPClient class library. Many thanks to Markus for your excellent work.


Running on the ESP, we can call a remote Apache server for our new .tft file.

Then we poll an update connection to the Nextion Display over serial, and stream the new data. Data transfer speed is dependent on baud rate, the wifi appears to have no issues so far. After successful download, Nextion reboots to the new HMI. Simples (actually it wasn't :) 


I'll make the source code available.


Let's discuss....


Steve.


1 person likes this idea

Some obsevations,


The example source is here https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266HTTPClient/examples/StreamHttpClient/StreamHttpClient.ino


Further details here links2004.github.io/Arduino/dd/d8d/class_h_t_t_p_client.html#aee331934366e80758b190cdda7dcf556


I've spent hours doing tests, and the verification of Nextions 0x05 return bytes 'on the fly' has proved outside the scope of the method (for me anyway). I've found that any attempt to break out of the Serial.write 'while loop' to do Serial.read, breaks the Nextion upload irrecoverably. A whole 4k block of data goes awol !


On the whole this method works very reliably, after one or two refinements success is pretty much a given.


To emulate the 0x05 ack, I found it necessary to pause upload after ever 4k. Despite my dislike of using delay() I tested down to a minimum value of 250ms before it breaks again. I hope there is someone smarter than me that can figure something a little more refined.


Not to be totally out done by the lack of a Serial.read, it's a fact that the ESP retains a small incoming buffer (128 bytes I believe) regardless of what is going on elsewhere. I've used that to reasonable effect, by doing a read after the upload. First checking for a byte count (is it longer than 4k part count ?) then look for 0x88 as Nextion reboots. Errors here could be used to recall the whole method. For now I just have it as debug info.


Finally, this development thing is a whole lot easier if one implements the ArduinoOTA routine so you can reflash the ESP without having to disconnect the Nextion from Serial. I've tried Nextion on Software serial, but results are unsatisfactory. I use SoftwareSerial for debug instead :)


Source....I used a .ino extension in PlatformIO


This is a work in progress :) and only a demo of a greater whole. A method one could call on demand from the Nextion or a hardware button press or whatever. Use as you so desire.


  

/**
 * StreamHTTPClient.ino
 *
 *  Created on: 24.05.2015
 *
 */

#include <Arduino.h>
#include <ArduinoOTA.h>
#include <ESP8266mDNS.h>
#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include <ESP8266HTTPClient.h>
#include <SoftwareSerial.h>

SoftwareSerial SoftSerial(14, 12); // Rx on 14, Tx on 12
ESP8266WiFiMulti WiFiMulti;

void checkReply();
void Nextion_Update();

int FileSize;
String ackString;
int count = 0;
byte partNum;
int total;
int pCent;
byte ter [3] = {0xff, 0xff, 0xff};

void setup() {

    Serial.begin(115200);
    SoftSerial.begin(115200);

    SoftSerial.println();

    for(uint8_t t = 4; t > 0; t--) {
        SoftSerial.printf("[SETUP] WAIT %d...\n", t);
        SoftSerial.flush();
        delay(1000);
    }   

    WiFiMulti.addAP("SSID", "password");

    while ((WiFiMulti.run() !=WL_CONNECTED)) {
      SoftSerial.print(".");
      delay(500);
    }
    ArduinoOTA.setPort(8266);
    // Hostname defaults to esp8266-[ChipID]
    ArduinoOTA.setHostname("myesp8266");

    ArduinoOTA.onStart([]() { Serial.print(F("Start\n")); });
    ArduinoOTA.onEnd([]() { Serial.print(F("\nEnd\n")); });
    ArduinoOTA.onProgress([](unsigned int progress, unsigned int total) {
      SoftSerial.printf("Progress: %u%%\r", (progress / (total / 100)));
    });
    ArduinoOTA.onError([](ota_error_t error) {
      SoftSerial.printf("Error[%u]: ", error);
      if (error == OTA_AUTH_ERROR)
        SoftSerial.print(F("Auth Failed\n"));
      else if (error == OTA_BEGIN_ERROR)
        SoftSerial.print(F("Begin Failed\n"));
      else if (error == OTA_CONNECT_ERROR)
        SoftSerial.print(F("Connect Failed\n"));
      else if (error == OTA_RECEIVE_ERROR)
        SoftSerial.print(F("Receive Failed\n"));
      else if (error == OTA_END_ERROR)
        SoftSerial.print(F("End Failed\n"));
    });
    // ArduinoOTA.begin();
    Nextion_Update();
}

void Nextion_Update () {

        HTTPClient http;

        SoftSerial.print("[HTTP] begin...\n");
        // configure server and url
        http.begin("http://192.168.11.4/test.tft");//Modify for your Server/file 
        //http.begin("192.168.1.12", 80, "/test.html");
        SoftSerial.print("[HTTP] GET...\n");
        // start connection and send HTTP header
        int httpCode = http.GET();
        if(httpCode > 0) {
            // HTTP header has been send and Server response header has been handled
            SoftSerial.printf("[HTTP] GET... code: %d\n", httpCode);

            if(httpCode == HTTP_CODE_OK) {// file found at server
                // get lenght of document (is -1 when Server sends no Content-Length header)
                int len = http.getSize();
				        FileSize = len;
                int Parts = (len/4096) + 1;
                SoftSerial.printf("File found at Server. Size %u bytes. In %u parts.\n", len, Parts);
                // create buffer for read
                uint8_t buff[128] = {};
                // get tcp stream
                WiFiClient * stream = http.getStreamPtr();
                // read all data from server
                Serial.flush();
                SoftSerial.println("Contacting Nextion...");
                Serial.printf("%sconnect%s", ter, ter );
                delay(500);
                Serial.printf("whmi-wri %u,115200,0",FileSize);
                Serial.write(ter,3);
                delay(500);
                ackString = "";
                SoftSerial.print("Starting Download...\n");
                while(http.connected() && (len > 0 || len == -1)) {
                    // get available data size
                    size_t size = stream->available();
                    if(size) {
                        // read up to 128 byte
                        int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
                        // write it to Serial
                        Serial.write(buff, c);
                        count += c;
                        if (count == 4096) {
                          partNum ++;
                          total += count;
                          pCent = (total * 100) / FileSize;
                          count = 0;
                          delay (250);
                          SoftSerial.printf("Part: %u Bytes: %u Progress %u%%\n", partNum, total, pCent);                         
                        }
                        if(len > 0) {
                            len -= c;
                        }
                    }
                    delay(1);
                }
                partNum ++;
                delay (250);
                total += count;
                SoftSerial.printf("Last Part: %d Total Bytes: %d Progress 100%%\n\n", partNum, total);
                SoftSerial.printf("Size request: %d\n\n",FileSize);
                if (total == FileSize) SoftSerial.print("Server Byte count is Good !\n\n");
                else SoftSerial.print("Oh no, somethin' went wrong :(\n\n");                
                SoftSerial.print("[HTTP] connection closed or file end.\n\n");
            }
        } else {
            SoftSerial.printf("[HTTP] GET... failed, error: %s\n", http.errorToString(httpCode).c_str());
        }
        http.end();
        ArduinoOTA.begin();

}


void loop() {

  while (Serial.available()) {
    char inChar = (char)Serial.read();
    ackString += inChar;
    if (ackString.length() > partNum) {
      SoftSerial.printf("1.Nextion return Bytes: %d\n", ackString.length());
      ackString = "";
    }
    if (inChar == 0x88) {
      SoftSerial.println("2.Nextion Upload Success !");
    }
  }
  ArduinoOTA.handle();
} 

  


1 person likes this

If you get it working, here's a typical output you can expect from a SoftSerial terminal dispay...


It will help if you figure it through with the code above.

 

[SETUP] WAIT 4...
[SETUP] WAIT 3...
[SETUP] WAIT 2...
[SETUP] WAIT 1...
[HTTP] begin...
[HTTP] GET...
[HTTP] GET... code: 200
File found at Server. Size 190987 bytes. In 47 parts.
Contacting Nextion...
Starting Download...
Part: 1 Bytes: 4096 Progress 2%
Part: 2 Bytes: 8192 Progress 4%
Part: 3 Bytes: 12288 Progress 6%
Part: 4 Bytes: 16384 Progress 8%
Part: 5 Bytes: 20480 Progress 10%
Part: 6 Bytes: 24576 Progress 12%
Part: 7 Bytes: 28672 Progress 15%
Part: 8 Bytes: 32768 Progress 17%
Part: 9 Bytes: 36864 Progress 19%
Part: 10 Bytes: 40960 Progress 21%
Part: 11 Bytes: 45056 Progress 23%
Part: 12 Bytes: 49152 Progress 25%
Part: 13 Bytes: 53248 Progress 27%
Part: 14 Bytes: 57344 Progress 30%
Part: 15 Bytes: 61440 Progress 32%
Part: 16 Bytes: 65536 Progress 34%
Part: 17 Bytes: 69632 Progress 36%
Part: 18 Bytes: 73728 Progress 38%
Part: 19 Bytes: 77824 Progress 40%
Part: 20 Bytes: 81920 Progress 42%
Part: 21 Bytes: 86016 Progress 45%
Part: 22 Bytes: 90112 Progress 47%
Part: 23 Bytes: 94208 Progress 49%
Part: 24 Bytes: 98304 Progress 51%
Part: 25 Bytes: 102400 Progress 53%
Part: 26 Bytes: 106496 Progress 55%
Part: 27 Bytes: 110592 Progress 57%
Part: 28 Bytes: 114688 Progress 60%
Part: 29 Bytes: 118784 Progress 62%
Part: 30 Bytes: 122880 Progress 64%
Part: 31 Bytes: 126976 Progress 66%
Part: 32 Bytes: 131072 Progress 68%
Part: 33 Bytes: 135168 Progress 70%
Part: 34 Bytes: 139264 Progress 72%
Part: 35 Bytes: 143360 Progress 75%
Part: 36 Bytes: 147456 Progress 77%
Part: 37 Bytes: 151552 Progress 79%
Part: 38 Bytes: 155648 Progress 81%
Part: 39 Bytes: 159744 Progress 83%
Part: 40 Bytes: 163840 Progress 85%
Part: 41 Bytes: 167936 Progress 87%
Part: 42 Bytes: 172032 Progress 90%
Part: 43 Bytes: 176128 Progress 92%
Part: 44 Bytes: 180224 Progress 94%
Part: 45 Bytes: 184320 Progress 96%
Part: 46 Bytes: 188416 Progress 98%
Last Part: 47 Total Bytes: 190987 Progress 100%

Size request: 190987

Server Byte count is Good !

[HTTP] connection closed or file end.

1.Nextion return Bytes: 48
1.Nextion return Bytes: 48
2.Nextion Upload Success !

 

Note the Nextion is connected to the hardware Serial of the ESP.


The above output is obtained by using an FTDI or similar USB to serial adaptor to pins 14,12 (softwareSerial).


14 is ESP (soft) RX 

12 is ESP (soft) TX


In most cases we just need to connect TX 12 (D6 nodemcu) to see the output.

So according to the published spec


There is a maximum allowed 500ms between the 4K chunks for the Nextion to return that 0x05 byte.

I know that this delay is needed for the Nextion to complete processing of the 4K data chunk that it has received and process and place the data from the 4K to the SPI flash.  I am going to assume from behaviour, it is most likely received into SRAM and then burned into the appropriate 16 pages in SPI.

So this method of processing will create such a need for delay()


I am only getting bits of time here and there to work on the communication aspect.


Arduino also notes they have limitations in their SoftwareSerial library and point to another SoftwareSerial that while it overcomes some limitations found in Arduino's - it creates new ones of its own.


Hi Patrick,


For the most part I've used SoftwareSerial as the primary method for connecting Nextion.


And for the average user generating little traffic it works well. The main reason for going that route while in a development setup, is that hardware serial is needed for flashing the ESP.


However as I noted, shifting bulk data at 115200 over SoftwareSerial is prone to errors even when the byte count received on Nextion is correct. I don't think SoftwareSerial should even be run so fast, even though it works most of the time. That in itself creates another issue, should it have to be run at say 9600 to keep the transfer clean.


In a production device we wouldn't need SoftwareSerial anyway. Using hardware Serial gives reliable results at high speed (so far :).


In further experiments, I'll test some higher speeds, I think someone noted that while not documented, Nextion can handle quite a bit more. 

The STM chip could handle a much higher rate.  But the firmware is limited to 115200 baud.

But various conditions would be needed in theory - CTS/RTS would certainly be helpful.


It gets into a deeper specifics to a specific application.  So yeah, all of a sudden there is a separate lib if it is in this config with this exact hardware used for this purpose - and another lib with changes.


So the use of software serial bit banging two pins comes in handy.  The gist of software serial being used between two none matching clocks is one of timing.  Since there is no way to tell when the signal for the bit has actually started, it is assumed that if one samples from "what should be" center of a 8 or 16 over sampling, that we should be safe to say - that is our intended bit value.


Even with two matching chips, we are close to - but never exact.

The hardware USART on the STM32 is synchronous capable, but few use it - not implemented. So the asynchronous  USART is available, but we certainly aren't using any hardware flow control which requires 4 lines to include the RTS/CTS lines - not implemented - we only have the two RX/TX.  Then what library is being used in the firmware.  It is possible to use software serial on two hardware pins.


On the GD32 (on our enhanced models), it is closer to the same scenario as the STM32.  The same design for the MCU core is provided by ARM, but the implementation of the modules is slightly left to the implementor.  On the STM32, ST is the elder more experienced manufacture.  If you don't use RTS/CTS lines, the STM32 can reconfigure those pins separately and put them back into the GPIO pool.  With the GD32 design, all four lines belong to the USART module and can not be reassigned if the UART pins are in use.  GigaDevices also chooses to push the clock rate limit higher (thus 104MHz over STs 72MHz).


The Nextion firmware has limited which baudrates can be used, in the Nextion Instruction Set.


But hey - isn't communications fun?

Hi Everyone,


Just to bump this back up the list so to speak, I did refine the code and got it working as I'd originally envisaged.


The 0x05 issue is solved and the whole thing runs as fast as your Nextion will allow.


Timings showed that once the first block was in, the process ran with an average return ack (0x05) time of 21ms per 4k chunk.


I also did some work on file serving from another 'remote ESP8266'. Using the 3MB of Spiffs available on chip. I can send the .tft from an ordinary web browser for storage and serving as you would do with something like Apache on a PC. Technically you could get the file from anywhere on the Internet. Using a key on Nextion the Update can be called on demand, all very seamlessly and fuss free.


I hope we can now generate some interest here :)


Steve. 

Well done Steve.


Just to be clear, that was ~21ms between 4096th byte until 0x05 is received?

Hi Patrick,


That is correct. And also to be clear, send is immediately resumed once that byte is in. So no blocking set length timeouts to wait for :)

So, the Upload Protocol as published is correct.


I worry the interpretation of is a cause of confusion

Nextion device will return 0x05 when all data received.

Is misinterpreting "all data" by many as the last byte, and
should be interpreted "all" data meaning every full/partial chunk.

Perhaps I can see if they will make this more clear.


The whole description could be a little better I think.


Nextion device will return 0x05 in 500ms, you can send tft bin file to the serial port after you receive it. You should split the data in 4096 byte each packet. And send all the remains data (<4096byte) in the last packet. Nextion device will return 0x05 when all data received. 


Could be..


You can send the .tft bin file to the serial port after you receive it, you should split the data to 4096 bytes each packet. And send the remainder (<4096byte) in the last packet.

Nextion device will return 0x05 within 500ms of receiving each 4096 byte packet, and also on receipt of the last packet.


:)


Like the post delete, is that new ? An edit feature would be nicer, as you get on other forums :)

Post delete has been around for a while,  But "edit" is amongst the last commands the board Gods have as theirs. =) Just kidding.  It is such a shame that the package upgrade needed for the feature is such a huge price difference in the package prices. They gouge at every chance they get.

Oh well, Copy/Delete/NewPaste/Edit it is then =)

You have friends in high places if you need a real edit made. =)

Have others been able to make the code provided by indev2 above to work successfully?  I've copied and pasted the code as provided, changing the SSID, WPA password, and http: file url leaving everything else untouched.  I've wired up a WeMos D1 mini per the sketch, with the panel on RX/TX for hardware UART and an external serial interface connected to D6 (GPIO14) to monitor the process.  The file downloads and the process reports a success but the HMI is never applied and I never get receive the "Nextion Upload Success !" message shown in the example above.  


Here is a link to the serial output I receive when running this script: https://hastebin.com/abayaqokoz


Other details:

  • The panel has had "bauds=115200" applied so it should be running at the correct speed.
  • The target TFT file I'm looking to flash is able to flash using the Nextion editor process which suggests it should be correct for the panel.
  • Issuing a "connect" command via serial (outside of the sketch) returns the response outlined in the update process docs. 

Login or Signup to post a comment