New Tasmota32 safeboot and larger firmwares

[s-hadinger]

We (Tasmota dev team) are exploring a new flash setup that would allow for larger firmwares and larger file-systems. Currently the OTA firmware uploads needs 2 partitions of equal sizes. Each upgrade is writing to the non-used partitions and actunforive partition is swapped at next restart. This is expensive on 4MB flash devices and sets the limit of usable firmware at ~1800Kb. For most usages it is ok, but for lvgl/openhasp it can be a significant limit.

The new layout should look like:

• a first factory partition of 832kB containing a minimal tasmota32-safeboot partition. This partition would be enabled each time you need to flash a new Tasmota32 firmware, but in itself if wouldn't need to be reflashed. This safeboot includes minimal support for MQTT and HTTP, including TLS support.
• a unique app0 OTA partition
• a SPIFFS file-system partition

This new scheme has the following benefits:

• Only 896kB are used for house-keeping, the rest of the 3200kB can be used for firmware+filesystem
• In case of a faulty firmware, the safeboot partition is still usable

The downside is that we need to change the firmware upgrade mechanism to be two-steps: 1/ switch back to safeboot firmware, 2/ flash the OTA firmware, 3/ reboot

This is a follow-up of #15493. There is currently partial support for boards/esp32_4M_2APP.json with the following setup:

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
factory,  app,  factory, 0x010000,0x0D0000,
app0,     app,  ota_0,   0x0E0000, 0x2D0000,
spiffs,   data, spiffs,  0x3B0000,0x50000,

[s-hadinger]

For the record, when flashing with the layout above, you can find the following useful:

Print the current partition layout:

import partition_core var p=partition_core.Partition() print(p)

Example:

<instance: Partition([
  <instance: Partition_info(1 (data),2 (nvs),0x00009000,0x00005000,'nvs',0x0)>
  <instance: Partition_info(1 (data),0 (otadata),0x0000E000,0x00002000,'otadata',0x0)>
  <instance: Partition_info(0 (app),0 (factory),0x00010000,0x000D0000,'factory',0x0)>
  <instance: Partition_info(0 (app),16 (ota0),0x000E0000,0x002D0000,'app0',0x0)>
  <instance: Partition_info(1 (data),130 (spiffs),0x003B0000,0x00050000,'spiffs',0x0)>
],
  <instance: Partition_otadata(ota_active:0, ota_seq=[1,], ota_max=0)>
)>

Change factory to app1 partition

You can temporarily change the factory partition type to app1 which allows to update the safeboot partition using standard OTA machanism (and considering the new safeboot code is small enough to fit in 832kB). The function returns true if successful.

def change_factory_to_ota()
  import partition_core
  var part_info = partition_core.Partition()

  if part_info.ota_max() == 0
    for pp:part_info.slots
      if pp.is_factory()
        pp.subtype = 0x10 + part_info.ota_max() + 1
        part_info.save()
        return true
      end
    end
  end
  return false
end

Change back app1 to factory partition

This does the opposite as above. Once the new safeboot partition is re-flashed, you can mark it back as factory type. The functions returns true if succesful.

def change_app1_to_factory()
  import partition_core
  var part_info = partition_core.Partition()

  var pp = part_info.slots[2]
  if pp.is_ota() == 1
    pp.subtype = 0      # `factory`
    part_info.save()
    return true
  end
  return false
end

Force restart of safeboot partition

This function forces next restart to boot on safeboot partition

def boot_on_factory()
  import partition_core

  var part_info = partition_core.Partition()
  part_info.switch_factory()
  return true
end

Switch from safeboot to app0 partition

Currently there is no programmatic way to switch back to app0 partition, since Berry is not enabled in safeboot firmware. Work is ongoing, currently the only way is to OTA flash a new firmware.

[Jason2866]

Use Partition Wizard to convert old scheme to new scheme. It is a Berry tapp. Just load in Tasmota Filesystem and restart. After device is back you see a new menu entry.

[sfromis]

I see no guide to Partition Wizard and Upgrade 2 on https://tasmota.github.io/docs/Safeboot/ 😉

[s-hadinger]

Yup, it's a rare operation so no guide (yet). Any volunteer to write the guide?

[bs-github]

The Partition Wizard did nothing when I tired it (despite of it showing ready for all 4 steps).

The solution to this dilemma in my case was to:

1. backup the config,
2. reset the device (in my case a Sonoff Dual R3),
3. then run the Partition Wizard and
4. finally restore the config.

[tomm1ed]

The Partition Wizard did nothing when I tired it (despite of it showing ready for all 4 steps).

The solution to this dilemma in my case was to:

1. backup the config,
2. reset the device (in my case a Sonoff Dual R3),
3. then run the Partition Wizard and
4. finally restore the config.

Thanks so much, ran into the same with a Dual R3 and this did indeed work as intended :)

[Jason2866]

One additional thing. To try the new feature you have to activate a newer PlatformiO setup.
Thats done by changing in platformio_override.ini to.

[env:tasmota32_base]
; *** Uncomment next line ";" to enable development Tasmota Arduino version ESP32
platform  = https://github.com/Jason2866/platform-espressif32.git

Adding a new Tasmota32 env should be done too for easy use. Add to platformio_tasmota_cenv.ini

[env:tasmota32safemode]
extends  = env:tasmota32_base
board = esp32_4M_2APP
build_flags = ${env:tasmota32_base.build_flags} -DFIRMWARE_TASMOTA32

You can compile and flash (as every other variant) now tasmota32safemode

[s-hadinger]

There is now a simple HTTP API to force switch to factory partition, and switch back to app0 OTA partition. #15506

[TKFrozen]

Sorry for question 87 years later:

I had an old Shelly Plus Plug S and converted it to Tasmota (12.5).
Everything was good and in the end, I wanted to update to 13.2 like my other plugs.
Wasn't possible, because not enough space and I had two partitions (why with 12.x ?).
After some reading I flashed to 13.2-safeboot and now I'm stuck to update and now I'm stuck.
Can't update to full 13.2 (not enough space), can't go back to 12.5 either.

What now?

[sfromis]

Just don't flash a safeboot binary separately, outside of the standard process for updating from one safeboot to a newer one (only when you are already in the safeboot partition scheme).

Here in the Discussions tab, there is a pinned post at the top about
ESP32: Can't Update to 13.x.x due To Flash Size
This is what you should have read.

However, by doing the bad flashing of safeboot without having the partition scheme for it, I think that you've broken the structure, and I'd not expect any way forward based on the standard procedures. When the safeboot binary is used for upgrades, this is based on it already being in it's own partition, with another larger OTA partition for flashing the newer version standard binary. But lacking that, it can't do much.

I'm not aware of any repair procedure, outside of the very basic approach of wired flashing of a current binary (where the .factory binaries already includes the safeboot scheme).

[TKFrozen]

I was afraid of that. I should have switched to Safeboot in Partition Wizard beforehand. My mistake.
Anyway, thank you for your quick and competent response. I'll get myself a new plug then.

[Jason2866]

Mh, try if restart 3 works. This should switch to the safeboot firmware in partition safeboot. Try to load firmware v.12.5 than.
If this works you can start the convert process

[TKFrozen]

That's the problem. There is no safeboot partition, because I didn't convert the filesystem before.
I installed the safeboot in one of the two normal partitions. can't switch, can't install anything, because safeboot needs ~780kb and I have only 1536kb free space left. And actually in safeboot "Restart 3" does'nt work. It doesn't know the command.

But thanks for the idea. I appreciate it.