[Bug]: T-Echo broke by itself -> reboot loop

[Tie-fighter]

Category

Other

Hardware

T-Echo

Firmware Version

2.5.14.f2ee0df

Description

Hi folks.

I have a T-Echo and installed 2.5.14
I added a 6000mAh battery and was looking to see how long it would last, but in the morning I realized that the T-Echo already stopped working in the night. Bluetooth wouldn't connect and the LEDs would flash blue-red-red.

Plugging it in on my PC via USB would just give reconnects every ~2 seconds.
I found a help article here and started following it (Xinyuan-LilyGO/T-Echo#18 (comment))
Burning the bootloader worked and uploading different firmwares works, but the device is still stuck in the reboot loop.
I yet have to successfully delete the filesystem.
Serial Monitor ends an error, assert block lfs, so I suspect a corrupted filesystem might be the ultimate reason.

I worked on this kinda in between other things today, so I am sorry this report is not as detailed as it could be and that I didn't grab the original logs.
I hope this still might come in useful to somebody. If I manage to fix the device, I will document it here.

Relevant log output

No response

[thebentern]

Try the factory erase uf2 process
https://meshtastic.org/docs/getting-started/flashing-firmware/nrf52/nrf52-erase/

[Tie-fighter]

Formatting the filesystem fixed my T-Echo 🎉
I don't know the root cause why it broke, so let's see if it happens again...

I installed meshtastic CLI in Windows Subsystem for Linux (WSL) , but that didn't work on the first try, so I used Arduino IDE instead.

Could we adapt the documentation regarding that any serial terminal works, not just "meshtastic --proto"?

[Tie-fighter]

Hi folks.

So I have tried two different meshtastic versions now, and with both, the T-Echo breaks after some time. This time there are logs.

First I tried 2.5.15.

`//\ E S H T /\ S T / C

�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mFilesystem files:
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m prefs (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m channels.proto (57 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m config.proto (169 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m module.proto (96 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m db.proto.tmp (7615 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m adafruit (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m bond_prph (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34m bond_cntr (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mPower::lipoInit lipo sensor is not ready yet
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mUse analog input 4 for battery level
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mScan for i2c devices
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mScan for I2C devices on port 1
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mI2C device found at address 0x51
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mPCF8563 RTC found
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mI2C device found at address 0x77
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mWire.available() = 1
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mBME-280 sensor found at address 0x77
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32m2 I2C devices found
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34macc_info = 0
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mfound i2c sensor meshtastic_TelemetrySensorType_BME280
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mS:B:7,2.5.14.f2ee0df
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mBuild timestamp: 1732406137
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mReset reason: 0x0
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mSet random seed 327275686
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mInit NodeDB
�[0m�[31mERROR �[0m| ??:??:?? 1 �[31mCould not open / read /prefs/db.proto
�[0m�[33mWARN �[0m| ??:??:?? 1 �[33mDevicestate 0 is old, discard
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mInstall default DeviceState
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mInitial packet id 182225040
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mPartially randomized packet id 3592900753
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mUse nodenum 0xc78f228c
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoad /prefs/config.proto
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded /prefs/config.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded saved config version 23
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoad /prefs/module.proto
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded /prefs/module.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded saved moduleConfig version 23
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoad /prefs/channels.proto
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded /prefs/channels.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded saved channelFile version 23
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mcleanupMeshDB purged 0 entries
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mUse nodenum 0xc78f228c
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mAdding node to database with 1 nodes and 158504 bytes free!
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mExpand short PSK #1
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mWanted region 3, using EU_868
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mCoerce telemetry to min of 30 minutes on defaults
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mSave /prefs/db.proto
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded /prefs/module.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded saved moduleConfig version 23
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoad /prefs/channels.proto
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded /prefs/channels.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mLoaded saved channelFile version 23
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mcleanupMeshDB purged 0 entries
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mUse nodenum 0xc78f228c
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mAdding node to database with 1 nodes and 158504 bytes free!
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mExpand short PSK #1
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mWanted region 3, using EU_868
�[0m�[34mDEBUG �[0m| ??:??:?? 1 �[34mCoerce telemetry to min of 30 minutes on defaults
�[0m�[32mINFO �[0m| ??:??:?? 1 �[32mSave /prefs/db.proto
�[0m�[31mERROR �[0m| ??:??:?? 1 �[31mLFS assert: head >= 2 && head <= lfs->cfg->block_count
�[0m�[31mERROR �[0m| ??:??:?? 1 �[31mLFS ass`

Then I tried 2.5.11, which ran for 2 days now but ended breaking too:

`//\ E S H T /\ S T / C

�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mFilesystem files:
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m prefs (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m channels.proto (57 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m config.proto (169 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m module.proto (96 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m db.proto.tmp (8486 Bytes)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m adafruit (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m bond_prph (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34m bond_cntr (directory)
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mPower::lipoInit lipo sensor is not ready yet
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mUse analog input 4 for battery level
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mScan for i2c devices...
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mScan for I2C devices on port 1
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mI2C device found at
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mI2C device found at address 0x77
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mWire.available() = 1
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mBME-280 sensor found at address 0x77
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32m2 I2C devices found
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34macc_info = 0
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mfound i2c sensor meshtastic_TelemetrySensorType_BME280
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mS:B:7,2.5.11.8e2a3e5
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mBuild timestamp: 1730809408
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mReset reason: 0x0
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mSet random seed 1662396303
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mInit NodeDB
�[0m�[31mERROR �[0m| ??:??:?? 2 �[31mCould not open / read /prefs/db.proto
�[0m�[33mWARN �[0m| ??:??:?? 2 �[33mDevicestate 0 is old, discard
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mInstall default DeviceState
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mInitial packet id 1383903487
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mPartially randomized packet id 4267576576
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mUse nodenum 0xc78f228c
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoad /prefs/config.proto
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoaded /prefs/config.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoaded saved config version 23
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoad /prefs/module.proto
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoaded /prefs/module.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoaded saved moduleConfig version 23
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoad /prefs/channels.proto
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoaded /prefs/channels.proto successfully
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mLoaded saved channelFile version 23
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mcleanupMeshDB purged 0 entries
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mUse nodenum 0xc78f228c
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mAdding node to database with 1 nodes and 155192 bytes free!
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mExpand short PSK #1
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mWanted region 3, using EU_868
�[0m�[34mDEBUG �[0m| ??:??:?? 2 �[34mCoerce telemetry to min of 30 minutes on defaults
�[0m�[32mINFO �[0m| ??:??:?? 2 �[32mSave /prefs/db.proto
�[0m�[31mERROR �[0m| ??:??:?? 2 �[31mLFS assert: block`

In both cases the configuration was default, except setting the region.

Does anybody have an idea how to fix this?

[tavdog]

This just happened to me. Do we have any fixes for this problem ?

[tavdog]

I managed to fix this by forcing a call to format(). Since auto format only happens on mount failure and this error seems to be about a corrupted file system that does mount but causes the LFS assert: error. PR to add a call to FSCom.format() at the end of lfs_assert in incoming. #5670