I2C Call Inside Timer Core Dump ESP32

Any bugs you encounter with Flowcode should be discussed here.
Post Reply
unity-control
Posts: 54
http://meble-kuchenne.info.pl
Joined: Wed Sep 08, 2021 10:36 pm
Has thanked: 26 times
Been thanked: 11 times

I2C Call Inside Timer Core Dump ESP32

Post by unity-control »

Hi guys,

I've got a very simple program whereby I update an I2C device periodically once a second via an interrupt (Timer0).

This is causing a core dump and I am not sure what else to try here, I've reduced the program only to the I2C and the Timer routines and means all my other peripherials are not the cause of this problem.

I used to do this without trouble on PIC16/PIC32 and not sure if I'm doing something wrong here, but it's causing a CORE DUMP :-(

I have tried all other Timers Timer0~3 with the same results :-( I've tried different intervals in ms and same problem happens...

Code: Select all

Guru Meditation Error: Core  0 panic'ed (Interrupt wdt timeout on CPU0)
Core 0 register dump:
PC      : 0x40088e74  PS      : 0x00060d34  A0      : 0x80088418  A1      : 0x3ffbe110
A2      : 0x3ffb0b14  A3      : 0x3ffb27ac  A4      : 0x00060d21  A5      : 0x00000001
A6      : 0x00060d21  A7      : 0x3ffbe990  A8      : 0x3ffb27ac  A9      : 0x3ffb27ac
A10     : 0x00000019  A11     : 0x00000019  A12     : 0x3ffae6fc  A13     : 0x00000001
A14     : 0x00060d23  A15     : 0x40001d48  SAR     : 0x00000000  EXCCAUSE: 0x00000005
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff
Core 0 was running in ISR context:
EPC1    : 0x400d56ca  EPC2    : 0x00000000  EPC3    : 0x00000000  EPC4    : 0x40088e74
Thanks for your help!
R
Attachments
TEST_INT_I2C.fcfx
(27.97 KiB) Downloaded 55 times

mnfisher
Valued Contributor
Posts: 953
Joined: Wed Dec 09, 2020 9:37 pm
Has thanked: 104 times
Been thanked: 507 times

Re: I2C Call Inside Timer Core Dump ESP32

Post by mnfisher »

If you disable the i2c::Transaction_Write in Timer_1s does it still crash?

It might be that i2c relies on interrupts for timing and attempting this from an ISR (where interrupts are disabled) - is causing a lockup (until the wdt times it out)

Martin

unity-control
Posts: 54
Joined: Wed Sep 08, 2021 10:36 pm
Has thanked: 26 times
Been thanked: 11 times

Re: I2C Call Inside Timer Core Dump ESP32

Post by unity-control »

Thanks Martin,

It won't crash, cause the Call is inside the Interrupt. I have found an excellent answer to this question. I just need to get my head around this and think how to re-engineer my mind as I come from the PIC background and new to ESP32.

But anyway, it's a good read and makes total sense:
Re: Question: How to access an I2C device from an ISR handler
Postby ESP_Sprite » Fri Jul 20, 2018 10:14 am

Okay, I think the issue is that you come from a more simple embedded environment, with just a bunch of peripherals and no complex things like WiFi, BT, USB or whatever, possibly without an OS to get between you and the bare metal. (Not implying there's anything wrong with that, I've written code for that environment many times and still do.) In this environment, it's indeed perfectly acceptable to just stash whatever you need in the ISR; it's a mostly elegant and simple solution to what you want.

However, the ESP32 is not like this: it has a RTOS, which complicates life with task switching. You may ask yourself: why did we do this? It only stands between you and writing your bare metal code, and introduces race conditions and other frustrating things, right?

Well, the answer is twofold: first of all, you're not alone on the CPU anymore, and secondly, you (or others) may want to use this 2x240MHz powerhouse for all CPU power it's worth.

Let's start off with the second bit. Say you're decoding an MP3 on your ESP32. The ESP32 does not have many buffers, so you probably want to keep decoding this MP3 at a steady speed. Now, for the user interface, you have written an interrupt handler that receives a touch interrupt and then asks the touch controller the data. Now what happens? The IRQ starts, the CPU tells the I2C controller to read the data... and then just twiddles its thumbs until the answer comes. The CPU is pretty fast and I2C is really slow, but because you're in an interrupt, the CPU can't just go back to decoding MP3s, it has to actively wait until the I2C-controller is done. And when it's finally done, the MP3 decoder may not have had enough time to do its decoding and your audio stutters.

For the first scenario, you have to imagine the MP3 data comes from somewhere. As the ESP32 is a WiFi-enabled chip, it's not unlikely it's streamed over WiFi. This means an entire WiFi-driver and IP-stack are hard at work keeping the connection open by recieving and responding to packets at a breakneck speed. If you then hold up the CPU in a interrupt handler, this stream gets stopped; the server may think the line is congested and throttle its output, and again, you get stutters in your audio stream.

So how is this solved? By keeping your interrupts to a minimum and using a task for anything that doesn't need to happen the exact microsecond the interrupt arrives. If you use a queue to communicate between the interrupt and the task, the task will normally be asleep and use zero percent CPU time, As soon as it awakes, you can use the I2C driver to interrogate the touch driver. And what'd'ya know, because you're now calling the I2C driver from a task, while it's waiting it can now ask the OS to do other things instead, and resume the task as soon as data gets in. So instead of twiddling its virtual thumbs, your CPU happily resumes decoding MP3s and answering to WiFi packets.

So is the absence of an IRQ-capable I2C drivers a bug? I'd say it is not, it may even be a feature, making it harder to shoot yourself in the foot and introducing all kinds of CPU-power and latency issues in other places. Instead, use vonnida's suggestion: it's not a workaround and as I said, unlike your poll-in-a-thread method it doesn't use any CPU power until an interrupt happens. It's actually not a workaround but a pretty well-known pattern; the Linux kernel for instance calls it separation of the top half and the bottom half of an interrupt handler.
Re: Question: How to access an I2C device from an ISR handler
Postby vonnieda » Thu Jul 19, 2018 7:56 pm

It's best to do as little as possible in an ISR and let another task handle any heavy lifting. What I do for this kind of thing is post an event to a queue and let another task process the event.

So, in pseudocode:

1. Create a queue with xQueueCreate.
2. Create a task with xTaskCreate and in it's handler function run an infinite loop calling xQueueReceive. This will block most of the time.
3. In your ISR, use xQueueSendFromISR to post an event to the queue. It can just be a 0 if you don't have any specific info to send.
4. Back in your task, when the ISR posts an event the queue will wake up and you can do whatever you need to, including access I2C.

Jason

stefan.erni
Valued Contributor
Posts: 758
Joined: Wed Dec 02, 2020 10:53 am
Has thanked: 149 times
Been thanked: 171 times

Re: I2C Call Inside Timer Core Dump ESP32

Post by stefan.erni »

Hi to All

How can this be done with Flowcode? An idea?

regards

Stefan

unity-control
Posts: 54
Joined: Wed Sep 08, 2021 10:36 pm
Has thanked: 26 times
Been thanked: 11 times

Re: I2C Call Inside Timer Core Dump ESP32

Post by unity-control »

Hi Stefan,

I think implementing tasks indeed... I was thinking that whatever code runs at the moment on ESP32 given the nature of things is on Core1 (from documentation I've read - I am not a programmer).

So at least maybe a possibility of starting a task within Flowcode on the other core would be great.

But again I'm not a programmer, however have read that's fairly straightforward to implement tasks...

Having said this, I for one do not use the embedded BT or Wifi modules on ESP32 myself.

That kind of thing would be taking running a program on ESP32 to a whole new level, even if you can offload a regular piece of code (whatever subtask that one thinks about) and force it to run on the second core. You could even alternate between 2 or even more tasks. It all depends I think how complex the program is. But being able to use the second core would be amazing. Question is indeed how.

I guess we could put in a feature request for FC to implement tasks with a component? Again the scope of a component like this would be up for discussion...

Regards,
R

mnfisher
Valued Contributor
Posts: 953
Joined: Wed Dec 09, 2020 9:37 pm
Has thanked: 104 times
Been thanked: 507 times

Re: I2C Call Inside Timer Core Dump ESP32

Post by mnfisher »

I posted a bit about xTaskCreate at https://flowcode.co.uk/forums/viewtopic ... sp32+xtask

Martin

unity-control
Posts: 54
Joined: Wed Sep 08, 2021 10:36 pm
Has thanked: 26 times
Been thanked: 11 times

Re: I2C Call Inside Timer Core Dump ESP32

Post by unity-control »

I just found this thread looking for something else :-)

Excellent, thanks Martin, this is VERY useful stuff!

R

Post Reply