Calltrace does not continue after first time hitting `break`

[sebglatz]

TL;DR

px4_breaktrace does not continue to run after break despite continue
The MCU halts forever and thus only one function call is sampled.

What?

I want to create a callgraph for some SPI function. To do so, I run:
python3 -m emdbg.analyze.calltrace -v --px4-dir ~/PX4-Autopilot --target px4_fmu-v5_default --jlink --sample 60 -ex "px4_calltrace_spi_exchange"

with px4_calltrace_spi_exchange defined like this in fmu.gdb:

define px4_calltrace_spi_exchange
    px4_breaktrace spi_exchange
end

Expected behavior

The MCU should continue to run after hitting the break point.
Note: The function px4_commands_backtrace does call continue after break and some printf

[niklaut]

Hm, I cannot reproduce this:

python3 -m emdbg.analyze.calltrace -v --target px4_fmu-v5x_default --stlink --sample 60 -ex "px4_calltrace_spi_exchange"
python3 -m emdbg.analyze.calltrace -v --target px4_fmu-v5x_default --stlink --sample 60 -ex "break spi_exchange"

Perhaps it's because I'm using an STLink, let me try with a JLink.

[sebglatz]

Thank you.
I have just tried it with a stlink v2.
The MCU also halts forever.

[niklaut]

Hm, that sucks. What GDB version are you using (arm-none-eabi-gdb-py3 --version)? Is the ELF file in sync with the firmware? ie. try

python3 -m emdbg.analyze.calltrace -v --target px4_fmu-v5x_default --stlink --sample 60 -ex "load" -ex "break spi_exchange"

[sebglatz]

• arm-none-eabi-gdb-py3 --version
  -> GNU gdb (xPack GNU Arm Embedded GCC x86_64) 12.1.90.20221210-git

• .elf is in sync with the fw. Only difference I see between our experiments is that my fmu is the v5, yours is the v5x...

[niklaut]

Oh I thought it was a typo at first. I'll check if I can find a v5 somewhere. But at first glance it looks pretty similar to the v5x.

• Can you use GDB to put a breakpoint on spi_exchange and then continue?
• You can reduce the SWD clock speed in fmu_v5x_stlink.cfg from 8MHz down to maybe 1MHz and see if it works?

[sebglatz]

• Yes, that's so confusing to me: break spi_exchange and then executing the commands of px4_commands_backtrace "by hand" (printf..., px4_backtrace, continue) during a gdb session works nicely with the jlink (have to try stlink, but prob. same).

• Reducing the stlink speed to 1MHz did not help, unfortunately...

[niklaut]

Hm, maybe it's a dependency problem? Here's what I have installed:

$ pip3.8 install --upgrade-strategy=eager --upgrade emdbg
Requirement already satisfied: emdbg in /opt/homebrew/lib/python3.8/site-packages (1.3.9.post2+git.4f062ccb.dirty)
Requirement already satisfied: pyserial in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (3.5)
Requirement already satisfied: graphviz in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (0.20.1)
Requirement already satisfied: pygdbmi in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (0.11.0.0)
Requirement already satisfied: cmsis-svd in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (0.4)
Requirement already satisfied: arm-gdb in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (0.9.9)
Requirement already satisfied: pydwf in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (1.1.19)
Requirement already satisfied: yoctopuce in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (2.0.59414)
Requirement already satisfied: rpyc in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (5.3.1)
Requirement already satisfied: rich in /opt/homebrew/lib/python3.8/site-packages (from emdbg) (13.7.0)
Requirement already satisfied: six>=1.10 in /opt/homebrew/lib/python3.8/site-packages (from cmsis-svd->emdbg) (1.16.0)
Requirement already satisfied: numpy in /opt/homebrew/lib/python3.8/site-packages (from pydwf->emdbg) (1.24.4)
Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/homebrew/lib/python3.8/site-packages (from rich->emdbg) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/homebrew/lib/python3.8/site-packages (from rich->emdbg) (2.17.2)
Requirement already satisfied: typing-extensions<5.0,>=4.0.0 in /opt/homebrew/lib/python3.8/site-packages (from rich->emdbg) (4.9.0)
Requirement already satisfied: plumbum in /opt/homebrew/lib/python3.8/site-packages (from rpyc->emdbg) (1.8.2)
Requirement already satisfied: mdurl~=0.1 in /opt/homebrew/lib/python3.8/site-packages (from markdown-it-py>=2.2.0->rich->emdbg) (0.1.2)

Probably not relevant for these issues:

$ JLinkGDBServer --version
SEGGER J-Link GDB Server V7.88f Command Line Version
$ openocd --version
Open On-Chip Debugger 0.12.0+dev-01472-gadcc8ef87 (2024-01-11-16:25)

[sebglatz]

Thank you.
Exact same dependencies. (except emdbg which is 1.3.9, not the dirty hash)
Also set it up freshly in a virtual env (python 3.8.18).
Still not sampling more than once...

[niklaut]

I'm not sure what else could be the problem… did this magically solve itself?