Need help upgrade ESPIDF

[MJU20]

For the umpteenth time, I've tried to upgrade my ESP-IDF from version 4.2 to 5.1, and again there are more errors than I ever feared.

I restored my PC to a previous version before I attempted to upgrade the ESP-IDF.

My PC runs Windows 11 with the latest updates.
I have GIT 2.21 from the ESP-IDF toolchain (the toolchain that was downloaded from the Flowcode website).
At this point, I had to revert to Python 3.7.
And I have ESP-IDF 4.2 running.

I think it's better to first upgrade GIT and Python to a known working version before attempting to upgrade the ESP-IDF??
But will things break if I first download the newest versions (GIT and Python)? I've tried Python 3.12, and things went wrong.

Please help me (and others who are having difficulties upgrading) step by step to get the latest, trusted, working version of the ESP-IDF.

Please only proven, working ways to get things working!

[chipfryer27]

Hi

My fun with the toolchain is well documented. If it works leave it alone

My only experience related to W11 was a clean install on a new machine. I followed the guides and things went well. However I think I had a newer version of GIT (or Python) than recommended as something I installed prior to FC installed as part of that process.

It has been stable, so I can't complain.

Regards

[medelec35]

Hello, if you follow the ESP32 Toolchain instructions to the letter, then the toolchain should work well.
If you deviate with different version than the recommended, then you could face issues.
Changing from one version to the other, Ben posted instructions here

What I do when changing is Uninstall the esp32 idf and manually remove both folders of the main IDF and the toolchain.
It does not do any harm to also uninstall git and python to start from scratch.

The esp32 toolchain is being investigated, to see if it can be improved on.

[MJU20]

Thanks already for your advice, but the reason for this post is that I've done all this.

I used the tutorial on the FC website, the suggestions on the forum, and the descriptions from other people who struggled.

I have paid for the ESP pack, but can't use all the devices because of the toolchain.
Every time I try, there is yet another problem that occurs (a version of Python that is too old, idf.py that is corrupt, and so on).

I have read about all these problems on ESP forums and Espressif pages, but every suggestion takes me further away from a solution.

It's like I solve one problem and two others occur.

[medelec35]

Can you post a compiled demo.fcfx that shows the issue along with the .msg.txt file.

[MJU20]

Can you post a compiled demo.fcfx that shows the issue along with the .msg.txt file.

No actually I can't.

After every attempt to change things, other errors occurred. And after 3 days of changing things I reverted my PC back to the state it was before I started the changes.
I use Acronis to make backups and if I don't get progress, after a while I go back to it's original state before the changes.

I've bought the ESP chip pack for FC10, but some types of ESP32 don't compile (because I still have the V4.4x of the IDF).

It would be nice to have a toolchain that does all the mentioned things in one action (delete the dirs, uninstall the ESP IDF and so on).
Things I've done tens of times before posting this help request.

Frustrating for me

[medelec35]

I'm sorry to hear about your frustration, but unfortunately, we have no control over the toolchain as its created by an external company.
When you compile to hex, even though project does not compile successfully, does it not produce a .msg.txt file within the project folder?
I'm interested to see what text the compiler produced while trying to compile.
Do you get any popups e.g. Stating toolchain not installed?
We can't help further until we know what is going wrong, that is why I asked for your project file and msg.txt file if one was produced?

[MJU20]

I've listed the steps and the outcome from my attempt to upgrade from V4.4 to V5.x in this:

Summary of Issues and Outcomes

Initial Issue:
Problem: 'idf.py' is not recognized as an internal or external command.
Attempted Solution: Ensure ESP-IDF and Python are correctly installed and idf.py is in the system PATH.
Result: The issue persisted with additional errors related to Python version.

Python Version Compatibility:
Problem: ESP-IDF requires Python 3.8 or newer, but Python 3.7.3 was installed.
Attempted Solution: Upgraded Python to version 3.12.
Result: The new version was recognized, but issues with the virtual environment emerged.

Virtual Environment Issue:
Problem: Error when installing packages: ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
Attempted Solution: Created and activated a new virtual environment.
Result: The activation command was not recognized, indicating a path or environment issue.

Installing Packages from requirements.txt:
Problem: Error installing packages: ERROR: Could not find a version that satisfies the requirement package1 (from versions: none).
Attempted Solution: Checked package names on PyPI and updated requirements.txt.
Result: Continued errors about missing or incorrect package names.
Specific Python Environment Problems:

Problem: C:\esptools\Espressif\python_env\idf5.2_py3.12_env\Scripts\python.exe doesn't exist!
Attempted Solution: Ran idf_tools.py install-python-env.
Result: The Python environment was restored, but issues with missing modules persisted.
Click Module Issue:

Problem: Cannot import module "click".
Attempted Solution: Activated the virtual environment and manually installed the Click module.
Result: The Click module was installed, but the initial idf.py issue persisted.
This summary highlights the primary issues encountered, the attempted solutions, and their outcomes.

[MJU20]

So I've manually upgraded the ESP-version, the Python version and the GIT version that was already on my PC.

But with every step there were other problems that occurred.
So now I want to ask what steps I can do to update the IDF with succes..

I've read enough on the forum but for one or the other reason things don't work for me.
I remember a post: viewtopic.php?p=11214#p11214 where I've tried the suggested sollution from BenR, but after that everything nothing was suggested anymore.

So please give me a handle of what steps I can do right now to upgrade..

[mnfisher]

I've just installed 5.1.4 on a new Windows install.

I messed it up - and solved it. So my advice (for what it is worth)

Install Python and Git first (use the latest versions) Make sure they added to the environment variables (but I used all defaults).
Then install esp-idf. Use default locations (again use all default values). This will install to c:\espressif\ with the individual framework in C:\Espressif\frameworks\esp-idf-v5.1.4 (or other version number - you can have multiple versions though swapping is a bit of a pain))

Add this to FC paths (so C:\Espressif\frameworks\esp-idf-v5.1.4) Save and attempt to compile a simple esop32 program (empty will do). If it works huzzah - if it doesn't don't panic.. This creates the necessary directory structures.

If failed.
Try and compile this simple FC program from the command line. Note - do not use the esp5.1 cmd prompt - use

Code: Select all

windows-key Command

then

Code: Select all

"C:\Espressif\frameworks\esp-idf-v5.1.4\export"

cd to where your program is then cd to it's directory - so if you had flowcode1.fcfx in C:\Projects you'll need to do cd c:\projects\flowcode1
do

Code: Select all

 idf.py build

If it works all should be well - if not do

Code: Select all

C:\Espressif\frameworks\esp-idf-v5.1.4\install

(this should do something with progress bars - if it doesn't check you are in a plain command prompt and not the esp5.1 (which is PowerShell)
then

Code: Select all

idf.py fullclean
idf.py build

- again if it compiles good. Return to FC and try again.

More things that can go wrong...

I install esp-idf first and then python -- and FC threw a python is not in environment error. Edit environment variables (windowskey environment - edit environment) Check C:\Users\Martin\AppData\Local\Programs\Python\Python312 is in the path.
Secondly check - C:\Users\Martin\AppData\Local\Programs\Python\Python312 - in my install 'python' was just call 'py' - I renamed it python

The python script I posted to toggle between versions works:

Code: Select all

#!python3
import os

cmds = ["git fetch", "git checkout release/v4.1", "git pull", "git submodule update -f --init --recursive", "install.bat", "python tools/idf_tools.py uninstall"]

os.chdir("C:/Esp-idf/frameworks/esp-idf-v4.1.4")
for cmd in cmds:
    os.system(cmd)

Again run from a command prompt (and adjust the version / path to suit)

Martin