-
Notifications
You must be signed in to change notification settings - Fork 152
Description
First of all, thanks for the cool project, I finally embedded python in my application using CMake!
I embed python in a static and isolated configuration, without using modules on the filesystem. After setting all compile time and runtime defines I got stuck with an error on initialization (same as in #339):
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = (not set)
...
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00002008 (most recent call first):
<no Python frame>
While in most cases this means problems with the path to the core "*.py" libraries, in my case it means that the "encodings" module is not frozen (in memory).
After a couple of days of fighting and deep research I came up with a solution and figured out what was going on.
All frozen python modules must be generated in two places:
- cpython/Python/frozen.c
- deepfreeze/deepfreeze.c and frozen_modules/*.h headers (for frozen.c or custom lists like
struct _frozen *_PyImport_FrozenModules[]
).
The files from the second case are generated in this project by hardcoded statements in python-cmake-buildsystem/blob/master/cmake/libpython/CMakeLists.txt (COMMAND _freeze_importlib
, LIBPYTHON_FROZEN_SOURCES
, COMMAND _bootstrap_python
, LIBPYTHON_DEEPFREEZE_SOURCES
, ...).
But to add frozen modules to the global lists in frozen.c the script cpython/Tools/build/freeze_modules.py is used.
I also found that the "encodings" module is non-frozen not by accident and can be restored (see: https://github.com/python/cpython/blob/8ac7613dc8b8f82253d7c0e2b6ef6ed703a0a1ee/Tools/build/freeze_modules.py#L56).
So, putting it all together, I created a helper CMakeLists.txt in my project that does all the magic for my static runtime, and everything works: CMakeLists.txt
I think it would be great to implement such patches and generation here, in python-cmake-buildsystem project to not patch the project that patches CPython :D