Running pip from within FreeCAD

Need help, or want to share a macro? Post here!
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Running pip from within FreeCAD

Post by chennes »

onekk wrote: Fri Jan 07, 2022 3:40 pm

Code: Select all

MODULE_PATH = packages_dir

if MODULE_PATH not in sys.path:
    sys.path.insert(-1, MODULE_PATH)
Thanks, this is quite interesting -- my one concern is the last bit: this way it does not persist between restarts of FreeCAD, right? So really I have to have FreeCAD's core startup code know about the new location we are creating, right?
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Running pip from within FreeCAD

Post by chennes »

@wmayer what do you think about this strategy?
  1. We maintain a simple (no-version information) list of allowed Python packages: any requirements.txt entry that doesn't have an entry gets removed and is not installed. This only applies to actual entries in the requirements.txt file, not to any dependencies those bring in.
  2. All python packages are installed in $USER_APP_DATA_DIR/PythonPackages (or some other suitable name). We add this path to the end of FreeCAD's standard Python path (e.g. so a later-installed system package would override, etc.).
  3. Addon Manager has a preference for the Python executable to use, and if not set, searches as described by @onekk above. This seems to be the most conservative approach: if we can't figure out the exact right executable, don't guess, don't install.
We don't deal at all with a system where the user might want to install using their system's package manager: if that is a better option for them, they should use the "Ignore" or "Cancel" options in the dependency request window, rather than "Yes" (Cancel is the default button). This provides the convenience of letting users just get on with life and run their Addons, but helps us protect them against malware and/or systems where these packages may create conflicts.
wmayer wrote: Thu Jan 06, 2022 11:49 am Ping
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
User avatar
onekk
Veteran
Posts: 6144
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: Running pip from within FreeCAD

Post by onekk »

chennes wrote: Mon Jan 10, 2022 4:16 am Thanks, this is quite interesting -- my one concern is the last bit: this way it does not persist between restarts of FreeCAD, right? So really I have to have FreeCAD's core startup code know about the new location we are creating, right?
It is not persistent as AppImage have no an autoload mechanism. But maybe using the autoloading mechanism of user\Mod dir something could be achieved.

But as you install packages as dependencies of AddOns, maybe it is the plugin itselt that could had the "pip user dir" to the path if not already present or the AddOn Manager itself could have these lines in it.

Main concern could be the fact that installed packages are not showing in "pip list" but this could even an advantage to mantain a better isolation from the system.

Do you have seen pipx as it claims that it is able to mantain isolation and control over what is installed.

It is cited in Python official documentation so it should not be too bad.

Regards

Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
wmayer
Founder
Posts: 20242
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Running pip from within FreeCAD

Post by wmayer »

chennes wrote: Mon Jan 10, 2022 4:34 am @wmayer what do you think about this strategy?
That looks OK to me.
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Running pip from within FreeCAD

Post by chennes »

OK, I have a PR in that implements this strategy -- thanks to everyone for your assistance in making this happen. Now I need help testing it! Some of the tests I don't know how to do, because we don't really have a way to easily package up the PR as a flatpack, Docker image, Snap, etc. I'm open to testing suggestions on those fronts, if you have ideas.
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Running pip from within FreeCAD

Post by chennes »

For the initial allowed list of Python packages, I've got:

Code: Select all

ezdxf
markdown
numpy
olefile
pillow
pygit2
requests
scipy
xlrd
xlutils
xlwt
PyYAML
That covers all of the packages that are currently specified in metadata files in "official" addons. I would guess there are also dependencies out there that are not yet set in a metadata file, but I have no way of knowing what those are. Any objections to this initial list? Any obvious additions I should make?
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
nic
Posts: 135
Joined: Thu Apr 18, 2019 1:14 pm
Location: France

Re: Running pip from within FreeCAD

Post by nic »

TheMarkster
Veteran
Posts: 5505
Joined: Thu Apr 05, 2018 1:53 am

Re: Running pip from within FreeCAD

Post by TheMarkster »

I think scipy and numpy are already included. I don't recall installing them separately and I am able to import both in the python console without error.
User avatar
ebrahim raeyat
Posts: 619
Joined: Sun Sep 09, 2018 7:00 pm
Location: Iran
Contact:

Re: Running pip from within FreeCAD

Post by ebrahim raeyat »

nic wrote: Fri Jan 14, 2022 7:03 pm Here is my wishlist :
and for mine:
  • pandas
    openpyxl : for export to .xlsx format
    python-docx : for creating reports in .docx format
    comtypes: for communicating with windows software
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: Running pip from within FreeCAD

Post by chennes »

nic wrote: Fri Jan 14, 2022 7:03 pm Here is my wishlist :
Which workbench are you using these in? Can you add them to a requirements.txt file when you have a moment?
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
Post Reply