Let's talk about Conda

[sgrogan]

I thought of trying to create a wiki page explaining the interaction between FreeCAD and (mini)Conda explaining the utility of Conda to FreeCAD.
I first tried to create a bullet point list. This proved to quickly get disjointed.
So, I tried to create an outline. I started by trying to section use cases, but maybe OS would be better? That wasn't very coherent either.

So, how do you use Conda + FreeCAD?

Some Windows-centric rambling thoughts from me.

You can install miniconda and create an environment that you can install a stable (18.x) or development version into using @looo's weekly builds.
You can do the above and then use conda as a package manager and use conda build tools to build your own development builds using FreeCAD master or your own local codebase.
You can use miniconda as package manager/dependency solver and use native tools, external to conda, to build master or from your own code base.
You can use any of the above methods to package portable bundles from the builds.

On Windows we have basically two build systems/dependency managers. We have the Libpack and (mini)conda. Maybe three if you consider build and maintain all the dependencies your self (Conda can help with this too)

I've begun to think of the Libpack like Debian. It's a collection of stable packages that are more proven and work together.
Conda is like Arch, bleeding edge dependencies, they will be updated more often than the Libpack, but there might be temporary breakages do to cross-dependency problems or FreeCAD porting.
Self maintained dependencies or conda custom environments are like Ubuntu PPA's, somewhere in between.

Anyone have some thoughts how to organize this? TIA

[chrisb]

Thanks already in advance for digging into this.

From my personal interests and remembering what I would have needed when I first saw this conda thing I would have liked to see something like this

1.) What is this conda after all, what is it about
2.) Why and how is it used in FreeCAD packaging
3.) Why should I and how can I use it myself

4.) OS specific issues; they could well go into point 2.) or 3.) as well.

Whatever you do, it will be far far better than the nothing we have now!

[looo]

sry for not responding to this topic.
I feel like having failed badly with the documentation of the conda-stuff done so far. So I am happy if things get improved.
I guess conda itself is documented quite well, so we should refer as much as possible to the official docs from conda [1] and conda-forge [2]. For building with conda-build again the official docs should be used [3]. Building with conda locally is a bit more difficult as it is not really supported by conda (at least in the way how we are used to build freecad. I guess the best solution would be a tool (similar to conda-build) which setup the environment based on a recipe. But then again things like virtual-envs are not yet supported by c++ ide's and therefor things have to be setup manually (and this is quite frustrating). So in my eyes it would be best to talk about development environments with developers from conda / conda-forge. This also includes the topic of debug-builds which are not yet supported by conda. Creating a proposal for all this would be a first step in the right direction.

In my eyes it would be nice to add a faq-section to freecad-docs or to the freecad_conda recipe's docs, whcih is an easy way to create documentation for new-comers.

[1] https://docs.conda.io/en/latest/
[2] https://conda-forge.org/docs/
[3] https://docs.conda.io/projects/conda-build/en/latest/

[vocx]

I just wanted to bump this, because I feel I'm in the same boat as what chrisb mentioned.

I have been using Linux for 10 years and I have not needed Conda at all in that time. I have no idea what it is, really. A lot of talk about Conda here in this forum, but it's still mumbo jumbo to me.

1. Is Conda a package manager like Apt (apt-get)?
2. Is it a "repository" like Ubuntu's personal package archives? Or like Python's PyPi? Or Latex's CTAN?
3. Is it meant mostly for Windows? As Windows doesn't have package infrastructure like Linux, it sounds Conda is trying to be such repository.
4. Does Conda provide ready to install programs? Or is it meant to provide dependencies (libraries) so that users compile Linux software in Windows?
5. It sounds like Conda is similar to Cygwin in that case?

Originally I thought Conda was just some other way to distribute Python in Windows, like ActiveState's Python distribution. But if I'm correct its mission has been expanded since then.

Why should I care about Conda, if I don't use it at all? I hear all the time, "this and that has been updated in conda... conda-forge", "the conda packages aren't ready... have bugs... I'm fixing them", "new package in conda". It just doesn't seem to affect me at all while I continue using my Ubuntu system happily.

[looo]

I will answer the questions as short as possible as I wrote about it too much already...

1. Is Conda a package manager like Apt (apt-get)?

Conda aims to be a cross-platform package-manager. It relies on some basic system libraries but tries to be as independent as possible. Packages for linux should work on any linux (there are some limitations, but all in all it works pretty good).

2. Is it a "repository" like Ubuntu's personal package archives? Or like Python's PyPi? Or Latex's CTAN?

There are different channels to share conda packages. Most important for freecad are the channels main, conda-forge, freecad/label/dev. Here for example are the build of freecad 0.19_pre: https://anaconda.org/freecad/freecad/files

I guess it's very similar to pypi. But tries to be more consistent (compiler, environments).

3. Is it meant mostly for Windows? As Windows doesn't have package infrastructure like Linux, it sounds Conda is trying to be such repository.

Looking at the downloads of the python-package I don't think it's mainly used for windows: https://anaconda.org/conda-forge/python/files
I guess the biggest advantage of conda are the virtual-environments. This solves the problem of testing multiple versions of one package on one machine. I think this is coming from python and conda tries to establish virtual-envs for everything (similar to nix).

4. Does Conda provide ready to install programs? Or is it meant to provide dependencies (libraries) so that users compile Linux software in Windows?

It provides libraries, but every plattform has different default compilers:
eg. for cpp: linux uses gcc, osx uses clang and windows uses visual-studio compiler.

5. It sounds like Conda is similar to Cygwin in that case?

Hmm, I do not know much about cygwin. Conda provides a mingw-toolchain which can be used as an alternative to visual-studio compilers. We use it for example for the calculix windows package: https://github.com/conda-forge/calculix ... a.yaml#L22

Why should I care about Conda, if I don't use it at all? I hear all the time, "this and that has been updated in conda... conda-forge", "the conda packages aren't ready... have bugs... I'm fixing them", "new package in conda". It just doesn't seem to affect me at all while I continue using my Ubuntu system happily.

For sure, you don't have to care about it. But as the appimage and dmg are based on conda-packages I think it's fair to have some conversation on the forum.

When I started efforts to help with the python3-port I tried to create packages with apt, but soon gave up on this, because it was too difficult for me to get involved. Also it only created packages for one linux-distribution. So I tried conda and stick with it until now. To me it seems like conda is a good solution to provide cross-plattform packages with minimal maintenance. As FreeCAD tries to be cross-plattform it's not the worst match I guess.

[vocx]

... To me it seems like conda is a good solution to provide cross-plattform packages with minimal maintenance. As FreeCAD tries to be cross-plattform it's not the worst match I guess.

That seems like a good reason to use Conda.

However, to me the biggest hurdle I have with trying Conda, is that I don't seem to have a use for it. If I already have FreeCAD installed on my system, if I can get the source code and compile it myself with a few terminal commands, why should I try Conda?

How do I start using this Conda thing even? Do I just install conda, and then conda install freecad?

[looo]

However, to me the biggest hurdle I have with trying Conda, is that I don't seem to have a use for it. If I already have FreeCAD installed on my system, if I can get the source code and compile it myself with a few terminal commands, why should I try Conda?

The first step towards an alternative is the option to use an alternative... You really do not have to care about it, but actually I think having a utility to test freecad on 3 platforms is another big advantage. Also it's nice to keep dependencies synchronized. One time updating one dependency means it get updated for all supported platforms.
If this feels like advertisement, I don't want to do that. Try it if you want otherwise stay with whatever fits best.

In the end it's a no-no for freecad to fully depend on any packaging service. The possibility to choose your dependencies is very important for foss and in general.

[vocx]

...
If this feels like advertisement, I don't want to do that. Try it if you want otherwise stay with whatever fits best.
...

I don't see it as advertisement. It's fine. I just want to understand this Conda point of view, because it seems like a lot of your proposals about organization in FreeCAD depend a bit on how it's packaged or distributed. At least that's the impression I get from threads like Multiple versions of freecad in one system (bug or feature), Workbench-Starterkit refactor.

It seems to me that there is something that inherently needs to change in order to support Conda, so I just want to understand exactly what that is, or in which way it's better. Making things better in all platforms is good, I'm not opposed to that, I just want to understand your point of view.

[looo]

I don't see it as advertisement. It's fine. I just want to understand this Conda point of view, because it seems like a lot of your proposals about organization in FreeCAD depend a bit on how it's packaged or distributed. At least that's the impression I get from threads like Multiple versions of freecad in one system (bug or feature), Workbench-Starterkit refactor.

In my eyes it's the opposite. Virtual environments address the issues encountered when having multiple versions of the same package in one system. For virtual environments, you don't need multiple versions in one environment because you can have different environments for different dependency-trees.

So for conda it isn't a problem to place an initialization module (abstraction layer for FreeCAD) in sys.path. Because there is always only one FreeCAD-version and only one python version in one environment. So we always know which FreeCAD-version to import.
This is not the case for ubuntu/debian if more versions of FreeCAD are installed. There is simply no way for python to know which FreeCAD version to import by default. And letting this up to the user is also a no no, because code dependent on FreeCAD should run without getting into contact with such details...

[vocx]

...And letting this up to the user is also a no no, because code dependent on FreeCAD should run without getting into contact with such details...

I disagree with this. The users should always be accountable for what goes in their system. We cannot treat the users like babies, and hand hold them entirely. I think if the documentation is good enough, a competent user should be able to figure things out when it comes to dependencies, paths, and things of that nature. With that said, if this Conda thing really improves things as you say, fine, I'm not opposed to that; I just think the user should have some semblance of competence to start.