About PDM for FreeCAD

Have some feature requests, feedback, cool stuff to share, or want to know where FreeCAD is going? This is the place.
Forum rules
Be nice to others! Read the FreeCAD code of conduct!
Post Reply
grd
Posts: 328
Joined: Wed Apr 13, 2022 5:13 am
Location: Eindhoven, The Netherlands

About PDM for FreeCAD

Post by grd »

I am sorry. I made a --huge-- mistake. I meant a PDM, not a PLM! And I changed my text. For the rest, my text is still correct.

My entire working life I worked with a PDM (Product Data Manager) [1]. The benefit of a PDM is that you can organize the data that you create and that no data is accidentally deleted, once it is stored and that you can always pull out a previous version of your file. And you can work with a team on your data. Once you check out then no one can modify your data. You can also version your files and release a project and once the data is released then the software can also -for instance- automatically create print files or STEP files.

So this means that a PDM has several benefits. The downside is the administrative extra work that is always being involved with a PDM.

My question is whether there are people interested in developing a PDM for FreeCAD.

The question is always how to start? What can we fork? Or are we gonna start from scratch? Both are possible but both are gonna have a lot of pain points because when you fork you are always gonna have a ton of issues.

Maybe I can answer that question, at least partially.

At Google, a couple of guys created a programming language, named Go[2]. Go is a highly structurized and fast PL. One of the core developers, Brad Fitzpatrick, he created Camlistore (now Perkeep) [3]. "Perkeep lets you permanently keep your stuff, for life."

So, Perkeep is an open source database (Licensed under the Apache License, Version 2.0), it is a -fast- database that allows you to put in any kind of data. And you can also add any kind of attributes to that data, and any data is private by default. It is cross platform. It runs on any OS. And it is also a web server (although that part should be rewritten AFAIK since we are dealing with CAD data).

Personally I think that Perkeep can be a very good candidate for a PDM engine. I know that it is not written in C++ nor Python. But the glue code will be written in Python of course.

Just an idea.


[1] https://en.wikipedia.org/wiki/Product_data_management
[2] https://go.dev/
[3] https://perkeep.org/
Last edited by grd on Thu Apr 28, 2022 8:57 pm, edited 1 time in total.
About Nim. Latest Release 2.0.2. Here is Nim in 100 seconds and a Nim package. There are Qt and OCCT packages.
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: About PLM for FreeCAD

Post by adrianinsaval »

os-tree or git itself might be better ideas instead of a google product
User avatar
Kunda1
Veteran
Posts: 13434
Joined: Thu Jan 05, 2017 9:03 pm

Re: About PLM for FreeCAD

Post by Kunda1 »

An aside, @realthunder has a git version control capabilities in his branch as 'Save as directory' which unzips the FCStd file in to it's own directory.
Alone you go faster. Together we go farther
Please mark thread [Solved]
Want to contribute back to FC? Checkout:
'good first issues' | Open TODOs and FIXMEs | How to Help FreeCAD | How to report Bugs
grd
Posts: 328
Joined: Wed Apr 13, 2022 5:13 am
Location: Eindhoven, The Netherlands

Re: About PDM for FreeCAD

Post by grd »

adrianinsaval wrote: Thu Apr 28, 2022 7:03 pm os-tree or git itself might be better ideas instead of a google product
Except when you deal with teams. But Perkeep is not a Google product. It is open source. Yes, the main developer was a Google employee (he left there) but it is not a Google product.
About Nim. Latest Release 2.0.2. Here is Nim in 100 seconds and a Nim package. There are Qt and OCCT packages.
user1234
Veteran
Posts: 3319
Joined: Mon Jul 11, 2016 5:08 pm

Re: About PDM for FreeCAD

Post by user1234 »

grd wrote: Thu Apr 28, 2022 4:11 pm I meant a PDM, not a PLM!
The difference is minor. The most programs, which can PDM, are mostly able to do PLM, the base is the same, it is only more extended.

Just my 0.03€:

A PDM/PLM in FreeCAD would be very good, it is needed for bigger projects. I also thought about that and tested a little bit. But my skills and time to learn is atm way too limited.


Something like this never work with git or similar, because:

- in CAD projects you only can work with dedicated states, baselines, revisions, flags ..... . Working with git (or similar) is not practical in that case.

- the diffs would be every save too big, even for FreeCAD, which the documents are text based, because you only can change whole files. It is never possible to make line per line diffs in CAD. And also if, you save way too often and the diff lines are often also very very big. It is just not practical.


I would do that with a postgresql database, which has python API, which would be perfect for FreeCAD (as addon, because many do not need that).


Greetings
user1234
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: About PDM for FreeCAD

Post by adrianinsaval »

- in CAD projects you only can work with dedicated states, baselines, revisions, flags ..... . Working with git (or similar) is not practical in that case.
can you explain why? are you not saving many versions of the file anyways? or what is the difference? If only the diff of the xml files are saved having different versions wouldn't take too much space. I'm not familiar with how PLM/PDM work.
user1234
Veteran
Posts: 3319
Joined: Mon Jul 11, 2016 5:08 pm

Re: About PDM for FreeCAD

Post by user1234 »

Hard to explain in only a few words/sentences.

For example

- if you are not finished with construction, but are able to prelease for materials procurement, needs a separate state (often needed for ERPs like SAP or generally for overview).

- if you have more revision states of one part, which are only partly replaceable for future machine/projects or as replacements of existing machines.

- if some clients wants separate or extra quality checks / treatments or whatever

- if you make a bigger generally construction state check (FEM, CFD, any simulation or whatever), you need a discrete baseline

- if you need to disable (i do not know the right word for that, in german the term is "sperren") discrete revisions or parts

- much more

There also norms for that (partnumber, revision, state, owner, checker, approver, checkout, fork from, BoM (and also its number, revison, state, ..... ), .....)


This sound like much work, and it is. But if you have to maintain or repair machines for that, i make the repairing much easier. I also have now IRL one machine to repair without that. You need minimum 3 times more time to repair without all the information (of yours it depends of the complexity of the machine, but you will always much faster). And when the clients makes 25€ to 1000€ per hour with that, every day costs. Now you will think, why i do need all the old information while producing or all former repairing? Empirical you really need all information. Often when repairing, you only repair the symptoms, but not the reason why the machine is damaged. So when repairing, you also look on the parts history and check, if something went wrong in the past time. And sometimes, that is the case. And also you also need sometimes the old datas for legal issues.


Greetings
user1234
grd
Posts: 328
Joined: Wed Apr 13, 2022 5:13 am
Location: Eindhoven, The Netherlands

Re: About PDM for FreeCAD

Post by grd »

user1234 wrote: Thu Apr 28, 2022 10:59 pm I would do that with a postgresql database, which has python API, which would be perfect for FreeCAD (as addon, because many do not need that).
I agree. But if you think about minimalistic, then sqlite would be the better choice because you don't need to set up a SQL database. And sqlite can also have multi-user support [1] and python access [2].


[1] https://stackoverflow.com/questions/510 ... iple-users
[2] https://docs.python.org/3/library/sqlite3.html
Last edited by grd on Fri Apr 29, 2022 9:20 am, edited 1 time in total.
About Nim. Latest Release 2.0.2. Here is Nim in 100 seconds and a Nim package. There are Qt and OCCT packages.
grd
Posts: 328
Joined: Wed Apr 13, 2022 5:13 am
Location: Eindhoven, The Netherlands

Re: About PDM for FreeCAD

Post by grd »

user1234 wrote: Fri Apr 29, 2022 12:25 am Hard to explain in only a few words/sentences.

For example

- if you are not finished with construction, but are able to prelease for materials procurement, needs a separate state (often needed for ERPs like SAP or generally for overview).

- if you have more revision states of one part, which are only partly replaceable for future machine/projects or as replacements of existing machines.

- if some clients wants separate or extra quality checks / treatments or whatever

- if you make a bigger generally construction state check (FEM, CFD, any simulation or whatever), you need a discrete baseline

- if you need to disable (i do not know the right word for that, in german the term is "sperren") discrete revisions or parts

- much more

There also norms for that (partnumber, revision, state, owner, checker, approver, checkout, fork from, BoM (and also its number, revison, state, ..... ), .....)


This sound like much work, and it is. But if you have to maintain or repair machines for that, i make the repairing much easier. I also have now IRL one machine to repair without that. You need minimum 3 times more time to repair without all the information (of yours it depends of the complexity of the machine, but you will always much faster). And when the clients makes 25€ to 1000€ per hour with that, every day costs. Now you will think, why i do need all the old information while producing or all former repairing? Empirical you really need all information. Often when repairing, you only repair the symptoms, but not the reason why the machine is damaged. So when repairing, you also look on the parts history and check, if something went wrong in the past time. And sometimes, that is the case. And also you also need sometimes the old datas for legal issues.


Greetings
user1234
Yes, I agree with all of these shitty details. They are important. But how are we gonna manage that, these shitty details? We need to think about what approach we are taking. Top-down or bottom-up? We can think about every single detail and get lost about these or we can think about the global side and let go. We need to make a list of course and also a github repository (and use the issues and discussion from there to fill in the details). The problem is that we are all engineers. We only think about solving the problems and then releasing it but we are missing the bigger part: Dealing with the guys who actually use the software.

And of course, the web. Personally I don't have any problems with it but you can get lost very easily (such as how to secure your site), so I don't think that it is the right solution to start with, but I can be wrong. But you need to have a GUI with a parts list. That is inevitable. Probably with a MVC [1] approach.

And we need to update our findings to the public, regularly.


[1] https://en.wikipedia.org/wiki/Model%E2% ... controller
About Nim. Latest Release 2.0.2. Here is Nim in 100 seconds and a Nim package. There are Qt and OCCT packages.
heda
Veteran
Posts: 1348
Joined: Sat Dec 12, 2015 5:49 pm

Re: About PDM for FreeCAD

Post by heda »

at some point in time fc will need some sort of pdm, no doubt about it - question is who will bite the dust and start coding...

coding a pdm, hm, that is a whole lot of code, afaik some of the dragons have not originally done it themselves, but rather incorporated something by 3rd party (for whatever reason - suppose it is that it really is two separate core technologies that have nothing in common - well apart from that they from user perspective are joined at the hip)

i tend to think that if one considers too much to begin with it is an obstacle for progress.
and couple of prototypes ending up at the junk-yard is often the best way to understand what needs to be done when one does not have a clear view of the complexity / task at hand.

imho, pdm down to it's core is nothing else than a fancy file-handler, with possibility to define attributes along with the files and search/present views/define structures for those files. this is also how the problem should be approached...
well, there is also the "workflow" of things in pdm (like approver, push to other databases etc) but that is something that has to come after a functional file handling (and it really only makes sense if one really wants workflows that are pushing things around in multiple databases for different purposes - and that has to be coded for that specific environment anyways).

my 2 cents on the topic.

- do it in all python, do not bother with speed/scalability to begin with
- think limited capability to begin with
- make a "local" version first, i.e. just make it work from one fc session and one hard-disc
- make it without any changes in core (at minimum until the "local" version is sorted out for say 80% of functionality)
- use middleware for db, use an orm
- make it as a add-on workbench, with it's own independent tree-view
- make all "pdm" tasks independent of fc, i.e. freezing a status is something that is done in the "workbench" and fc is happily unaware of that status

practically this means:
- sqlalchemy as middleware, or similar, use sqlite as backend
- fc would always work from files on hard-disc in a temp folder, when opening a file from your pdm the first step is to create that temp folder, dump the db content in there and let fc load the files from there, and when saving to pdm it is just to upload the temp folder to the db.
- as option (a different working mode), do not store files in db, just store file-path - i would start with this one

a route like this could be done by anyone, and with time it could grow into something really useful - large tasks like this happens over time and different people will add small things here and there if there is a decent skeleton to begin with..., but without that skeleton nothing will happen (until someone just decides to go ahead and make a skeleton for others to build on).

once you have a skeleton up and running with decent functionality for the "pdm" part and a decent gui to match it, then it is time to start thinking what core changes are needed to make it an integrated pdm (i.e. getting rid of the temp folders).

ps: those s-y things are really what things work or not, so I tend to think that the s-y things are what matters (if you intend to actually make something - of course it does not matter so much if you just want someone else to code it...), as example - what is your plan to let people have the same name but different content, in other words - a global unique identfier is needed in the pdm db, and the name presented (i.e. "filename") is really only a "label" (to use fc-lingo) - come to think about it - this point is quite similar to tnp in fc :-)

ps2: if not clear by now, this is a push for you to just go ahead and start coding, once having started to code it is easier to see what the "next steps" are, and also it is easier to grasp what problems one really has to solve to make it work. ;)
Post Reply