About PDM for FreeCAD

[Jee-Bee]

Why do we need information out of the database? We need information out of the assy's and parts.

I always need metadata from parts or assemblies for working (mechanical engineer and also sometimes on construction sides). Do not want always recreate a part, which already is in a PDM. So i have to search it. That is a PDM for. Or when i want to use a purchase part, which is already in the PDM, i want to look in the datasheet, if suits my needs. Or when someone want to know, how heavy some assemblies are for lifting or transporting, and i do not want to open the whole project files, which maybe took a while. Or if need to coat some parts (corrosion protection, wear protection, or what ever) and i want quick summarized the area and type of it (often used to send it in a ERP for a purchaser). Also i have seen PDM, which saves a *.jt file for a quick 3D preview, even if you want with metadatas like colored surfaces which are coated or measuring points.

Yes, that is big and an only develop step by step, but for that is a PDM for. If i only need it for a project files, then svn will do the job, but then it is not a pdm. So maybe a sql/svn compound is good? Data stored as sql tables, while the filedatas link to a svn?

I think you partly agree with each other. but where @grd expect that the part / assembly give metadata to the database. You expect that you can read the data back. I have to be honest that i didn't thought about datasheets as metadata(i had thought that that is data, but not meta)...
I like the graphical view without opening the file... but a simple png file is fine enough for me

[grd]

Why do we need information out of the database? We need information out of the assy's and parts.

I always need metadata from parts or assemblies for working (mechanical engineer and also sometimes on construction sides). Do not want always recreate a part, which already is in a PDM. So i have to search it. That is a PDM for. Or when i want to use a purchase part, which is already in the PDM, i want to look in the datasheet, if suits my needs. Or when someone want to know, how heavy some assemblies are for lifting or transporting, and i do not want to open the whole project files, which maybe took a while. Or if need to coat some parts (corrosion protection, wear protection, or what ever) and i want quick summarized the area and type of it (often used to send it in a ERP for a purchaser). Also i have seen PDM, which saves a *.jt file for a quick 3D preview, even if you want with metadatas like colored surfaces which are coated or measuring points.

Yes, that is big and an only develop step by step, but for that is a PDM for. If i only need it for a project files, then svn will do the job, but then it is not a pdm. So maybe a sql/svn compound is good? Data stored as sql tables, while the filedatas link to a svn?

I think you partly agree with each other. but where @grd expect that the part / assembly give metadata to the database. You expect that you can read the data back. I have to be honest that i didn't thought about datasheets as metadata(i had thought that that is data, but not meta)...
I like the graphical view without opening the file... but a simple png file is fine enough for me

I am pretty sure that in PDM's the metadata and the data is stored separately, because you don't want to search every file attribute in that file. That takes way too much time. So the metadata is stored apart from the file into the PDM.

How can we do that in FC? That is the question.

We can make a script. But how that script looks like depends on how you want to organize your data. I don't know whether you can add some hooks into smartsvn for instance so that the script always runs. I am sure that it is possible, but storing metadata within SVN and searching that metadata, to me looks impossible.

[Jee-Bee]

I don't think it matters that (meta)data is stored at different locations. It is totally valid to store for example mass in the database while it is also stored in the CAD file. As long as the user can read the (meta)data from the PDM it don't matter from where the data is coming from.
The UI can pinpoint to different information streams at the same time.

[Jee-Bee]

...

SVN is a revision control software. A repository. So there is a central repository, and each user checks the entire model out into his computer. Therefore, all files are stored locally, and all files are edited locally, meaning that anyone can edit and change any file *locally*. A "release" means that the user uploads his modified files to the central repository, but a user can upload only into directories where he has write access.

If 2 users edit the same file where they have write access, then there will be a conflict during upload, and the second one will not be able to commit : this needs to be dealt with by hand.

What you describe here are my two main issues that i was thinking of. I don't say svn is not the tool we need. But we have to thing about how to resolve the required states we think are needed.
I don't know how to handle the read/ write access(check out as it is called in Windchill) that a user can get. Maybe the situation read /write rights can be resolved in FC it self, but best would be in both sides.
I shall think maybe there are solutions for this...

I have thought if there could be a solution how to handle the points above.

I have maybe an idea how i think that svn can work for a mechanical working environment.
According to https://stackoverflow.com/questions/852 ... repository it is possible to add read-only access to a repository.
I thought that if every part / Assembly get his own directory and every directory is it's own repository. The PDM can set the svn so that all users have only read access to the folders /repositories. The first user that want write access can request that by checking-out the desired folder / repository. From now on all other users can't edit that file anymore until the first users check it in.
In that case all needed files can added to the folder as long as they are related to that part.

This can potentially a big part of a solution. the next big thing is how the hierarchy is handled and how to do the read permission of projects / (sub)systems.

[heda]

"SQLite and PostgreSQL. But basically this are already two iterations."

not really if one uses middleware where the actual database is abstracted away...
it then becomes a oneliner changing engine/backend from one to another.
it becomes the choice of the one deploying it (for themselves) what backend it will run on.

as pointed out by others, pdm is not only version handling of files.

imho one should abstract the version handling away in the same way as for example sqlalchemy abstracts the database engine away.
this way people can choose what backend they want for version handling (perfectly fine to only have one to begin with but with some consideration of how the api is done it should not be overly hard to write an additional plug-in for another backend).

the core of "cad-related" meta data would be it's own db, and would just hold links/triggers to pick up the right stuff from a version control backend.
in a perfect world the "front-end" pdm would handle for example credentials to the version back-end.

one way to solve that is to skip qt, and go for a web-framework, maybe flask, user handling etc is ready to go, gui would be made in browser, i.e. one could choose to work from browser, or just render that webpage within fc. could be run remote or locally.

this way one can keep the local set-up if one wants to, i.e. run a local server & use "fossil" as backend.
or scale it to kilo users.

meta data should not be fixed, but configurable, after the basic set of meta data, there will always be someone that would find it really useful to have their own metadata.

if someone wants a fullblown pdm, they have odoo/tryton or anything else that is out there..., however these at least to me seems more like erp than pdm, nothing bad in that, just an observation.

[Jee-Bee]

as pointed out by others, pdm is not only version handling of files.

Correct but at the same time it is its core function. If a user can't say if a file is changes while it shouldn't it has a serious issue!

"SQLite and PostgreSQL. But basically this are already two iterations."

not really if one uses middleware where the actual database is abstracted away...
it then becomes a oneliner changing engine/backend from one to another.
it becomes the choice of the one deploying it (for themselves) what backend it will run on.

...

imho one should abstract the version handling away in the same way as for example sqlalchemy abstracts the database engine away.
this way people can choose what backend they want for version handling (perfectly fine to only have one to begin with but with some consideration of how the api is done it should not be overly hard to write an additional plug-in for another backend).

I hope your right on this one. on the other hand a few have pointed out that all version systems handle versions in a matter that is needed in a PDM

the core of "cad-related" meta data would be it's own db, and would just hold links/triggers to pick up the right stuff from a version control backend.
in a perfect world the "front-end" pdm would handle for example credentials to the version back-end.

This is what i pointed out above

one way to solve that is to skip qt, and go for a web-framework, maybe flask, user handling etc is ready to go, gui would be made in browser, i.e. one could choose to work from browser, or just render that webpage within fc. could be run remote or locally.

this way one can keep the local set-up if one wants to, i.e. run a local server & use "fossil" as backend.
or scale it to kilo users.

The discussion how to setup the ui part isn't done yet. Only requirement on this for now is not javascript. @grd don't want that because it is insecure. since i'm not an expert in that direction i just accept it.

meta data should not be fixed, but configurable, after the basic set of meta data, there will always be someone that would find it really useful to have their own metadata.

Is not intended to fix the metadata but good to point it out!

[user1234]

Just for completeness, metadata should be custom. For example if i have a metadata like "Area", then i must have a property in the file "Area" and a column in the database "Area". Then when i PDMsave, the property should transferred to the database. If the property is empty, nothing is transferred. When PDMsave and the Area is there and there is no column, then i should raise an error (or nothing is transfered, this is only an example).

I have to be honest that i didn't thought about datasheets as metadata(i had thought that that is data, but not meta)...
I like the graphical view without opening the file... but a simple png file is fine enough for me

Yes, datasheets (or docs) should be are an extra table. Abut also this can be customized. But some pdf are every imported, and only a png is way too less for a datasheet.

one way to solve that is to skip qt, and go for a web-framework, maybe flask, user handling etc is ready to go, gui would be made in browser, i.e. one could choose to work from browser, or just render that webpage within fc. could be run remote or locally.

https://en.wikipedia.org/wiki/Django_(web_framework)
python, web framework, no qt.

Greetings
user1234

[grd]

Just for completeness, metadata should be custom. For example if i have a metadata like "Area", then i must have a property in the file "Area" and a column in the database "Area". Then when i PDMsave, the property should transferred to the database. If the property is empty, nothing is transferred. When PDMsave and the Area is there and there is no column, then i should raise an error (or nothing is transfered, this is only an example).

Of course metadata should be custom. An architect has different needs than a mechanical engineer. That is why we need to have roles and an Admin page where you can pick which role you can select. Apart from that we also need to make a selection to pick a material, and one for surface finish should be there too. But things such as weight should always be there. This counts too for assemblies.

This means that we also need to create a metadata source. A database for storing the metadata. In the past I thought about using sqlite but now I think about using PostgreSQL (or make it a select-able data source in the Admin page). But to start with sqlite of course because sqlite is much easier to deal with. Later on we switch go to PostgreSQL or whatever one you like.

But to think about it, this should also apply to the data source. This should all also be optional and selectable in the same Admin page. A data source should be optional (but it would become pretty useless without one) and the metadata source (again, this would become pretty useless if you don't take one). When you can select your data source, then you can also select which kind of data source you want to take. You can pick one such as SVN, GIT, or one of the proprietary ones, but the benefit of picking a data source is that you can also use that for checking in /out your data. Because every data source has it's own ways to do that. Personally I prefer SVN, but another guy insists on using GIT and yet another guys insists on using MS Azure, and the last one says that he wants to use your local filesystem. So we also need to have roles for having a data source. On wikipedia is a comparison of the data sources
https://en.wikipedia.org/wiki/Compariso ... l_software

one way to solve that is to skip qt, and go for a web-framework, maybe flask, user handling etc is ready to go, gui would be made in browser, i.e. one could choose to work from browser, or just render that webpage within fc. could be run remote or locally.

https://en.wikipedia.org/wiki/Django_(web_framework)
python, web framework, no qt.

I think that we should stay away from the web as long as we can, because sooner or later all your data is gone, copied or encrypted. Securing a web site is a tough job and we are no experts in that area.

But when you have the metadata source then you can make your own web site if you want to. That should be "easy".

[Kunda1]

An aside, when the time comes, it could be beneficial to use one of the projects listed in the Tracking FreeCAD collaboration tools thread (https://forum.freecadweb.org/viewtopic.php?f=8&t=62080) as a testcase for this PDM project?

• Thor - FOSS Robotic Arm: Repo Dev: @_AngelLM
• PUMA - 3D-Printed Multimodality Microscope: Repo Dev: @PJT Thread: link
• Index Machine - FOSS Pick and Place Machine - Repo Dev: (github) | (twitter)
• Monster Kossel Repo Dev: geoland86 (github) | (twitter) | (forum) Thread: link
• M19O2 - DiY Oxygen Concentrator Repo
• SnakeOil-XY https://github.com/ChipCE/SnakeOil-XY
• Cenital https://github.com/Bibliohack/Cenital

[grd]

An aside, when the time comes, it could be beneficial to use one of the projects listed in the Tracking FreeCAD collaboration tools thread (https://forum.freecadweb.org/viewtopic.php?f=8&t=62080) as a testcase for this PDM project?

• Thor - FOSS Robotic Arm: Repo Dev: @_AngelLM
• PUMA - 3D-Printed Multimodality Microscope: Repo Dev: @PJT Thread: link
• Index Machine - FOSS Pick and Place Machine - Repo Dev: (github) | (twitter)
• Monster Kossel Repo Dev: geoland86 (github) | (twitter) | (forum) Thread: link
• M19O2 - DiY Oxygen Concentrator Repo
• SnakeOil-XY https://github.com/ChipCE/SnakeOil-XY
• Cenital https://github.com/Bibliohack/Cenital

Of course, but it's gonna take time. I haven't written one line of code yet