chennes wrote: ↑Fri May 12, 2023 3:12 pm
n which case I propose a closed tag system based on a flat text file hosted on GitHub for the time being. Adding tags to that file adds them to the interface.
As in the proposal, I am planning to implement tags in database too. What are the suggestion on using database for storage of both metadata and its related tags?
The ER model for the same is given in the proposal.
I think you can only put tags in a database if you are prepared to implement a tag management UI. Otherwise it is too difficult for someone to propose new tags, and you basically have to use an open system, which will be a mess.
I think you need to really consider whether a SQL database is the right direction at all. It has a number of major downsides, including requiring another authentication setup, and making it much more difficult for an individual or company to host a private repository of parts. I am very interested in others’ feedback on this point.
The system is designed for flexibility so that it is easy for any individual or company who has an online repository of parts (maintained by themselves). They just need to specify the URL to the component and fill the required metadata fields.
I think I might be missing some point on how will this be difficult for individual/company to add new parts to the system.
The database will be exposed through an API framework (like Flask, FastAPI, etc) so the authentication can be handled in that layer.
Like in most open-source projects, we like to keep things as simple as possible in FreeCAD... We must always think about other developers working on top of our work, and possibly too non-developers, that is, just FreeCAD users with little or no programming knowledge annoyed by something and deciding to try to fix it themselves
If you are designing some form of database, I'd go for something VERY simple like a text file or a csv file. Something the FreeCAD UI or any other UI or script can easily fetch and parse, with no extra depencency or hassle.
Ok I get it. I think using of csv files will be much better as it will preserve the original idea of using tables based storage and will also be easy to understand and parse. And what about using an ORM on top of it?
From what I am thinking,
using ORM we can achieve a certain level of abstraction
make code more pythonic
relationship management can be easier
It might also increase the readability of the code
- Even with a "database" structure to store some metadata such as tags, most of the info you'll need to retrieve will still be stored into files. For example, the size, the number of objects, the type of model, the thumbnail of a FreeCAD model, all this you'll retrieve by reading files, not accessing a database
- We must expect people simply throwing models into the library, like they do now. They won't fill forms, they won't format things properly, etc. This is something good, it's like a wiki, the low entry barrier permits that people contribute a lot. But we must not rely too much on very clean/predictable data structure
- Also keep in mind to keep new dependencies low or even better, not add any new dependencies. When designing a new system that will be built in FreeCAD, this is a crucial point. Each new piece of software FreeCAD will depend on needs to be analysed very carefully in terms of license compatibility and general multiplatform availability, and as much as possible avoided, because it gives extra headache for maintainers and packagers.
Basically I would try not to design something too complex for now, as it could become counterproductive later on.
I'm not sure the dependency problem is as acute as @yorik is concerned about: the database part of this project is server-side, so doesn't need to be installed by end users. My understanding of the proposal is that even if @a.paritosh decides to use a database backend, the access will be via a REST API, so the frontend in FreeCAD doesn't need to know anything about it.
My whole concern about the database boils down to two things:
First, long-term maintainability of the server itself. We will need to keep the database software up-do-date, etc., which imposes extra work (mostly on @kkremitzki probably, though I should be able to help).
Second, if a company wants to have a fully private component library, they would need to build their own database server, which is a big ask compared to asking your IT team for some company-wide file storage space.