[GSoC 2023] UI tool for fetching online content

Contributions from the participants, questions and answers to their projects.
Discussions of proposals for upcoming events.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
a.paritosh
Posts: 21
Joined: Fri May 05, 2023 4:36 am
Location: Ranchi, Jharkhand, India
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by a.paritosh »

chennes wrote: Fri May 12, 2023 3:12 pm n which case I propose a closed tag system based on a flat text file hosted on GitHub for the time being. Adding tags to that file adds them to the interface.
As in the proposal, I am planning to implement tags in database too. What are the suggestion on using database for storage of both metadata and its related tags?

The ER model for the same is given in the proposal.
Amulya Paritosh
GSoC 2023
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by chennes »

I think you can only put tags in a database if you are prepared to implement a tag management UI. Otherwise it is too difficult for someone to propose new tags, and you basically have to use an open system, which will be a mess.

I think you need to really consider whether a SQL database is the right direction at all. It has a number of major downsides, including requiring another authentication setup, and making it much more difficult for an individual or company to host a private repository of parts. I am very interested in others’ feedback on this point.
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
User avatar
a.paritosh
Posts: 21
Joined: Fri May 05, 2023 4:36 am
Location: Ranchi, Jharkhand, India
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by a.paritosh »

The system is designed for flexibility so that it is easy for any individual or company who has an online repository of parts (maintained by themselves). They just need to specify the URL to the component and fill the required metadata fields.
I think I might be missing some point on how will this be difficult for individual/company to add new parts to the system.

The database will be exposed through an API framework (like Flask, FastAPI, etc) so the authentication can be handled in that layer.
Amulya Paritosh
GSoC 2023
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by yorik »

Like in most open-source projects, we like to keep things as simple as possible in FreeCAD... We must always think about other developers working on top of our work, and possibly too non-developers, that is, just FreeCAD users with little or no programming knowledge annoyed by something and deciding to try to fix it themselves
If you are designing some form of database, I'd go for something VERY simple like a text file or a csv file. Something the FreeCAD UI or any other UI or script can easily fetch and parse, with no extra depencency or hassle.
User avatar
a.paritosh
Posts: 21
Joined: Fri May 05, 2023 4:36 am
Location: Ranchi, Jharkhand, India
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by a.paritosh »

Ok I get it. I think using of csv files will be much better as it will preserve the original idea of using tables based storage and will also be easy to understand and parse. And what about using an ORM on top of it?
From what I am thinking,
  • using ORM we can achieve a certain level of abstraction
  • make code more pythonic
  • relationship management can be easier
  • It might also increase the readability of the code
Amulya Paritosh
GSoC 2023
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by yorik »

Could be, but keep in mind:

- Even with a "database" structure to store some metadata such as tags, most of the info you'll need to retrieve will still be stored into files. For example, the size, the number of objects, the type of model, the thumbnail of a FreeCAD model, all this you'll retrieve by reading files, not accessing a database

- We must expect people simply throwing models into the library, like they do now. They won't fill forms, they won't format things properly, etc. This is something good, it's like a wiki, the low entry barrier permits that people contribute a lot. But we must not rely too much on very clean/predictable data structure

- Also keep in mind to keep new dependencies low or even better, not add any new dependencies. When designing a new system that will be built in FreeCAD, this is a crucial point. Each new piece of software FreeCAD will depend on needs to be analysed very carefully in terms of license compatibility and general multiplatform availability, and as much as possible avoided, because it gives extra headache for maintainers and packagers.

Basically I would try not to design something too complex for now, as it could become counterproductive later on.
User avatar
chennes
Veteran
Posts: 3877
Joined: Fri Dec 23, 2016 3:38 pm
Location: Norman, OK, USA
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by chennes »

I'm not sure the dependency problem is as acute as @yorik is concerned about: the database part of this project is server-side, so doesn't need to be installed by end users. My understanding of the proposal is that even if @a.paritosh decides to use a database backend, the access will be via a REST API, so the frontend in FreeCAD doesn't need to know anything about it.

My whole concern about the database boils down to two things:

First, long-term maintainability of the server itself. We will need to keep the database software up-do-date, etc., which imposes extra work (mostly on @kkremitzki probably, though I should be able to help).

Second, if a company wants to have a fully private component library, they would need to build their own database server, which is a big ask compared to asking your IT team for some company-wide file storage space.
Chris Hennes
Pioneer Library System
GitHub profile, LinkedIn profile, chrishennes.com
berniev
Posts: 247
Joined: Wed Apr 13, 2022 10:45 pm
Location: Oz

Re: [GSoC 2023] UI tool for fetching online content

Post by berniev »

The objections to database seem to be based on fear of traditional server hosted monoliths run by people in white coats.

Sqlite is a modern alternative. It is single file. It is ubiquitous. Yet can be huge.

For storing and retrieving data, nothing beats a database. Sure beats mucking about with XML file(s). Edit: Or csv files..
Last edited by berniev on Sat May 27, 2023 11:49 am, edited 1 time in total.
berniev
Posts: 247
Joined: Wed Apr 13, 2022 10:45 pm
Location: Oz

Re: [GSoC 2023] UI tool for fetching online content

Post by berniev »

And Sqlite is directly supported by Python out of the box. (Sqlite3)
User avatar
yorik
Founder
Posts: 13640
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: [GSoC 2023] UI tool for fetching online content

Post by yorik »

okay i'm convinced :D
Post Reply