[Announcement] GPT4FreeCAD Plug-In

[revhappy]

Hi FreeCADers,

This is something I have imagined for a while. Imagine natural language prompts to create a model. I have an exciting release for a new plug in, GPT4FreeCAD. This plug in integrates OpenAI's GPT-4 model to generate Python scripts for creating sketches and 3D models based on user input.

I decided to focus on this free, open source release for FreeCAD since it goes along with the philosophy and open source nature of FreeCAD

This is a fun demo that works well with precise prompts, creating and even editing models, but it can and does fall apart... but when it works, it's magic! Consider this as a starting point for further development and improvements.

Features:

GPT-4 integration for generating Python scripts based on user input
Basic UI for user input and execution of generated code
Undo functionality
Conversation history
Please note that you will need an OpenAI API key to use this workbench.

You can find the GPT4FreeCAD workbench on GitHub: https://github.com/revhappy/GPT4FreeCAD

Looking forward to your feedback and suggestions!

[onekk]

...

Better not, as GPT4 is not a coding tools.

It is a sophisticated language parser and predicting system, if you see how experts in AI will say about generated code you will easily see that is simply based on aggregation and scoring of found elsewhere python code.

Regards

Carlo D.

[revhappy]

Mehh, as of now.. it's for fun and imagining future potential. Whether you like it or not, 'Prompt CAD' will be a thing. Also, don't knock it before you try it.. I've shown this to professional CAD heads and they were blown away because it got their wheels turning, and some with very precise prompts got it to do some cool stuff.

Also, I don't adhere to 'Better Not'.. in order to push things ahead, you have to ignore it.

[freedman]

I could see something like this used on an assembly level. There could be preset connection points designed into a model and those could be used as verbal prompts.

The concept of using it as a basic design tool; When working on a design I would say 80% of my focus is on the visual image that I have created in my mind, repeatedly running thru a list of requirements for the model. I don't see the GPt4 helping me design, I see it as just another type of distraction. The distraction being "I have a thought" and now I have to create it to see if it works, verbal or CAD drawn, what's the difference, it's going to be very technical so it takes a bunch of time to put it in the 3D.

[revhappy]

Fair points, and yes... like you, I'm more of a visual designer and playing with shapes is how you can get exactly what you want.

I think of the future potential here, and lowering the bar of entry for CAD users, the same way software CAD tools did vs drafting on tables.

An example prompt I gave it was 'A ninja star, 3D printable with a diameter of 6 inches' and it went to work and presented an STL file of a rough 3D printable object that.... while not perfect was close enough to what I imagined. And think that the GPT models are only going to get better from here as well as potential for fine tuning/training on a specific trough of data, such as python scripting for CAD.. and you can imagine where we can end up.

[rstech]

This could be a very interesting tool for reversed engineering. A feature recognition from scanned 3d data?

[onekk]

...

Also, I don't adhere to 'Better Not'.. in order to push things ahead, you have to ignore it.

But AI is already here, sadly "this AI" is only a mere way for someone to leverage a "not mature" things to make money.

Speaking of GPT (Generative Pre-trained Transformer) in general.

You are dealing with a "aggregator" and a "score system" (the pre-trained part) that is validated by humans, (some news tells that they are mostly from Africa an payed 2 dollars per hours) not much in innovations.

It is not a news that it will pass university tests as they are based usually on questions and answers, and probably these questions an answers are a "limited set" on which you have to choose, and even here a carefully taylored test will defeat the AI (It has been done by some AI experts to test the system).

Actually AI is not very far from the "old inference motors" dating around 1970 (search for MYCIN and EMYCIN around), used in creating medical diagnosis that even in this old time and with far less computing power have been surpassed in identifying disease from elaborating images and medical data the score of a "panel of experts".

Sorry fo the long post, I will not post here anymore.

Kind Regards

Carlo D.

[freedman]

I don't have much interest in this but I do have interest in making CAD faster. Personally I don't like CAD, it puts me in a chair for hours and in general can be hard on ones health. I do like to design so I use CAD as best I can to do what I like. Most all my efforts here on FreeCAD are to make work go faster and easier, that way I can get the job done and go outside or go for a walk.

If GPT4 can cut my time in the chair then I have some interest. If you really get some kind of test bed going then I would start with trying to get known objects (from a library) to pop up on the screen, maybe something like quick access fasteners.

Good luck

[revhappy]

The idea with this is open source. I'm not making any money from this. You can download it and play with it within 5 minutes given you have an Open AI API key(they give free credits initially). I did this only because I think the idea is cool and want to share and perhaps people can build upon it.

Also, as GPT models can eventually be fine tuned and learn from context, and as they get more intuitive, this will only get better.
These are early days.

[naxq0]

Sounds cool

I'm excited to see where these things go, I was blown away by https://github.com/openai/shap-e