Sketcher performance - why is Python the culprit?

[abdullah]

UI c++ code, calls Python code, the latter calling c++ code.

This architecture decision accounts for a huge percentage of the success of FreeCAD, as effectively it enables about everything being controllable from Python, making macros and even Python WBs possible, which in terms of flexibility is difficult to match. The fact that Python, specially in the sketcher, essentially calls c++ code, should not make it inherently inefficient.

I do not use VC or Windows, so I am not sure what is actually being considered under Python in VC's profiler.

When it is necessary to add multiple geometries and constraints, it is best from a performance point of view to do it all at once:
https://github.com/FreeCAD/FreeCAD/blob ... Polygon.py

When adding each object properties needs to recalculate, the solver needs to be called and this is time, changes in properties generate calls to objects monitoring changes in properties, which involves also redrawings at UI level, that is also time. Again I do not know how c++ code called from Python is accounted for in your profiling.

That said, it would be great if you could dig out what exactly is taking the Lion's share of that time. It may be improvable.

[acolomitchi]

UI c++ code, calls Python code, the latter calling c++ code.

This architecture decision accounts for a huge percentage of the success of FreeCAD, as effectively it enables about everything being controllable from Python, making macros and even Python WBs possible, which in terms of flexibility is difficult to match.

Don't take me wrong, I'm not contesting the benefits of scripting or the decision to base it on Python, my "why" is not challenging the things.
My "why" is asking for reasons I'm seeing what I'm seeing and/or maybe someone pointing to eventual mistakes that I may be making when looking into the guts of FC. Or any piece of information that would help me get a better understanding and help me get to the answers myself.

The opening "UI c++ code, calls Python code, the latter calling c++ code." is one such piece of the puzzle, thank you for that.

The fact that Python, specially in the sketcher, essentially calls c++ code, should not make it inherently inefficient.

That's part of the things I'm trying to understand (and, possibly, if other obligations allow) mitigate.
I mean, I started to look at the code and find things like

Code: Select all

deriv += ( (-n2.dx)*n2.y / pow(n2.length(),2)  +  n2.dy*n2.x / pow(n2.length(),2) )

(using pow, which promotes everything to double and performs the equiv of exp(2*ln(x)) in this case. And this to get the square of the len of n2 - why did I compute the sqrt(n2.x*n2.x+n2.y*n2.y) in the first place if I actually need now the dot(n2, n2)? And then, why divide all the terms of a sum first and add them later instead of summing all and dividing a single time?)
My very first impulse was to correct that (and many others), but then I asked myself if there's gonna be of any notable benefit.
This is how my slide into looking at the performance as a whole started. (and don't ask me why I started to look at the code)

I do not use VC or Windows, so I am not sure what is actually being considered under Python in VC's profiler.
...
When adding each object properties needs to recalculate, the solver needs to be called and this is time, changes in properties generate calls to objects monitoring changes in properties, which involves also redrawings at UI level, that is also time. Again I do not know how c++ code called from Python is accounted for in your profiling.

No worries, mate, I'll get to the bottom of it.

When it is necessary to add multiple geometries and constraints, it is best from a performance point of view to do it all at once:
https://github.com/FreeCAD/FreeCAD/blob ... Polygon.py

Oh, wow. Thanks, adding list of geos and list of constraints, that's new to me, nothing that I stumbled upon suggested it as possible (the API wiki pages are a little bit... ummm... sketchy).

That said, it would be great if you could dig out what exactly is taking the Lion's share of that time. It may be improvable.

Pinky promise, that I'll do. When I'm getting to the bottom of it, I'll probably start a new thread.

[abdullah]

Don't take me wrong, I'm not contesting the benefits of scripting or the decision to base it on Python, my "why" is not challenging the things.

I did profiling in the past too and there was quite a lot of improvement in some areas thanks to it. I am happy if you identify some area of improvement, I am happy if you decide to contribute some code and I am happy if you are just interested in the guts and learn your way through this way.

This is how my slide into looking at the performance as a whole started. (and don't ask me why I started to look at the code)

Oh! It is always good to look at the code. You do not have to justify your answer. I did the same some years ago

[onekk]

...
Oh, wow. Thanks, adding list of geos and list of constraints, that's new to me, nothing that I stumbled upon suggested it as possible (the API wiki pages are a little bit... ummm... sketchy).
...

Sadly there is not yet a real Sketcher scripting page, like many other WB in FreeCAD.

But from my personal experience, I could say that what abdullah is saying it true, (when he speaks about the scriptability of FC).

Take in account a couple of things, that in future will improve things:

- FreeCAD is not static, and usually if something could be improved, developers are open to improvement and suggestion. (Some of them maybe seem hostile at a first glance, but usually they are not)
- Python with 3.11 version is improving his speed as lead developers have the speed improvement among their goals, so probably with future versions, (maybe even in 1.0) something will be visible on this side.

Speaking of python, some work could be done in trying to optimize the code, as python is very flexible, something is will suffice to use tuples instead of lists for "disposable" intermediate constructs, to save memory and speed as tuples are immutable.

Sadly there is no a real way of "dispose things" once used, but usually garbage collection is efficient.

Sometimes it will hang, due to error in code and sadly the only way you have is to close FreeCAD, (sadly this happens to me form time to time), but even if you use the internal editor, the first thing that FreeCAD will do when launching a script is to save the file, so your work is safe, and you could amend things before relaunching the script.

Usually the real "bottleneck" is OCCT, in the sense that what you ask OCCT to do is a computational heavy task, and FreeCAD is "single threaded" so even if you have a 16 core processor, it will use one core only (something is changing, but introducing such changes is very "delicate" and so things are slow.), something could be done and in FreeCAD some part of code have been optimized to use more thread to execute some operation in parallel, (or at least someone is working to do such changes, as this matter was discussed sometimes in developer forum).

This is not realted to Sketcher, but probably could be interesting.

I've even manage to obtain some fillets (that are very delicate things) using Python and Part WB, so I'm quite happy with it (the real Fillet operation performed by OCCT),

part_wd_filllet1.png (20.94 KiB) Viewed 1507 times

part_wd_filllet2.png (20.8 KiB) Viewed 1507 times

part_wd_filllet3.png (20.68 KiB) Viewed 1507 times

part_wd_filllet4.png (9.39 KiB) Viewed 1504 times

Strangely enough even if the surface was cuved it has worked at the first attempt, the only drawback is that I had to find the edge number using the Gui to pass it correctly in the code:

Code: Select all

    obj_b2 = obj_b1.makeFillet(1.5, 1.5, [obj_b1.Edges[1]])

I've fiddled a little, as error reports were not too easy to guess but usually some print() statement will make some light.

But probably I'm going off topic now.

Sorry for any inconvenience.

Kind Regards.

Carlo D.

[acolomitchi]

This is how my slide into looking at the performance as a whole started. (and don't ask me why I started to look at the code)

Oh! It is always good to look at the code. You do not have to justify your answer. I did the same some years ago

Yeah, nah, mate. Wasn't trying to stop anyone asking for justifications, rather the implied continuation would be along the line of "it's embarrassing".
Rookie mistake - spending more than a day to connect the "wire" that I obtained by making an 90+ elements array of the basic "ridge" pattern (to obtain a ridged surface by padding the resulted sketch) using the GUI. 'bout 300 elements in a sketch with zillions of coincident constraints to be manually added one by one after dragging the elements resulted from the array operation one by one in its approx place before starting to constrain it.

This is how I got into scripting (and saw that even that is not the way to go) and this is how I learned that "keep you sketches simple, make more of them if you need to" is the way of life in Part Design. Hard leaned one too, won't forget it that easy.

The best part: I do have an explanation for the impacted performance (new post pending).

[acolomitchi]

(Breaking news for newbies like me:)
Python is NOT the culprit, new data shows

[tonyaimer]

Can I make an observation here plus a question.

I have read a little on how the sketcher works and I am wondering if some of the issue is to do with
large arrays caused by complex sketches.

If there are say 100 constraints perhaps a 100x100 matrix is created ie 10 000 elements.

If there are 2 sketches of 50 constraints perhaps 2x50x50 is created ie 5 000 elements.

Could this be a reason, or a contributing factor, to the advice that we should have more
simpler sketches when possible because it keeps the number of elements down.

I am assuming that the sketcher module does not use any sparse matrix techniques - which I
studied in some depth and needed when I was writing my own structural analysis package about 35 years ago.

I would appreciate comments in this regard from Abdullah as this is his baby.

Regards

Tony Aimer

[adrianinsaval]

If there are say 100 constraints perhaps a 100x100 matrix is created ie 10 000 elements.

If there are 2 sketches of 50 constraints perhaps 2x50x50 is created ie 5 000 elements.

kinda, but more than reducing the total amount of elements the idea is to reduce the amount of elements that have to be solved at once, when you split your sketch only one of them can be edited at a time so only one of them will be solved at a time, so only 50 constrained DOF get solved so a 2500 element matrix vs 10000 (using your example)

[chrisb]

Could this be a reason, or a contributing factor, to the advice that we should have more
simpler sketches when possible because it keeps the number of elements down.
...
I would appreciate comments in this regard from Abdullah as this is his baby.

I can neither comment in Abdullah's name nor the solver side at all, but the reason to keep sketches simple is not only a technical one. From a user's point of view, simple sketches are, well, simpler and thus easier to understand and to maintain. The ancient romans knew already the principle of "divide et impera", divide and rule.
It seems that human brains works in analogy to a 2 or 3 dimensional matrix (sad to say: sometimes it is only a sparsely filled one), where problems of quadratic growth cause quadratic pain.

[onekk]

...
It seems that human brains works in analogy to a 2 or 3 dimensional matrix (sad to say: sometimes it is only a sparsely filled one), where problems of quadratic growth cause quadratic pain.

It is almost a rule that a "normal" human brain could not easily retains more than 9 or 10 sequences, (You could easily see that usually 10 is used very widely, ad not only in the Bible).

another thing is that usually odd things, are most likely to be retained, some arguments that the fact that you have 2 hands with 5 fingers, it is the proof that you must not use more than 10 numbers,

The following is mostly a joke, but if someone has some scientific writings to submit as a link I will be very happy to hear some other theories.

Some people argue than you have even the foot, so French use some strange ways of composing numbers, 80 is quatrevingt, so four time twenty, so probably they were used to use feet to count, and not only hands, although I 've ever find difficult to bend my footh fingers.

No scientific discusssion have been done about the 12 used by some anglo saxons to divide things, probably they have six fingers per hands or probably use two other appendices to count, but what appendices could be bent to retain the sequences maybe ears? Even here I have always tried to bend my ears with no luck.


Sorry for the jokes if they could be inappropriate.

Regards

Carlo D.