GSoC 2020: A better IPython and Jupyter Notebook Integration for FreeCAD

[wmayer]

I had a look at the VRML2 export when doing from the Gui. I forget an important function call to correctly export the content as VRML2:

Code: Select all

import FreeCADGui
Gui = FreeCADGui
Gui.setupWithoutGUI()

doc = App.newDocument()
doc.addObject("Part::Box","Box")
doc.addObject("Part::Cylinder","Cylinder")
doc.addObject("Part::Sphere","Sphere")
doc.addObject("Part::Torus","Torus")
doc.recompute()

from pivy import coin
root = coin.SoSeparator()
for obj in doc.Objects:
    root.addChild(Gui.subgraphFromObject(obj))

# when exporting to VRML2 the Switches must be replaced
root2 = Gui.replaceSwitchNodes(root)
tovrml2 = coin.SoToVRML2Action()
tovrml2.apply(root2)
vrmlRoot = tovrml2.getVRML2SceneGraph()
vrmlRoot.setInstancePrefix(coin.SbString("o"))

out =  coin.SoOutput()
out.openFile("/tmp/output.wrl")
out.setHeaderString("#VRML V2.0 utf8")
wa = coin.SoWriteAction(out)
wa.apply(vrmlRoot)
out.closeFile()

This fixes all the parser errors. However, it still doesn't fix the bad rendering that is caused by a Coin3d bug. It exports a Normal node with an incorrect number of elements. To fix the VRML2 output from within C++ we added a class SoVRMLAction as a workaround to change the SoNormalBinding before converting to VRML2.

[wmayer]

See git commit f948a39bdb

Now you can write

Code: Select all

doc = App.newDocument()
doc.addObject("Part::Box","Box")
doc.addObject("Part::Cylinder","Cylinder")
doc.addObject("Part::Sphere","Sphere")
doc.addObject("Part::Torus","Torus")
doc.recompute()

from pivy import coin
root = coin.SoSeparator()
for obj in doc.Objects:
    root.addChild(Gui.subgraphFromObject(obj))

out=open("/tmp/output.wrl","w")
Gui.exportSubgraph(root, out)
out.close()

For later having FreeCADGui.exportVRML2(documentObject) or better FreeCAD.exportVRML2(documentObject), available that will return the VRML2 as text instead of saving it to a file might be nice.

The second argument is a file-like object which can also be an io.StringIO object.

So, you can also write

Code: Select all

...
import io
out = io.StringIO()
Gui.exportSubgraph(root, out)
txt = out.getvalue()

[wmayer]

With Py3 it matters if a bytes or str object must be written to a file-like object depending on whether it's opened in binary mode or not.
With git commit 864555bc9 the underlying C++ streambuf determines this at runtime.

This change now allows to also write a compressed VRML file:

Code: Select all

...
import gzip
out = gzp.open("/tmp/output.wrz", "wb")
Gui.exportSubgraph(root, out)
out.close()

[kryptokommunist]

This fixes all the parser errors. However, it still doesn't fix the bad rendering that is caused by a Coin3d bug. It exports a Normal node with an incorrect number of elements. To fix the VRML2 output from within C++ we added a class SoVRMLAction as a workaround to change the SoNormalBinding before converting to VRML2.

See git commit f948a39bdb

Now you can write

Code: Select all

...

from pivy import coin
root = coin.SoSeparator()
for obj in doc.Objects:
    root.addChild(Gui.subgraphFromObject(obj))

...

Thanks so much that's more than I asked for, so do I understand it correctly that the commit above still contains the incorrect number of elements bug? I tried running the example code and I still couldn't render the file except one time and then later it stopped working again which could also be something with three.js.

[wmayer]

so do I understand it correctly that the commit above still contains the incorrect number of elements bug?

No, the bug is avoided because no Normal node is created.
With the above Python code I get the exact same file as when doing over the GUI.

I tried running the example code and I still couldn't render the file except one time and then later it stopped working again which could also be something with three.js.

I used the tool view3dscene and there the rendering was good and it didn't complain about incorrect content of the files.

[kryptokommunist]

• Setup a server to host the current state of my project, so everyone can easily play around with it. (No hidden configuration, automate setup as much as possible. Maybe Docker?)

blocked, my friend who promised me a vserver stillt didn't get to setting it up for me

done, if you want to take a look just write me a pm and I'll send you the password for the server that runs Jupyter Notebook under https://freecad.kryptokommun.ist. Fun side note, thanks to the 16 vCores I can compile FreeCAD from scratch in 10 minutes \o/ Now I wonder what would happen with 32 cores...

photo6035193907702249954.jpg (675.82 KiB) Viewed 3637 times

• Create a working example in Jupyter Notebook that displays a simple three.js scene

I managed to create a running example with pythree.js but failed to do so with just using the javascript library three.js directly. Somehow Jupyter Notebook executes things differently.

• create a function that outputs VRML 2.0 equivalent to the export function of FreeCAD (utilising or copying it)

props to Werner, he did this!

• create an example that creates objects in Jupyter Notebook and then displays them inside

Even though I can't get the javascript to run inside Jupyter, three.js loads the exported VRML2 (exported via GUI) correctly: demo. Choose freecad_exported in the selection menu on the top right corner.

My goals for this week are (kanban link):

• open an issue describing my problems with importing the three.js javascript library in the Jupyter Notebook repo and if they can't help there also in the three.js repository and potentially in the pythree.js repository.
• build a simple example in (py)three.js that displays an object and attaches labels to it's surfaces and edges (@ickby reminded me that this would be a major use case, since the naming is abitrary and one would need to check the names to continue coding in FreeCAD)
• build a function that iterates over the scene graph that just extracts the edges and outputs them into an (py)three.js compatible format (this is plan B in case VRML2 won't work or makes it hard to incorporate the label information into the 3D object)

[yorik]

It might also be interesting to explore the path of taking the FreeCAD .iv output, even with its "FreeCAD-specific" slang, and convert it to something more meaningful to threejs (maybe json? https://github.com/mrdoob/three.js/wiki ... e-format-4 ). I started something here: https://github.com/yorikvanhavre/WebToo ... ab.py#L172 to convert FreeCAD .iv into canonical .iv (mostly FreeCAD extends some entities that already exist in pure iv, so it's not too hard) but it still has some hiccups...

[kryptokommunist]

• open an issue describing my problems with importing the three.js javascript library in the Jupyter Notebook repo and if they can't help there also in the three.js repository and potentially in the pythree.js repository.

This week I focused on getting an Jupyter Notebook widget example running as I found a great example skeleton file to do so (widget folder). Building on the skeleton I tried integrating an pythreejs 3D view. The main places this is happening are here: extension javascript, python code on the server side, python code in notebook. This work is still ongoing, but it looks like there is someone willing to help over at pythreejs: issue.

• build a simple example in (py)three.js that displays an object and attaches labels to it's surfaces and edges (@ickby reminded me that this would be a major use case, since the naming is abitrary and one would need to check the names to continue coding in FreeCAD)

Not finished yet, currently I already managed to construct simple geometries from vertices and triangles between them. Now I am trying to figure out how one could include the labels in that.

• build a function that iterates over the scene graph that just extracts the edges and outputs them into an (py)three.js compatible format (this is plan B in case VRML2 won't work or makes it hard to incorporate the label information into the 3D object)

Not touched at all yet.


My goals for this week are (kanban link):

• continue resolving the issue as the response from the maintainer looks promising
• extending my current manually built pythreejs example by attaching labels to it's surfaces and edges
• build a function that iterates over the scene graph that just extracts the edges and outputs them into an (py)three.js compatible format

[kryptokommunist]

It might also be interesting to explore the path of taking the FreeCAD .iv output, even with its "FreeCAD-specific" slang, and convert it to something more meaningful to threejs (maybe json? https://github.com/mrdoob/three.js/wiki ... e-format-4 ).

The JSON format looks good, pretty much along the logic of the (py)threejs API. My thinking right now is wouldn't it be better to just create the scene using the pythreejs API and then using it's function to generate the JSON? Except if the formats are very similar. Otherwise maintaining a parser would be an additional layer of complexity. But the JSON looks like a good candidate to replace the current WebGL export. There we could just run the conversion and in the end export the JSON embedded in HTML that loads threejs.

Does the Inventor format contain the surface and edge label names? If not are they currently in the scene graph at all or do I have to extract them from the scene objects? I saw that they definitely aren't in the VRML 2.0 format unfortunately.

[kryptokommunist]

• continue resolving the issue as the response from the maintainer looks promising

blocked, after a very quick first response no more signs of life from the maintainer But since I have gotten my other solution to the issue below to work this should be fine.

• extending my current manually built pythreejs example by attaching labels to it's surfaces and edges

not started yet.

• build a function that iterates over the scene graph that just extracts the edges and outputs them into an (py)three.js compatible format

done. With help from @ickby on explaining the scene graph to me on the mentor call I could finally get this working. The relevant Jupyter Notebook with code at the end of it can be found here.

This currently still has some issues. First of all I noticed that the scene graph in my Code doesn't match the scene graph I see running the same Code inside FreeCAD and then inspecting FreeCAD>Tools>Scene Inspector. The parts that didn't match were the contents inside Flat Lines, Shaded, Wireframe nodes within the SwitchNodes. Everything is there, just the ordering is not matching with everything. Aside from that there are some more details that I need to add, e.g. setting up the camera, lighting etc. based on the coin scene graph.

freecad_3Dview_in_jupyter.gif (931.25 KiB) Viewed 3376 times

My goals for this week are (kanban link):

• find out where in FreeCAD I can access the names of edges and surfaces
• extending my current pythreejs scene graph function to attach labels to it's surfaces and edges
• extract color, lines, camera, lighting correctly from the FreeCAD scene graph as there currently are still some issues