Adding refraction index
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Be nice to others! Respect the FreeCAD code of conduct!
Adding refraction index
I was wondering if anyone has any objections or comments if I added "RefractionIndex" in the group Rendering?
My main focus is to have the data to be able to export it to ray tracers like cycles or luxrender. I did some digging about the refraction index and in theory it could also be derived from the materials relative permittivity and permeability. In PR https://github.com/FreeCAD/FreeCAD/pull/2467 someone already added relative permittivity.
My feeling is to add a refraction index anyway even if we run the risk that it could possibly clash with a value entered in RelativePermittivity. Maybe someone with a better understanding of physics can comment on this. I think the use cases and applicable wavelengths (optical vs microwave? Not sure) for those numbers are different enough.
I'm probably overthinking this
My main focus is to have the data to be able to export it to ray tracers like cycles or luxrender. I did some digging about the refraction index and in theory it could also be derived from the materials relative permittivity and permeability. In PR https://github.com/FreeCAD/FreeCAD/pull/2467 someone already added relative permittivity.
My feeling is to add a refraction index anyway even if we run the risk that it could possibly clash with a value entered in RelativePermittivity. Maybe someone with a better understanding of physics can comment on this. I think the use cases and applicable wavelengths (optical vs microwave? Not sure) for those numbers are different enough.
I'm probably overthinking this
- DeepSOIC
- Veteran
- Posts: 7896
- Joined: Fri Aug 29, 2014 12:45 am
- Location: used to be Saint-Petersburg, Russia
Re: Adding refraction index
Yes, but all these values are frequency-dependent. And at optical frequencies they are usually very different to those at DC-RF.Nocturnial wrote: ↑Sun Sep 01, 2019 8:56 pm it could also be derived from the materials relative permittivity and permeability.
Re: Adding refraction index
Thanks for checking. I'm going to wait for a bit to see if someone has a reason to not add "RefractionIndex" or has a suggestion to do it in a different way.
It might seem strange but if there's an agreement on the name and whether it can be added, I can split the changes into an arch part and a material part. The end goal is to render/ray trace without having to manually change materials and objects in whatever program you're using.
On the arch part it's about supporting multimaterials in windows and "glass" materials during export. On the material part it's about adding the refraction index and I think there's a problem with setting emissive, specular, etc.. colors. I still need to debug it properly but I think when you set it using the ui it uses values between 0-255 while it should be between 0.0 and 1.0. It's also possible I'm doing something wrong.
If the PR for the arch part get approved before the PR in the material part, the arch code changes will search for a refraction index that isn't there and export everything like it used to export.
If it's the other way around and emissive, specular works in the ui, then the old exporter code will suddenly export them because that code is already there.
It might seem strange but if there's an agreement on the name and whether it can be added, I can split the changes into an arch part and a material part. The end goal is to render/ray trace without having to manually change materials and objects in whatever program you're using.
On the arch part it's about supporting multimaterials in windows and "glass" materials during export. On the material part it's about adding the refraction index and I think there's a problem with setting emissive, specular, etc.. colors. I still need to debug it properly but I think when you set it using the ui it uses values between 0-255 while it should be between 0.0 and 1.0. It's also possible I'm doing something wrong.
If the PR for the arch part get approved before the PR in the material part, the arch code changes will search for a refraction index that isn't there and export everything like it used to export.
If it's the other way around and emissive, specular works in the ui, then the old exporter code will suddenly export them because that code is already there.
Re: Adding refraction index
I had this idea as well. It would be nice that you could export materials "ready for rendering", in say Blender. Therefore, some properties, even if not used in FreeCAD (metallness, roughness, reflectivity, etc.), could be there for when the object is exported to another software. In the case of Blender, they would use the appropriate options for the Principled BSDF shader with sensible defaults.Nocturnial wrote: ↑Sun Sep 01, 2019 11:56 pm It might seem strange but if there's an agreement on the name and whether it can be added, I can split the changes into an arch part and a material part. The end goal is to render/ray trace without having to manually change materials and objects in whatever program you're using.
I think the colors are internally saved as decimal values 0 to 1. There is code to convert different representations on the fly, so I don't think this is a problem in general.... I still need to debug it properly but I think when you set it using the ui it uses values between 0-255 while it should be between 0.0 and 1.0.
Last edited by vocx on Mon Sep 02, 2019 9:36 pm, edited 1 time in total.
Always add the important information to your posts if you need help. Also see Tutorials and Video tutorials.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
Re: Adding refraction index
The goal should indeed be "ready for rendering" but I was thinking of using a top-down approach. Instead of starting to add all the parameters for the principled bsdf shader, I was thinking to start with one parameter and check how all the programs interpret it.vocx wrote: ↑Mon Sep 02, 2019 1:47 am I had this idea as well. It would be nice that you could export materials "ready for rendering", in say Blender. Therefore, some properties, even if not used in FreeCAD (metallness, roughness, reflectivity, etc.), could be there for when the object is exported to another software. In the case of Blender, they would use the appropriate options for the Principled BSDF shader with sensible defaults.
Roughness is a good example of this. Read this discussion: https://developer.blender.org/T51895
In that case we have an "Rdq" and a "perceptual roughness" interpretation. In this case it's possible to easily convert them to each other. I'm not sure which one luxrender uses or maybe it uses another definition altogether.
Most of these engines use PBR and I'm hoping that a lot of parameters (after converting them) can be shared. Once those are done I'll add blender specific parameters.
Re: Adding refraction index
I wouldn't worry too much about "other" programs. Given that Blender is open source and popular, I think it makes sense first to focus on Blender, and similar open source projects. Later on we can see if other programs can be used as well with the parameters defined.Nocturnial wrote: ↑Mon Sep 02, 2019 11:27 am ... I was thinking to start with one parameter and check how all the programs interpret it.
...
Always add the important information to your posts if you need help. Also see Tutorials and Video tutorials.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
Re: Adding refraction index
Hehehe, I like that you put the word other in quotes.vocx wrote: ↑Mon Sep 02, 2019 9:40 pm I wouldn't worry too much about "other" programs. Given that Blender is open source and popular, I think it makes sense first to focus on Blender, and similar open source projects. Later on we can see if other programs can be used as well with the parameters defined.
You seem to feel strongly about this. Would you be willing to test my code and if so what OS are you using? It would be great if you're on linux because I'm on windows.
https://github.com/justnope/FreeCAD/tree/multimaterial
This is for the wavefront obj exporter. It's mainly intended to understand how the data is stored in freecad and how I can extract it. It's a work in progress.
https://github.com/justnope/FreeCAD-render/tree/cycles
This is the render workbench. Currently it only contains fixes for windows (the os) and a fix for using transparency in the cycles standalone of blender. The next steps would be to replace the diffuse shader with a principled shader during export. I also want to add support for multimaterials from the things I learned in extending the wavefront exporter.
It might not be what you wanted because it's not a direct import into the gui of blender. The data format for the standalone cycles renderer is different than what the gui can currently load AFAIK. It does give us a way to verify that the data we exported is correct for blender.
Re: Adding refraction index
I'm on Linux.Nocturnial wrote: ↑Tue Sep 03, 2019 1:21 am Would you be willing to test my code and if so what OS are you using? It would be great if you're on linux because I'm on windows.
Abouthttps://github.com/justnope/FreeCAD/tree/multimaterial
This is for the wavefront obj exporter. It's mainly intended to understand how the data is stored in freecad and how I can extract it. It's a work in progress.
Code: Select all
kinds = {"AmbientColor":"Ka ","DiffuseColor":"Kd ","SpecularColor":"Ks ","EmissiveColor":"Ke ","Transparency":"Tr "}
Code: Select all
kinds = {"AmbientColor":"Ka ","DiffuseColor":"Kd ","SpecularColor":"Ks ","EmissiveColor":"Ke "}
As I read in Wikipedia, there is a basic, standard OBJ format, but there are other versions with certain extensions, supported by different programs.
Ideally Blender should support the standard format, but if they only support their own Blender-OBJ flavor, then I have no problem supporting only that. Is this handled differently in Blender 2.79 and 2.80?
Also, please look into Yorik's Blender plugin to import FreeCAD files. Maybe there you get some ideas on what type of information Blender expects to receive from FreeCAD. In that case FreeCAD itself (FreeCAD.open(filename)) is used to import the shapes and material information into Blender.
Also, please provide a test file. I can try to build a file with a wall, a window, multi-materials for the window and for the wall, and all that, but I'm not sure if it's the same things you are testing with your files.
I need to compile Cycles first before trying. But I think closing the file descriptor should work the same in Linux. Probably in Linux Python closes the descriptor automatically when the function returns.https://github.com/justnope/FreeCAD-render/tree/cycles
This is the render workbench. Currently it only contains fixes for windows (the os) and a fix for using transparency in the cycles standalone of blender.
Just an advice when you do these types of changes.
Code: Select all
- def writeObject(viewobj,mesh,color,alpha):
+ def writeObject(name,mesh,material):
The best course of action is to just add an element to the function, in order to keep compatibility with older code. For example,
Code: Select all
- def writeObject(viewobj,mesh,color,alpha):
+ def writeObject(viewobj,mesh,color,material=None):
This doesn't mean that you shouldn't rework the code, it just means that you should try to keep it compatible. In Python this is easy to do as you can create small functions that encapsulate other functions, and you can add default values to variables. In this way you can prepare the function for deprecation, if it were to happen at some point in the future.
Code: Select all
def writeObject(viewobj, mesh, color=None, alpha=None, material=None)
if material:
writeObject_new(name=viewobj, mesh=mesh, material=None)
else:
writeObject_old(viewobj=viewobj, mesh=mesh, color=color, alpha=alpha)
Now, this is all without actually testing the code. The code looks fine. It's just that if you change the parameters of a function, you will be introducing some incompatibilities with older code. Even if this isn't very serious in this case, it's better to be vigilant of such problems.
I don't mean a direct import like that, but rather that you can export an OBJ (or another format) and it produces a file that is ready to render by Blender with the hit of a button. The least amount of manual setup would be nice, say, 3 steps, instead of 5 steps. I think my idea was not exporting an OBJ, but rather exporting an actual .blend file from FreeCAD; that would mean already setting up the geometry, and some options like lightning and materials.It might not be what you wanted because it's not a direct import into the gui of blender....
As I said, check Yorik's plugin. His plugin does exactly this. It is launched from Blender, to open a FreeCAD file directly.
FreeCAD .FCStd importer for Blender 2.80
Always add the important information to your posts if you need help. Also see Tutorials and Video tutorials.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
Re: Adding refraction index
I'm glad you brought this up because it forced me to reread the standard. I'm using version 4.2 and couldn't find anything more recent online and offline. The change from transparency "Tr" to dissolve "d" is correct. Transparency "Tr" isn't in the standard but dissolve is. From what I gathered is that Tr = 1.0 - d. I think "Tr" is an extension introduced by some programs. However I might do a better job in selecting the illumination model "illum".vocx wrote: ↑Wed Sep 04, 2019 12:49 am Why is this? Is it because Blender cannot use the Transparency value but it can use illum? I don't know if the OBJ file format is standardized somewhere, but I think you should better ask Yorik where he got the values that he uses.
As I read in Wikipedia, there is a basic, standard OBJ format, but there are other versions with certain extensions, supported by different programs.
Feel free to skip the following paragraphs. It's nitpicking because I "technically" follow the standard but not the spirit of the standard and probably only affects real-time engines.
The standard is a bit confusing when they use the term transparent when you don't fully read the standard. They use it when they talk about alpha blending pixels (the dissolve parameter) but they use it differently when talking about the illumination model. In the illumination model they are using it for reflection and refraction.
I think the authors of the blender wavefront importer made the same mistake as I initially did. They enabled alpha blending based on the illumination model while the standard says it should always be enabled when the dissolve parameter is not 1.0. I'll submit a patch to blender and see what they say about this.
That's a good point and I'll keep it in mind before submitting a final version.vocx wrote: ↑Wed Sep 04, 2019 12:49 am You have to assume that somebody in the world has already used the function Cycles.writeObject(viewobj, mesh, color, alpha) before to automate rendering. Therefore, if you change the interface of the functions, most probably you will be breaking scripts that were written for a specific version of FreeCAD or the Render workbench.
The current version of the interface isn't set in stone and could change in the next commits to my repo. One change I'm thinking about but am a bit reluctant to do, is to split writeObject into writeMesh and writeMaterial. I've also removed the viewobj from the function call but maybe it's
still useful and I have to re-add it.
I think I might be overambitious. I was hoping to eventually use zero steps.vocx wrote: ↑Wed Sep 04, 2019 12:49 am The least amount of manual setup would be nice, say, 3 steps, instead of 5 steps. I think my idea was not exporting an OBJ, but rather exporting an actual .blend file from FreeCAD; that would mean already setting up the geometry, and some options like lightning and materials.
For example, you have freecad and blender open at the same time. In blender you have a read-only model of what you created in freecad. You make some edits in freecad and then save it. Blender automagically reloads the changes and you can continu working in blender with the new model. The lights, cameras and other things you added are read-write and are unaffected when it reloads.
I've worked in such an environment so it's possible. I could edit a 3d model and when I saved it, the other program detected the file changes and reloaded it.
I'm not saying I could achieve this.
In the end I'm hoping to get to that. I think if I start with that I'll be overwhelmed with the details of the internals of both blender and freecad.
btw, thanks for the feedback. I really appreciate it.
Re: Adding refraction index
Ok. Thanks for the explanation.Nocturnial wrote: ↑Wed Sep 04, 2019 6:36 pm I'm glad you brought this up because it forced me to reread the standard. I'm using version 4.2 and couldn't find anything more recent online and offline. The change from transparency "Tr" to dissolve "d" is correct. Transparency "Tr" isn't in the standard but dissolve is. From what I gathered is that Tr = 1.0 - d. I think "Tr" is an extension introduced by some programs. However I might do a better job in selecting the illumination model "illum".
Ok. Talk to Yorik, maybe that's why the Transparency value was there, to be compatible with Blender.I think the authors of the blender wavefront importer made the same mistake as I initially did. They enabled alpha blending based on the illumination model while the standard says it should always be enabled when the dissolve parameter is not 1.0. I'll submit a patch to blender and see what they say about this.
That sounds very nice. I was just saying that if it cannot be completely automated, maybe at least the number of manual steps can be reduced. Obviously having seamless integration would be ideal. Good work.I think I might be overambitious. I was hoping to eventually use zero steps.
...
Always add the important information to your posts if you need help. Also see Tutorials and Video tutorials.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.
To support the documentation effort, and code development, your donation is appreciated: liberapay.com/FreeCAD.