CfdOF and OpenFoam server ?

A subforum specific to the development of the OpenFoam-based workbenches ( Cfd https://github.com/qingfengxia/Cfd and CfdOF https://github.com/jaheyns/CfdOF )

Moderator: oliveroxtoby

techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

Cobras62 wrote: Wed Feb 08, 2023 8:07 am Hi @techGuy !
Hi !
Thanks for the pics, I had seen them on the github (https://github.com/jaheyns/CfdOF/issues/70) ;)

You're really doing a great job! I'll be able to test this out and give you feedback when I get a chance to try it :)
I'd like to get it in the hands of people to test before I submit it to CfdOF formally. I'll reach out when it is ready.

Is your "supercomputer" a single computer or a cluster ? Does it run Linux ? Do you have admin privileges on it ?
It is a cluster: https://www-calculco.univ-littoral.fr/r ... quipements
I've never worked with a cluster. My enhancement assumes the remote computer is a single computer running Linux.

My software communicates with the remote computer using ssh. How does are the cases (mesh or OF) generated differently for a cluster versus a single computer ? Are you using MPI ? Does that hide the details of the cluster ?
Attention, I am only discovering CfdOF and OpenFoam, and for the cluster,
So be it. We all start somewhere.
I am going to work on it with a colleague engineer specialized on numerical calculation and who knows very well the cluster.

I'm having a meeting with him this afternoon to talk about it, I'm going to present him CfdOF, and your work :)
Could you get me an account on your system for testing and access to your friend ? I'm not using a cluster right now, but it would be pretty interesting thing to do. And someday I may want to use a cluster.

Thanks for the interest.
User avatar
oliveroxtoby
Posts: 812
Joined: Fri Dec 23, 2016 9:43 am
Location: South Africa

Re: CfdOF and OpenFoam server ?

Post by oliveroxtoby »

techGuy wrote: Wed Feb 08, 2023 7:38 pm My software communicates with the remote computer using ssh. How does are the cases (mesh or OF) generated differently for a cluster versus a single computer ? Are you using MPI ? Does that hide the details of the cluster ?
MPI does pretty much hide the details of the cluster, but you generally need to start the program by submitting the job to a scheduler rather than executing it directly.
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

adrianinsaval wrote: Wed Feb 08, 2023 1:50 pm a couple suggestions, instead of separate buttons for remote or local operations on the task view consider using a checkbox or a combobox to switch between local or remote run so you don't have to have duplicate buttons
I thought of doing that. However, it is just so handy to have the option to run something locally or run it remotely and back and forth. "Oh, the server is busy, so I'll run this mesh locally." "Oh, the server is free, I'll run this on the server." And so on. The current layout supports rapidly changing between running things locally and remotely.

In the bigger picture there will be multiple remote hosts. And the UI will display the remote host CPU usage so that the user can select a remote computer without having to run an ssh terminal to see what host is free. And the user will select which remote host they want to use when running remotely.

At one point I was going to enable one FreeCAD instance to be running multiple processes (meshing, OF, etc) But that didn't make sense because CFD is a linear process: model-> meshCase -> mesh -> OFCase -> Run OF. So the only way running multiple processes makes sense is if you have multiple models (files) going in the same instance. But a better way to handle that use case is to have multiple instances of FC open at the same time. Or open multiple models in one FC instance.

My envisioned use case for CfdOF is one or more users running one or more instances of FC accessing a group of remote hosts to do the processing. Running 2 different cases on 2 different servers at the same time is faster than running 1 case at a time on 2 clustered servers.

Most people doing CFD work are doing optimizations that involve A/B comparisons, thus running multiple cases. Ideally the remote computers would have schedulers on them so that users could park a process on a server and get notified when it is complete. But users will have to manually schedule things for now.

In future releases there will be a set of buttons for the local computer, just because that is the common use case and then another set of buttons for the remote host as there are now, but with a drop down to select which remote host the user wants to use.

I personally have 2 remote hosts right now. A Ryzen machine (david) that is great for meshing and an Epyc machine (goliath) that is great for running OF. I can start a mesh locally just to see that I have it set up properly. Then I can stop it. Then I can run it on david. Then I can run the OF on goliath. Then I can jump to the next case (A/B, remember ?) check that it runs locally. When david is done I can write the mesh case to it and mesh it there. If goliath isn't done with case A, I can run case A OF on david. Or if goliath gets done, I can run it on goliath. Or I could run it locally. All the options are right there on the UI.

I used to do this all manually ! Constantly moving files, opening Parview, etc. It was exhausting. The new functionality in OF will make this sort of work much, much easier. One user, 3 computers - a workstation for editing and viewing, a meshing computer for running meshes and an OF computer or two for running OF stuff. Extreme productivity.
(btw your screenshot has a typo, it says remote sover instead of solver), also is it worth running checkMesh on the remote computer or will this always run locally?
All the meshes are copied back to the local machine after being run remotely. Checking meshes is a pretty fast process and the edits/fixes will be done locally so it makes sense (to me) to check them locally. I guess they could be checked remotely. I'll see how it plays out in real world use going forward.
In the remote preference page, consider adding the ability of having remote profiles, allowing to easily switch between different remote computers, this is of course low priority.
I don't think that is low priority at all. But I'm getting the simplest case running first.

There are 2 more use cases that I should mention.

1) CFD works often involves looking at more than one data point. With a wing, for example, people want to see what lift and drag look like over a range of angles of attack. Each one of those data points is a CFD run, which takes a lot of time. Ditto for different flow rates, Reynolds Numbers, etc.

However, as we know, FC is parametric, based on Python and supports macros. So we can easily write a macro to change the AoA, mesh and run OF for 20 different values. Start the macro at 2 PM, walk away and look at the results next morning. I do not know of any other CFD program that can do this. I'm saying AoA here, but we could be talking about all sorts of other parametric changes. AoA is just such an easy one to point out.

2) AI is increasingly used to optimize designs. There are research papers written about giving AI control over various design parameters such as chord, wingspan, etc. and feeding the AI engine simulation results to learn from. With FC/CfdOF this entirely doable. However, it takes a large number of simulations to feed the AI engine enough data. But with a farm of remote servers running simulations, it is not only possible to do, but also relatively easy.
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

oliveroxtoby wrote: Wed Feb 08, 2023 8:18 pm
techGuy wrote: Wed Feb 08, 2023 7:38 pm My software communicates with the remote computer using ssh. How does are the cases (mesh or OF) generated differently for a cluster versus a single computer ? Are you using MPI ? Does that hide the details of the cluster ?
MPI does pretty much hide the details of the cluster, but you generally need to start the program by submitting the job to a scheduler rather than executing it directly.
As long as it can be done manually via ssh, I will have no problem with that. But right now the CfdOF UI is designed to remain synchronous with the running of the CfdOF processes. Ie the UI waits for the process to be done. If you submit it to a scheduler, it isn't going to run immediately and you won't be able to watch the run results.

At worst, you could submit it to the scheduler via a macro. Processes run via a macro aren't interactive. Or maybe I could add a "Send case to cluster" button.

In any event there are lots of possibilities here.
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

techGuy wrote: Wed Feb 08, 2023 8:37 pm
oliveroxtoby wrote: Wed Feb 08, 2023 8:18 pm
techGuy wrote: Wed Feb 08, 2023 7:38 pm My software communicates with the remote computer using ssh. How does are the cases (mesh or OF) generated differently for a cluster versus a single computer ? Are you using MPI ? Does that hide the details of the cluster ?
MPI does pretty much hide the details of the cluster, but you generally need to start the program by submitting the job to a scheduler rather than executing it directly.
As long as it can be done manually via ssh, I will have no problem doing it in CfdOF.

But right now the CfdOF UI is designed to remain synchronous with the running of the CfdOF processes. Ie the UI waits for the process to be done. If you submit it to a scheduler, it isn't going to run immediately and you won't be able to watch the run results.

At worst, you could submit it to the scheduler via a macro. Processes run via a macro aren't interactive. Or maybe I could add a "Send case to cluster" button.

In any event there are lots of possibilities here.
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

techGuy wrote: Wed Feb 08, 2023 8:33 pm
adrianinsaval wrote: Wed Feb 08, 2023 1:50 pm a couple suggestions, instead of separate buttons for remote or local operations on the task view consider using a checkbox or a combobox to switch between local or remote run so you don't have to have duplicate buttons
I thought of doing that. However, it is just so handy to have the option to run something locally or run it remotely and back and forth. "Oh, the server is busy, so I'll run this mesh locally." "Oh, the server is free, I'll run this on the server." And so on. The current layout supports rapidly changing between running things locally and remotely.

In the bigger picture there will be multiple remote hosts. And the UI will display the remote host CPU usage so that the user can select a remote computer without having to run an ssh terminal to see what host is free. And the user will select which remote host they want to use when running remotely.

At one point I was going to enable one FreeCAD instance to be running multiple processes (meshing, OF, etc) But that didn't make sense because CFD is a linear process: model-> meshCase -> mesh -> OFCase -> Run OF. So the only way running multiple processes makes sense is if you have multiple models (files) going in the same instance. But a better way to handle that use case is to have multiple instances of FC open at the same time. Or open multiple models in one FC instance.

My envisioned use case for CfdOF is one or more users running one or more instances of FC accessing a group of remote hosts to do the processing. Running 2 different cases on 2 different servers at the same time is faster than running 1 case at a time on 2 clustered servers.

Most people doing CFD work are doing optimizations that involve A/B comparisons, thus running multiple cases. Ideally the remote computers would have schedulers on them so that users could park a process on a server and get notified when it is complete. But users will have to manually schedule things for now.

In future releases there will be a set of buttons for the local computer, just because that is the common use case and then another set of buttons for the remote host as there are now, but with a drop down to select which remote host the user wants to use.

I personally have 2 remote hosts right now. A Ryzen machine (david) that is great for meshing and an Epyc machine (goliath) that is great for running OF. I can start a mesh locally just to see that I have it set up properly. Then I can stop it. Then I can run it on david. Then I can run the OF on goliath. Then I can jump to the next case (A/B, remember ?) check that it runs locally. When david is done I can write the mesh case to it and mesh it there. If goliath isn't done with case A, I can run case A OF on david. Or if goliath gets done, I can run it on goliath. Or I could run it locally. All the options are right there on the UI.

I used to do this all manually ! Constantly moving files, opening Paraview, etc. It was exhausting. The new functionality in OF will make this sort of work much, much easier. One user, 3 computers - a workstation for editing and viewing, a meshing computer for running meshes and an OF computer or two for running OF stuff. Extreme productivity.
(btw your screenshot has a typo, it says remote sover instead of solver), also is it worth running checkMesh on the remote computer or will this always run locally?
All the meshes are copied back to the local machine after being run remotely. Checking meshes is a pretty fast process and the edits/fixes will be done locally so it makes sense (to me) to check them locally. I guess they could be checked remotely. I'll see how it plays out in real world use going forward.
In the remote preference page, consider adding the ability of having remote profiles, allowing to easily switch between different remote computers, this is of course low priority.
I don't think that is low priority at all. But I'm getting the simplest case running first.

There are 2 more use cases that I should mention.

1) CFD works often involves looking at more than one data point. With a wing, for example, people want to see what lift and drag look like over a range of angles of attack. Each one of those data points is a CFD run, which takes a lot of time. Ditto for different flow rates, Reynolds Numbers, etc.

However, as we know, FC is parametric, based on Python and supports macros. So we can easily write a macro to change the AoA, mesh and run OF for 20 different values. Start the macro at 2 PM, walk away and look at the results next morning. I do not know of any other CFD program that can do this. I'm saying AoA here, but we could be talking about all sorts of other parametric changes. AoA is just such an easy one to point out.

2) AI is increasingly used to optimize designs. There are research papers written about giving AI control over various design parameters such as chord, wingspan, etc. and feeding the AI engine simulation results to learn from. With FC/CfdOF this entirely doable. However, it takes a large number of simulations to feed the AI engine enough data. But with a farm of remote servers running simulations, it is not only possible to do, but also relatively easy.
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

Mods, please clean up my mess of empty posts. For some reason a new post is being made when I edit an existing post. Sorry.
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: CfdOF and OpenFoam server ?

Post by adrianinsaval »

techGuy wrote: Wed Feb 08, 2023 8:33 pm
adrianinsaval wrote: Wed Feb 08, 2023 1:50 pm a couple suggestions, instead of separate buttons for remote or local operations on the task view consider using a checkbox or a combobox to switch between local or remote run so you don't have to have duplicate buttons
I thought of doing that. However, it is just so handy to have the option to run something locally or run it remotely and back and forth. "Oh, the server is busy, so I'll run this mesh locally." "Oh, the server is free, I'll run this on the server." And so on. The current layout supports rapidly changing between running things locally and remotely.
My proposal does not change this, the checkbox/combobox would be in the task panel, not the preference page, I think this is a more logical way of switching back and forth than having duplicate controls.
In future releases there will be a set of buttons for the local computer, just because that is the common use case and then another set of buttons for the remote host as there are now, but with a drop down to select which remote host the user wants to use.
To me it seems more logical to have just one set of button then the dropdown (this is what I called a combobox) let's you choose between local, remote1, remote2 and so on.

Either way, thank you for your efforts! This is a very cool feature
1) CFD works often involves looking at more than one data point. With a wing, for example, people want to see what lift and drag look like over a range of angles of attack. Each one of those data points is a CFD run, which takes a lot of time. Ditto for different flow rates, Reynolds Numbers, etc.

However, as we know, FC is parametric, based on Python and supports macros. So we can easily write a macro to change the AoA, mesh and run OF for 20 different values. Start the macro at 2 PM, walk away and look at the results next morning. I do not know of any other CFD program that can do this. I'm saying AoA here, but we could be talking about all sorts of other parametric changes. AoA is just such an easy one to point out.
Indeed, and as matter of fact I had done exactly that and ended up submitting some fixes/tweaks to the workbench that allowed to run my scripts better, even when something doesn't work out of the box you can make it work because it's FOSS! you can't find this kind of flexibility anywhere else, extremely good stuff.
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

Multiple Remote Hosts Preferences.png
Multiple Remote Hosts Preferences.png (142.6 KiB) Viewed 823 times
techGuy
Posts: 126
Joined: Sat Jun 18, 2022 12:48 am

Re: CfdOF and OpenFoam server ?

Post by techGuy »

local host meshing.png
local host meshing.png (59.75 KiB) Viewed 803 times
Post Reply