Two new scripting methods

Need help, or want to share a macro? Post here!
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
jbi
Posts: 116
Joined: Sun Apr 24, 2016 3:28 pm

Re: Two new scripting methods

Post by jbi »

Here is an example of the structure for multiprocessing, so you can study the idea - sorry only python:
Breaking out into a subprocess from the GUI is possible - and from this subprocess you can go into multiprocessing (I have tested and verified);
I have used an external python interpreter which get started by the subrocess, maybe it is also possible to use the python interpreter delivered with FreeCAD:

Code: Select all

	

import subprocess
import os
start=time.time()
f=open(globalbasepath+args.cdatafile,'rb')
x=f.read()
f.close()
c_data=pickle.loads(x)

####################################################

env_o3d='/home/jb/miniconda3/envs/py369_open3d/bin/python' #path to the external python interpreter
reconst_script='/home/jb/pgr3/clx.py' #this script gets started
args='aaa' #could be a filename

p=subprocess.Popen([env_o3d, reconst_script, args])
while p.poll() is None:
	time.sleep(0.1)

####################################################

end=time.time()
print("overall reconstructprocess:  "+ str(round(end-start,2)) + " [sec]"+'\n')		


in this called script you can start with multiprocessing - I recommend working with data and the result as files (this could be numpy arrays in your case), since no slow interprocess communication will be needed:

Code: Select all


#make this your script clx.py

import numpy as np
import multiprocessing
from multiprocessing import Process, Pipe
import os
import pickle
import sys
import shutil
sys.path.append("/home/jb/pgr3")

#############################################################
#############################################################


def main(argv):
	print('argv: ',argv)
	#f=open(argv[0],'rb')
	#x=f.read()
	#f.close()
	#c_data=pickle.loads(x)

	nbcpu=multiprocessing.cpu_count()-0

	conns=[]
	processes=[]
	s=[]

	for i in range(nbcpu):
		parent_conn, child_conn = Pipe()
		conns.append([parent_conn, child_conn])
		control=['slice',False,False]
		p = Process(target=workercadslice, args=(some_args,)) #for time profiling
		processes.append(p)

	for j,p in enumerate(processes):
		p.start()
		print(conns[j][0].send(control))   # prints "[42, None, 'hello']"
		print('start: ',j)

	for j,p in enumerate(processes):
		p.join()

	return None


#############################################################
#############################################################

def workercadslice(args): #for time profiling
	import time
	time.sleep(10)
	#do your work here

#############################################################
#############################################################

if __name__ == '__main__':
    main(sys.argv[1:])
Edit: Verified, that the idea actually works, rework the code structure
User avatar
xxxajk
Posts: 10
Joined: Sat Jan 04, 2020 5:30 am

Re: Two new scripting methods

Post by xxxajk »

I've actually fixed this on the C++ side, No need for the multiprocessing module. Just had to unlock and lock the stupid GIL.
I'll be using that for now, eventually submitting a PR, after I test it for a few more days.
It's wonderful to be able to launch 400 threads. Completion happens now in just a couple of seconds for large lists, not minutes or hours.
Post Reply