Page 1 of 2
External full Python WB Unit Tests
Posted: Thu Aug 23, 2018 3:46 pm
by gflorent
I have some questions, for the more experienced folks with FreeCAD, regarding the preferred unit testing strategy for external workbenches.
1. Is any code organisation and testing framework ok (as long as it works) or should an organisation and framework (unittest I guess) be preferred for potential integration with the Test WB?
2. Is it possible to add a test suite for an external WB to the Test WB (without editing the Python code in the Test WB of the installed FreeCAD) or should the usual test runners from a testing framework be used?
3. Should the TemplatePyMod way of writing tests be considered as the example to follow to write tests (since the FreeCAD/Workbench-Starterkit repo does not seem to mention tests)?
Maybe this is all explained somewhere but I could not find the info.
Thanks in advance to anyone who may help with that.
Re: External full Python WB Unit Tests
Posted: Thu Aug 23, 2018 5:06 pm
by DeepSOIC
I hope to be proven wrong... but it looks like, you can't. The test modules are explicitly hard-coded here:
https://github.com/FreeCAD/FreeCAD/blob ... App.py#L56
Code: Select all
def All():
# Base system tests
tests = [ "UnicodeTests",
"Document",
"UnitTests",
"BaseTests" ]
# Base system gui test
if (FreeCAD.GuiUp == 1):
tests += [ "Workbench",
"Menu" ]
# add the module tests
tests += [ "TestFem",
"MeshTestsApp",
"TestSketcherApp",
"TestPartApp",
"TestPartDesignApp",
"TestSpreadsheet",
"TestTechDrawApp",
"TestPathApp" ]
# gui tests of modules
if (FreeCAD.GuiUp == 1):
tests += [ "TestSketcherGui",
"TestPartGui",
"TestPartDesignGui",
"TestDraft",
"TestArch" ]
suite = unittest.TestSuite()
for test in tests:
suite.addTest(tryLoadingTest(test))
return suite
I guess, you can hack in by overwriting the function All() to inject your own tests, but that doesn't sound great.
EDIT: even worse, it's hard-coded yet again here:
https://github.com/FreeCAD/FreeCAD/blob ... Gui.py#L43
Re: External full Python WB Unit Tests
Posted: Mon Aug 27, 2018 4:09 pm
by gflorent
There is a nice solution in the InitGui.py of
https://github.com/microelly2/freecad-nurbs near line 140. The same dialog as the one used in the 'official' Test WB appears (when clicking a 'Test' command) with a drop-down list of unit tests scripts.
Here is the snippet extracted from InitGui.py
Code: Select all
class MyTestCmd2:
"""Opens a Qt dialog with all inserted unit tests"""
def Activated(self):
import QtUnitGui
QtUnitGui.addTest("nurbswb.TestNurbsGui")
QtUnitGui.addTest("nurbswb.TestNurbs")
QtUnitGui.addTest("nurbswb.TestMeinAll.Col1")
QtUnitGui.addTest("nurbswb.TestMeinAll.Col2")
QtUnitGui.addTest("TestMeinAll.Col2")
def GetResources(self):
return {'MenuText': 'Test-test...',
'ToolTip': 'Runs the self-test for the workbench'}
FreeCADGui.addCommand('My_Test2', MyTestCmd2())
But, unfortunately, I could not find the actual tests scripts in microelly2's repo because of the following line in the .gitignore
@microelly2: I like your testing solution in the Nurbs WB, and I am sure many others will, would you mind sharing the test files?
Re: External full Python WB Unit Tests
Posted: Mon Aug 27, 2018 4:50 pm
by DeepSOIC
I have a different plan. The plan is to make a specialized tester document object, which is to remember and verify the result of a sequence of features. Then I am to make a test project, where I use each feature, visually verify the correctness, and attach a tester object to remember it. And the test is then to load the project, and force-recompute it.
This will probably work OK for Lattice2, which is very feature-centric. But it won't work on part-o-magic, as it is more of a gui/interactivity workbench.
Re: External full Python WB Unit Tests
Posted: Mon Aug 27, 2018 7:02 pm
by gflorent
Maybe we can combine both. The tests ran from the test runner could either check automatically that placement and other properties are as expected or display a dialog where you can click OK or NOT OK based on what you see and controlling the test result. What do you think?
Re: External full Python WB Unit Tests
Posted: Mon Aug 27, 2018 7:05 pm
by DeepSOIC
gflorent wrote: ↑Mon Aug 27, 2018 7:02 pm
display a dialog where you can click OK or NOT OK based on what you see and controlling the test result.
If I see it's NOT OK, there's no point clicking the button. I can go straight ahead and debug/fix it.
Re: External full Python WB Unit Tests
Posted: Mon Aug 27, 2018 9:20 pm
by sgrogan
gflorent wrote: ↑Mon Aug 27, 2018 7:02 pm
Maybe we can combine both. The tests ran from the test runner could either check automatically that placement and other properties are as expected or display a dialog where you can click OK or NOT OK based on what you see and controlling the test result. What do you think?
FreeCAD must run without the gui. So these type of tests should only run when the gui is up.
Re: External full Python WB Unit Tests
Posted: Tue Aug 28, 2018 9:43 am
by wmayer
1. Is any code organisation and testing framework ok (as long as it works) or should an organisation and framework (unittest I guess) be preferred for potential integration with the Test WB?
If everything works with your own testing framework then I think it's fine, too.
3. Should the TemplatePyMod way of writing tests be considered as the example to follow to write tests (since the FreeCAD/Workbench-Starterkit repo does not seem to mention tests)?
The file Tests.py shows how to basically write a unit test, i.e. inherit from unittest.TestCase, offer a setUp and tearDown method and offer methods starting with "test"
2. Is it possible to add a test suite for an external WB to the Test WB (without editing the Python code in the Test WB of the installed FreeCAD) or should the usual test runners from a testing framework be used?
Yes, you can add your tests to the Test WB without changing its source. However, a few things must be polished a bit to improve the integration. Assuming you have a module called MyTest which resides in the Mod directory then you can do the following things:
- Create a file __init__.py
- Create a directory Tests with another __init__.py and e.g. TestCase.py
- The content of Tests/__init__.py can e.g. be:
Code: Select all
import QtUnitGui
QtUnitGui.addTest("MyTest.Tests.TestCase")
- and TestCase.py
Code: Select all
import unittest
class MyTestCases(unittest.TestCase):
def setUp(self):
pass
def testNothing(self):
self.assertTrue(True)
def tearDown(self):
pass
If you now import MyTest.Tests then the tests will be registered and the dialog comes up to run the unit test.
At the moment I haven't found a way to register a test (preferably at startup) without invoking the test dialog. I think this can be done by providing a simple list in the FreeCAD and FreeCADGui modules which stores the module names which will be considered in TestApp.py and TestGui.py. This way you could add tests directly in your Init.py or InitGui.py files.
Re: External full Python WB Unit Tests
Posted: Tue Aug 28, 2018 11:19 am
by wmayer
With
git commit 67b5ee0931 now there are no lists in TestApp.py and TestGui.py but instead each module registers its unit tests via Init.py or InitGui.py. So, with external module it has become much easier to register tests. All what you have to do is adding this line to Init.py or InitGui.py:
Code: Select all
FreeCAD.__unit_test__ += [ "Name_of_my_test_module" ]
Re: External full Python WB Unit Tests
Posted: Tue Aug 28, 2018 12:28 pm
by DeepSOIC
wmayer wrote: ↑Tue Aug 28, 2018 11:19 am
With
git commit 67b5ee0931 now there are no lists in TestApp.py and TestGui.py but instead each module registers its unit tests via Init.py or InitGui.py. So, with external module it has become much easier to register tests. All what you have to do is adding this line to Init.py or InitGui.py:
Code: Select all
FreeCAD.__unit_test__ += [ "Name_of_my_test_module" ]
Oh, wow, thanks!