After a careful design and crafting phase, the test boards for the planned 3D sensor benchmarking are finally ready. The test frames will allow a qualitative and cuantitative comparison of state-of-the art 3D cameras and ensure the selection of the best sensor for each use-case of the REMODEL project.
It took a lot of manual work to create the five test sets, composed from white plastics, copper, acrylic glass, as well as polished and sanded stainless steel pieces. Each consists of a 50 by 50 centimeter large board and two frames with eleven bars and wires reaching from 0.25 to 18mm diameter. The variety of used materials represents a selection of the most common surfaces in industrial use cases. Moreover the inclusion of transparent and glossy objects will allow to benchmark the sensors under hardest conditions.
Figure 2. Manufacturing of the benchmark frames
The frames will be used to systematically evaluate the performance of 3D depth sensors. REMODEL partner TUM will use them not only to measure their distortion and depth accuracy in relation to the surface orientation and curvature, but also to introduce more use-case -oriented metrics such as the smallest diameter measurable on free-standing-objects or placed on a planar background. The final benchmark results are expected around end of August or early September.
Figure 3. Schematic sketch of a full set of benchmark objects
Written by TUM