In July 2020 we posted about the completion of our benchmark targets for the evaluation of 3D sensors. While we already summarized the results in our internal deliverable D4.1, which was submitted in January 2021, we are now happy to present our results at 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) in Prague, Czech Republic
The presented paper compares five industrial grade and three consumer grade depth cameras from price ranges between a few hundred up to multiple thousands of Euros. REMODEL partner TUM further introduced two new metrics for sensor quality, particularity of interest for the REMODEL project: While common metrics, as the sensor’s z-precision are commonly measured by scanning a plane, and thus can be be easily tricked by applying integrated de-noising filter, the Radius Reconstruction Accuracy (RRA) gives an impression about a sensor’s capacity to accurately reconstruct fine structures. The idea is to scan multiple cylindrical objects, such as pipes or wires and compare the standard deviation of the reconstructed radii to a known ground truth. On top, our second metric estimates the point cloud’s Continuity. It again requires the scanning of cylindrical objects with varying diameters, but computes the completeness of the scan over several batches along the object.
The above plots show the measured RRA (top) and Continuity (bottom) on different surfaces. In case the Continuity dropped below 30% we did not compute the standard deviation of the radii anymore, since the was to little data for a meaningful result.
For more information please join our presentation at IROS 2021 . The full text publication is available at mediaTUM.
Written by TUM