Obtain dense depth images with the rc_randomdot projector¶
This tutorial shows how to set up the rc_visard with the rc_randomdot projector to increase the density of depth images, as shown in Fig. 12. This is particularly recommended for objects and workspaces with little or no visual texture.
Fig. 12 A scene including objects without visual texture (i.e. the intensity of the image is uniform over large parts of the scene) and the computed depth image without the rc_randomdot projector (center) and with the rc_randomdot projector (right).
Note
This tutorial assumes that the previous tutorial Tuning of image parameters has been completed.
Before we start¶
Before going through this tutorial, the following prerequisites should be met.
- The rc_visard is running the latest firmware (version 24.01) and the rc_visard’s license includes the IOControl module. This can be verified on the page of the Web GUI.
- The rc_randomdot projector, the network cable connecting it to the rc_visard, and a 24 V, 60 W power supply are available.
Warning
Please ensure that a 24 V, 60 W power supply is used for powering the rc_visard and the rc_randomdot projector together.
Note
The rc_randomdot projector is controlled via the GPIO Out 1 of the rc_visard (see IO and Projector Control). In this tutorial, Out 1 mode refers to the rc_randomdot projector mode.
Mounting the projector¶
Mounting the projector is described in detail in the projector’s manual. Please see Installation.
Note
For projectors sold until 12/2020, the mounting instructions are available on the Roboception website.
Focusing the projector¶
For the focus configuration, one should turn the rc_randomdot projector permanently on. This can be done on the page of the Web GUI by setting the Out 1 mode to High.
Before focusing the projector, one needs to remove the lens protection tube temporarily. Please unscrew the lens protection tube patently, until the rings of the lens are visible. Now, the focus can be adjusted by rotating one of the rings of the lens until the random dot pattern is sharp (Fig. 13).
Fig. 13 Blurred (left) and sharp (right) random dot pattern.
Configuring the projector brightness¶
For tuning the projector brightness, one should set the Out 1 mode to ExposureAlternateActive on the page of the Web GUI. Next, on the Camera page, the exposure must be set to Auto and the Auto Exposure Mode to AdaptiveOut1.
The text line under the live images on the Camera page now shows the Out1 Reduction value in percent. This value describes the reduction of brightness of the image without projection as compared to the image with projection. The value should be between 25% to 50%. If the value is outside this range, the brightness of the projector should be adapted.
For adapting the projector brightness, the aperture ring of the projector lens can be adjusted by rotating it.
After a basic adjustment, the darkest object that is possible in the application should be placed in the field of view of the sensor. If the dark object is not properly visible in the depth image on the Depth Image page of the Web GUI, then the projector brightness must be increased, despite high reduction values.
Selecting projector working mode¶
Single shot mode¶
If the projector is used in combination with rc_reason modules (ItemPick, BoxPick or SilhouetteMatch), it can work in a single shot mode. This means that it will only be activated on demand when the module requests a new depth image.
This mode is enabled by setting the Acquisition Mode to Single + Out1 in the Depth Image page of the Web GUI.
Continuous mode¶
If the projector has to be used in continuous mode (stereo matching is performed continuously without a trigger), the proper Out 1 mode in the Web GUI’s page has to be set.
By default, the Out 1 mode of the rc_visard is set to Low, i.e. projector is off.
Typically, the user needs to select the ExposureAlternateActive Out 1 mode, in which the rc_randomdot projector is on only for the exposure time of every second image. Images with projected pattern are used for computing depth images. Images without projected pattern can be used as texture or for other image processing modules.
Configuring the exposure¶
As last step, one needs to go back to the Web GUI’s Camera page and set the exposure time, either manually, as explained in the Exposure and Gain Settings section of the Tuning of image parameters tutorial, or automatically using the auto exposure. In this case, the Auto Exposure Mode should be set to AdaptiveOut1 to avoid over exposure due to the projector.
Note
If the exposure is configured to automatic mode, when the projector is turned on, it may take a few seconds until the exposure and gain settings become stable.
More optimization tricks¶
In some cases, the density of the depth image can be further optimized by changing the focus of the rc_randomdot projector to introduce some blur in the projected pattern. This can be done while looking at the Confidence image in the Web GUI’s Depth Image page and trying to obtain as many white pixels as possible.
If the camera image includes highlights that are not reconstructed in the depth image (Fig. 14), one can verify whether these highlights are introduced by the rc_randomdot projector (e.g. by simply turning the projector off). In such cases, we recommend using the Double-Shot mode that is available on the Depth Image page.
Alternatively, the mounting of the rc_randomdot projector could be changed in order to have a different angle between the rc_visard and the projector itself. Reducing the aperture of the projector can also help in removing highlights.
Fig. 14 Highlight in the camera image due to the rc_randomdot projector.