Why do I see edges in my point cloud at the border of an object or between dark and bright regions?
A phenomenon called the "Laplacian effect" may occur whenever there is a sudden transition or intensity change in brightness in a target scene. This Laplacian distortion is typical in abrupt transitions between dark and bright or bright and dark points.
You can see an example of this effect in the meshed point cloud image below:
Figure 1 - Laplacian distortion on a black square within a white plane
The distortion either happens on a surface that has a wide dynamic range of brightness or at the edge of an object where the background is much brighter or darker than the object itself. An example could be the edges of a white cardboard box placed on a black table.
Below we're applying a Laplacian filter to a regular 2D image. Notice how the filter highlights edges or transitions in the color 2D image wherever the brightness changes.
Figure 2 - 2D color image with Laplace filter
The Laplacian effect is dependent on the gradient of the change in brightness within a small region of pixels. A sharp or significant gradient produces a more pronounced distortion.
It is also important to note that this effect only occurs along the camera's baseline, the X-axis, and not the Y-axis. See the figure below:
Figure 3 - Laplacian distortion vs. Brightness gradient
Let's look at two similar cylinders, A and B. Cylinder B has a higher level of dark and a bright area along its body compared to cylinder A.
The transition where the light is shining on the cylinder is much narrower, or steeper, for cylinder B than A.
These two effects enhance the Laplacian effect on cylinder B, which gets a taller and steeper distortion than cylinder A.
An example of such a distortion is shown below. The left image shows a very polished metal cylinder which is aligned parallel to the 3D sensor's Y-axis and projector. The strong reflection of light along the top of the cylinder is seen as a white stripe along the length of the cylinder. Projected light that hits the sides are reflected away and thus appear very dark.
Together, the two reflections creating a strong Laplacian effect seen as a "black hole" in the center image.
In the resulting point cloud to the right, the over-exposure along the top of the cylinder creates a "ridge" which distorts and deforms the cylinder from its true, cylindrical shape.
Figure 4 - Laplacian effect on shiny cylinder
What can I do about it?
Maximize the dynamic range of 3D sensor
The Zivid 3D camera has a wide dynamic range, making it possible to take images of both dark and bright objects.
Utilizing its HDR function to cover the entire dynamic range of the scene can help mitigate Laplacian effects a great deal.
The most common issue is highlights in a scene. By using e.g. Zivid Studio, follow these steps:
- Reduce the exposure value so that the entire 2D image is completely dark, or there are no points in the 3D point cloud.
- Increase the exposure value (adjusting iris, exposure time, gain and brightness) until the strongest highlights in the region of interest becomes visible (either as "grey" areas in the 2D image, or as green points in the point cloud). This is your first frame.
- Add another frame.
- Increase the exposure value (adjusting iris, exposure time, gain and brightness) to make more pixels in the region of interest visible, without creating an overexposure region between this frame and the previous.
- Repeat 3. and 4. until no more pixels become available in the region of interest (diminishing return).
Very challenging scenes typically require 3 HDR frames or more. Challenging scenes should typically have:
- 1-2 frames to cover the strongest highlights (very low exposure),
- 1-2 frames to cover most of the scene (medium exposure).
- 1-2 frames to cover the darkest regions (very high exposure).
Rotation and alignment of objects in the scene
The first thing to remember is that this is an effect that occurs in the 3D sensor's X-axis. The Laplacian effect can be greatly mitigated if your application allows for rotating troublesome regions in the camera's Y-axis to its X-axis. By rotating for instance, a shiny cylinder 90°, the overexposed region along the cylinder follows the camera's baseline, as illustrated in the figure below.
Figure 5 - Mitigation of Laplacian effect by object alignment and rotation
For example, if you are doing bin-picking of long, shiny metal objects that tend to orient themselves in a specific direction in the bin, such as pipes in a rectangular bin whose length matches that of the pipes, aligning the camera and the bin so that the pipes are parallel to the camera baseline is optimal.
Matching the background to the brightness of the specific object
A good rule of thumb is to try to use similar brightness or color for the background of the scene as the objects that you're imaging:
- For a bright object, use a bright background (ideally white Lambertian).
- For a dark object, use a dark background (e.g. black rubber as used by most conveyor belts).
- For most colored, non-glossy objects, use a background of similar brightness (e.g. for bananas, use a grey or yellow background).
- For shiny metallic objects, especially cylindrical, conical and spherical objects, use a dark absorptive background such as black rubber. This is because the target light is typically reflected away from the object near its visible edges, making them appear very dark (see image below). At the same time, light from surrounding regions may be reflected to the cylinder edge.
Figure 6 - Light reflected away from edges of shiny cylinders, making them appear dark