Friday, 9 September 2022

Convert from RGB to LAB using pytorch? Opencv's color conversion does not match their own formula

In order to speed up some of my past opencv/numpy powered operations, I am attempting to translate them to PyTorch, to run on GPU. One of the first of these steps it RGB to LAB conversion.

I referred to:

I have managed to produce this code:

import torch

def tensor_like(source_data, target_tensor):
    return torch.tensor(
        source_data,
        device=target_tensor.device,
        dtype=target_tensor.dtype,
    )


RGB_TO_XYZ_FACTOR_MATRIX = [
    [0.412453, 0.357580, 0.180423],
    [0.212671, 0.715160, 0.072169],
    [0.019334, 0.119193, 0.950227],
]


def cie_f(input_: torch.Tensor):
    e = 6 / 29
    return torch.where(
        input_ > e ** 3,
        input_ ** (1 / 3),
        input_ / (3 * e ** 2) + 4 / 29,
    )



def tensor_rgb_to_lab(input_: torch.Tensor):
    # input_ is expected to be RGB 0-255, and of shape B, C, H, W
    # Implemented based on formulas written here (may need to scroll down):
    # https://docs.opencv.org/4.x/de/d25/imgproc_color_conversions.html
    dtype = input_.dtype
    if not torch.is_floating_point(input_):
        input_ = input_.float()

    input_ = input_.permute(0, 2, 3, 1)
    rgb_colors = input_.reshape(-1, 3)
    rgb_colors /= 255.0

    # RGB -> XYZ
    factor_matrix = tensor_like(RGB_TO_XYZ_FACTOR_MATRIX, rgb_colors)
    xyz_colors = torch.mm(rgb_colors, factor_matrix.T)
    # xyz_colors.mul_(255)

    # XYZ -> LAB
    xyz_colors *= tensor_like([1 / 0.950456, 1.0, 1 / 1.088754], xyz_colors)
    f_xyz_colors = cie_f(xyz_colors)
    fxyz_to_lab_factor_matrix = tensor_like([
        [0.0, 500.0, 0.0],
        [116.0, -500.0, 200.0],
        [0.0, 0.0, -200.0],
    ], f_xyz_colors)
    lab_colors = torch.mm(f_xyz_colors, fxyz_to_lab_factor_matrix)
    lab_colors += tensor_like([-16, 128, 128], lab_colors)
    lab_colors[:, 0] *= 255 / 100

    lab_colors = lab_colors.view_as(input_).permute(0, 3, 1, 2)
    lab_colors = lab_colors.round_().clamp_(0, 255).type(dtype)
    return lab_colors

When comparing my output against cv2.cvtColor(image, cv2.COLOR_RGB2LAB), my code does produce the correct results with RGB colors like (000, 000, 000), (255, 255, 255), (000, 255, 255), (255, 000, 255), (255, 255, 000), (255, 000, 000), (000, 255, 000), (000, 000, 255), but it fails on many other cases (more than 99% of RGB colors, according to my test).

One such failing case is RGB(128, 128, 128), which open-cv says corresponds to Lab(137, 128, 128), but my code indicates Lab(194, 128, 128). This is an interesting failing case since only 1 value is wrong, and I can manually go through the different steps of the formula for L only in an attempt to isolate the problem:

g = np.array((0.412453+0.357580+0.180423, 0.212671+0.715160+0.072169,0.019334+0.119193+0.950227)) * 128/255
g[0]/=0.950456
g[2]/=1.088754
# g => array([0.50196078, 0.50196078, 0.50196078]). Makes sense since 128/255=0.50196078
l = 116*g[1]**(1/3)-16  # since 0.50196078>0.008856
# l => 76.18945600835188  This is already wrong according to online RGB converters
l *= 255/100
# l => 194.2831128212973  This contradicts opencv's value and correlates with mine

After quadruple-checking my code, I was starting to think opencv might have a bug/rounding error in their code, or that they were doing an extra step differently than they say on their documentation. So I have looked up some online RGB to Lab converters (1 and 2), and after re-adjusting the output L value to the 255 range, they agree with opencv. I guess this really points to a mistake on my part (unless the websites internally rely on opencv?).

Considering how the result comes out correct for RGB values of both 0 and 255 (or 0 and 1 when scaled to 0..1), I think that all of the ax+b operations must be correct, and the t**(1/3) must be the issue... could the industry be somehow using a different exponent, maybe due to performance benefit?

What am I missing here? I am using opencv 4.4.0.46



from Convert from RGB to LAB using pytorch? Opencv's color conversion does not match their own formula

No comments:

Post a Comment