I have trained two models (forward and backward).
(The input to the models are images of type uint8, so I am dividing by 255)
After predicting on each model, I receive two arrays:
forward = np.load('f.npy')
backward = np.load('b.npy')
I also must use an image tiles_M
in order to follow these equations:
p1 = ( 1.0 / abs(forward - tiles_M/255.) ) / ( (1.0 / abs(forward - tiles_M/255.)) + (1.0 / abs(backward - tiles_M/255.)) )
p3 = ( 1.0 / abs(backward - tiles_M/255.) ) / ( (1.0 / abs(forward - tiles_M/255.)) + (1.0 / abs(backward - tiles_M/255.)) )
Note, that, I divide tiles_M
by 255 (the same I did in inputs for training the models) since it is an uint8 image.
Then, the prediction must use this equation:
pred = p1 * forward + p3 * backward
The problem, is when I try to reconstruct the image, I receive a black image (all zero values).
If I normalize pred
: pred = normalize_arr(pred)
I receive this image
I have tried various ways to normalize either pred
or p1
, p2
, forward
, backward
but now works as expected.
Now the interesting part comes from this.
If I use this equation (which is wrong and I accidentally typed at some point!):
p1 = ( 1.0 / abs(forward ) ) / ( (1.0 / abs(forward - tiles_M)) + (1.0 / abs(backward - tiles_M)) )
p3 = ( 1.0 / abs(backward) ) / ( (1.0 / abs(forward - tiles_M)) + (1.0 / abs(backward - tiles_M)) )
so, no tiles_M
scaling and no subtraction from tiles_M
in the numerator, I receive this correct image!!!
The equation is:
You can find the data here.
This is the code:
import numpy as np
import cv2
from PIL import Image
def normalize_arr(arr):
the_min = arr.min()
the_max = arr.max()
the_max -= the_min
arr = ((arr - the_min)/the_max) * 255.
return arr.astype(np.uint8)
def extract_tiles(size, im):
im = im[:, :, :3]
w = h = size
idxs = [(i, (i + h), j, (j + w)) for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]
tiles_asarrays = []
count = 0
for k, (i_start, i_end, j_start, j_end) in enumerate(idxs):
tile = im[i_start:i_end, j_start:j_end, ...]
if tile.shape[:2] != (h, w):
tile_ = tile
tile_size = (h, w) if tile.ndim == 2 else (h, w, tile.shape[2])
tile = np.zeros(tile_size, dtype=tile.dtype)
tile[:tile_.shape[0], :tile_.shape[1], ...] = tile_
count += 1
tiles_asarrays.append(tile)
return np.array(idxs), np.array(tiles_asarrays)
IMG_WIDTH = 32
# Load arrays
forward = np.load('f.npy')
backward = np.load('b.npy')
tiles_M = np.load('tiles_M.npy')
# Weighting params
p1 = ( 1.0 / abs(forward - tiles_M/255.) ) / ( (1.0 / abs(forward - tiles_M/255.)) + (1.0 / abs(backward - tiles_M/255.)) )
p3 = ( 1.0 / abs(backward - tiles_M/255.) ) / ( (1.0 / abs(forward - tiles_M/255.)) + (1.0 / abs(backward - tiles_M/255.)) )
# works but wrong equation and no tiles_M scaling
# p1 = ( 1.0 / abs(forward ) ) / ( (1.0 / abs(forward - tiles_M)) + (1.0 / abs(backward - tiles_M)) )
# p3 = ( 1.0 / abs(backward) ) / ( (1.0 / abs(forward - tiles_M)) + (1.0 / abs(backward - tiles_M)) )
pred = p1 * forward + p3 * backward
#pred = normalize_arr(pred)
# Load original image
img = cv2.imread('E2.tif',
cv2.IMREAD_UNCHANGED)
img = cv2.resize(img, (1408, 1408), interpolation=cv2.INTER_AREA)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# create tiles
idxs, tiles = extract_tiles(IMG_WIDTH, img)
# Initialize reconstructed array
reconstructed = np.zeros((img.shape[0],
img.shape[1],
img.shape[2]),
dtype=np.uint8)
# reconstruct
for tile, (y_start, y_end, x_start, x_end) in zip(pred, idxs):
y_end = min(y_end, img.shape[0])
x_end = min(x_end, img.shape[1])
reconstructed[y_start:y_end, x_start:x_end] = tile[:(y_end - y_start), :(x_end - x_start)]
# create image from array
im = Image.fromarray(reconstructed)
im = im.resize((1429, 1416))
im.show()
from reconstruction of image shows either black either square borders
No comments:
Post a Comment