Building a VGG Face model in TF: Running into shape mismatch and unsupported shape errors

Question: How do I set up a CNN so that the shapes match? It's my first time working with one since my background is in NLP, and I'm running into the error that says the shapes don't match.

What I've tried:

1. I've tried editing the filters and kernel_size variables.
2. I think kernel_size is correct. I've tried setting the filters variable to tf.constant(kernel).shape[-1] and tf.constant(kernel).shape[-2]. It seems to me that there aren't any other options, so I'm confused, although I think this is where the problem is.

Input weights matrix:

1. http://www.robots.ox.ac.uk/~vgg/software/vgg_face/
2. Download vgg_face_matconvnet.tar.gz and upzip.
3. vgg_face_matconvnet/Data/vgg_face.mat

Code:

# read layer info
model = tf.keras.Sequential()
model.add(tf.keras.Input([224, 224, 3]))
for layer in layers:
    layer_type = layer[0][0][0][0]
    name = layer[0][0][1][0]
    if layer_type == 'conv':
        print(layer_type)
        print(name)
        weights = layer[0][0][2][0]
        stride = layer[0][0][3][0]
        # pad =  layer[0][0][4][0]
        learningRate = layer[0][0][5][0]
        weightDecay = layer[0][0][6][0]
        momentum = layer[0][0][7][0]
        kernel, bias = weights
        # kernel = np.transpose(kernel, (1, 0, 2, 3))
        bias = np.squeeze(bias).reshape(-1)
        filters = tf.constant(kernel).shape[-1]
        kernel_size = (3,3) #[np.shape(kernel)[-3], np.shape(kernel)[-2]]
        bias_initializer = tf.constant_initializer(bias)
        strides=[1, stride[0]]
        if name[:2] == 'fc':
            padding = 'VALID'
            if name == 'fc6':
                model.add(tf.keras.layers.Flatten())
                dense_layer = tf.keras.layers.Dense(filters, kernel_initializer=tf.constant_initializer(kernel))
            if name == 'fc7':
                dense_layer = tf.keras.layers.Dense(filters, kernel_initializer=tf.constant_initializer(kernel))
            model.add(dense_layer)
        else:
            padding = 'SAME'
            conv2d_layer = tf.keras.layers.Conv2D(filters, kernel_size, strides=strides, kernel_initializer=tf.constant_initializer(kernel))
            model.add(conv2d_layer)
        print(f"{name} stride: {stride} kernel size: {np.shape(kernel)}")        
    elif layer_type == 'relu':
        model.add(tf.keras.layers.ReLU(max_value=None, negative_slope=0, threshold=0))
        print(name)
    elif layer_type == 'pool':
        pool = layer[0][0][3][0]
        stride = layer[0][0][4][0]
        model.add(tf.keras.layers.MaxPool2D(pool_size=pool, strides=(stride[0],stride[0]), padding='SAME', data_format="channels_last"))
        print(f"{name} stride: {stride}")
    elif layer_type == 'softmax':
        model.add(tf.keras.layers.Softmax())
        print(name)

Error Message:

TypeError: Eager execution of tf.constant with unsupported shape (value has 102760448 elements, shape is (2048, 4096) with 8388608 elements).

from Building a VGG Face model in TF: Running into shape mismatch and unsupported shape errors