Implement Causal CNN in Keras for multivariate time-series prediction

This question is a followup to my previous question here: Multi-feature causal CNN - Keras implementation, however, there are numerous things that are unclear to me that I think it warrants a new question. The model in question here has been built according to the accepted answer in the post mentioned above.

I am trying to apply a Causal CNN model on multivariate time-series data of 10 sequences with 5 features.

lookback, features = 10, 5

• What should filters and kernel be set to?

  • What is the effect of filters and kernel on the network?
  • Are these just an arbitrary number - i.e. number of neurons in ANN layer?
  • Or will they have an effect on how the net interprets the time-steps?

• What should dilations be set to?

  • Is this just an arbitrary number or does this represent the lookback of the model?

filters = 32
kernel = 5
dilations = 5
dilation_rates = [2 ** i for i in range(dilations)]

model = Sequential()
model.add(InputLayer(input_shape=(lookback, features)))
model.add(Reshape(target_shape=(features, lookback, 1), input_shape=(lookback, features)))

According to the previously mentioned answer, the input needs to be reshaped according to the following logic:

• After Reshape 5 input features are now treated as the temporal layer for the TimeDistributed layer

• When Conv1D is applied to each input feature, it thinks the shape of the layer is (10, 1)

• with the default "channels_last", therefore...

• 10 time-steps is the temporal dimension
• 1 is the "channel", the new location for the feature maps

# Add causal layers
for dilation_rate in dilation_rates:
    model.add(TimeDistributed(Conv1D(filters=filters,
                              kernel_size=kernel,
                              padding='causal',
                              dilation_rate=dilation_rate,
                              activation='elu')))

According to the mentioned answer, the model needs to be reshaped, according to the following logic:

• Stack feature maps on top of each other so each time step can look at all features produced earlier - (10 time steps, 5 features * 32 filters)

Next, causal layers are now applied to the 5 input features dependently.

• Why were they initially applied independently?
• Why are they now applied dependently?

model.add(Reshape(target_shape=(lookback, features * filters)))

next_dilations = 3
dilation_rates = [2 ** i for i in range(next_dilations)]
for dilation_rate in dilation_rates:
    model.add(Conv1D(filters=filters,
                     kernel_size=kernel,
                     padding='causal',
                     dilation_rate=dilation_rate,
                     activation='elu'))
    model.add(MaxPool1D())

model.add(Flatten())
model.add(Dense(units=1, activation='linear'))

model.summary()

SUMMARY

• What should filters and kernel be set to?
  • Will they have an effect on how the net interprets the time-steps?

• What should dilations be set to to represent lookback of 10?

• Why are causal layers initially applied independently?

• Why are they applied dependently after reshape?
  • Why not apply them dependently from the beginning?

===========================================================================

FULL CODE

lookback, features = 10, 5

filters = 32
kernel = 5
dilations = 5
dilation_rates = [2 ** i for i in range(dilations)]

model = Sequential()
model.add(InputLayer(input_shape=(lookback, features)))
model.add(Reshape(target_shape=(features, lookback, 1), input_shape=(lookback, features)))

# Add causal layers
for dilation_rate in dilation_rates:
    model.add(TimeDistributed(Conv1D(filters=filters,
                              kernel_size=kernel,
                              padding='causal',
                              dilation_rate=dilation_rate,
                              activation='elu')))


model.add(Reshape(target_shape=(lookback, features * filters)))

next_dilations = 3
dilation_rates = [2 ** i for i in range(next_dilations)]
for dilation_rate in dilation_rates:
    model.add(Conv1D(filters=filters,
                     kernel_size=kernel,
                     padding='causal',
                     dilation_rate=dilation_rate,
                     activation='elu'))
    model.add(MaxPool1D())

model.add(Flatten())
model.add(Dense(units=1, activation='linear'))

model.summary()

from Implement Causal CNN in Keras for multivariate time-series prediction