Monday, 21 October 2019

How can I load this data into a LSTM?

I want to classify data with a LSTM. I need a binary classification, thus 2 classes (0,1). The data is split in two folders (named 0 and 1), each folder with CSV-sequences from the respective class. In each folder there are .csv files, which have 3 columns and a variable number of rows (=variable timestemp length).

There's no class in the csv file, just features. Each csv looks like this:

x1,x2,x3
x1,x2,x3
...
x1,x2,x3

The code for my example model is:

model = Sequential()
model.add(CuDNNLSTM(3, input_shape=(None, 3), return_sequences=False))
model.add(Dropout(0.1))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
      optimizer=tf.keras.optimizers.Adam(lr=1e-2, decay=1e-7),
      metrics=['accuracy'])

Every sequence (so f.e. [[x1,x2,x3], [x1,x2,x3], [x1,x2,x3]]) should only have one label. Thus when predicting this sequence I'd get either 0 or 1.

Question 1:
- How can I create something like an iterator that I can pass for trainX and trainY containing every sequence with it's class? (Class=foldername or I can also provide labels with an array)
- How can I load my data from the two folders with their labels into this iterator?
Question 2: Is there anything else to make variable timesteps possible other than batchsize of 1? Question 3: Would this be correct code to have variable timestep length accepted by the LSTM? If not, please suggest a better way.

input_shape=(None, 3)


from How can I load this data into a LSTM?

No comments:

Post a Comment