Deep learning to classify a time series of xy spatial coordinates - python

I've got a few problems with a DL classification problem. I'll attach a brief example of the training data to help describe the problem.

The data is a time series of xy points, which is made up of smaller sub-sequences event. So each unique event is independent. I have two unique sequences (10,20) below of even time length. For a given sequence, each individual point has its own unique identifier user_id. The xy trace of these points will vary marginally over a given sequence, with the specific time period found in interval. I also have a separate xy point used as a reference (centre_x, center_y), which details the approx middle/centre of all points.

Lastly, the target_label classifies where these points are relative to each other. So using the centre_x, center_y as a reference, there are 5 class Middle, Top, Bottom, Right, Left. There can only be one label for each unique event.

Problems:

1. Obviously small dataset but I'm concerned with the accuracy accuracy. I think I need to incorporate the reference point (centre_x, center_y)

2. I'm getting all these warning for each test iteration. I think it has something to do with converting to a tensor but it doesn't;t help anything.

   WARNING:tensorflow:7 out of the last 7 calls to <function Model.make_test_function..test_function at 0x7faa21629820> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.

example df:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# number of intervals
n = 10

# center locations for points
locs_1 = {'A': (5,5),
      'B': (5,8),
      'C': (5,2),
      'D': (8,5)}

# initialize data 
data_1 = pd.DataFrame(index=range(n*len(locs_1)), columns=['x','y','user_id'])
for i, group in enumerate(locs_1.keys()):

    data_1.loc[i*n:((i+1)*n)-1,['x','y']] = np.random.normal(locs_1[group], 
                                                       [0.2,0.2], 
                                                       [n,2]) 
    data_1.loc[i*n:((i+1)*n)-1,['user_id']] = group

# generate time interavls
data_1['interval'] = data_1.groupby('user_id').cumcount() + 1

# assign unique string to differentiate sequences
data_1['event'] = 10

# center of all points for unqiue sequence 1
data_1['center_x'] = 5
data_1['center_y'] = 5

# classify labels
data_1['target_label'] = ['Middle' if ele  == 'A' else 'Top' if ele == 'B' else 'Bottom' if ele == 'C' else 'Right' for ele in data_1['user_id']]

# center locations for points
locs_2 = {'A': (14,15),
      'B': (16,15),
      'C': (15,12),
      'D': (19,15)}

# initialize data 
data_2 = pd.DataFrame(index=range(n*len(locs_2)), columns=['x','y','user_id'])
for i, group in enumerate(locs_2.keys()):

    data_2.loc[i*n:((i+1)*n)-1,['x','y']] = np.random.normal(locs_2[group], 
                                                       [0.2,0.2], 
                                                       [n,2]) 
    data_2.loc[i*n:((i+1)*n)-1,['user_id']] = group

# generate time interavls
data_2['interval'] = data_2.groupby('user_id').cumcount() + 1

# center of points for unqiue sequence 1
data_2['event'] = 20

# center of all points for unqiue sequence 2
data_2['center_x'] = 15
data_2['center_y'] = 15

# classify labels
data_2['target_label'] = ['Middle' if ele  == 'A' else 'Middle' if ele == 'B' else 'Bottom' if ele == 'C' else 'Right' for ele in data_2['user_id']]

df = pd.concat([data_1, data_2])

df = df.sort_values(by = ['event','interval','user_id']).reset_index(drop = True)

df:

            x          y user_id  interval  event  center_x  center_y target_label
0    5.288275   5.211246       A         1     10         5         5       Middle
1    4.765987   8.200895       B         1     10         5         5          Top
2    4.943518   1.645249       C         1     10         5         5       Bottom
3    7.930763   4.965233       D         1     10         5         5        Right
4    4.866746   4.980674       A         2     10         5         5       Middle
..        ...        ...     ...       ...    ...       ...       ...          ...
75  18.929254  15.297437       D         9     20        15        15        Right
76  13.701538  15.049276       A        10     20        15        15       Middle
77  16.028816  14.985672       B        10     20        15        15       Middle
78  15.044336  11.631358       C        10     20        15        15       Bottom
79   18.95508  15.217064       D        10     20        15        15        Right

Model:

labels = df['target_label'].dropna().sort_values().unique()

n_samples = df.groupby(['user_id', 'event']).ngroups
n_ints = 10

X = df[['x','y']].values.reshape(n_samples, n_ints, 2).astype('float32')

y = df.drop_duplicates(subset = ['event','user_id','target_label'])

y = np.array(y['target_label'].groupby(level = 0).apply(lambda x: [x.values[0]]).tolist())

y = label_binarize(y, classes = labels)

# test, train split
trainX, testX, trainy, testy = train_test_split(X, y, test_size = 0.2)

# load the dataset, returns train and test X and y elements
def load_dataset():

    # test, train split
    trainX, testX, trainy, testy = train_test_split(X, y, test_size = 0.2)

    return trainX, trainy, testX, testy

# fit and evaluate a model
def evaluate_model(trainX, trainy, testX, testy):
    verbose, epochs, batch_size = 0, 10, 32
    n_timesteps, n_features, n_outputs = trainX.shape[1], trainX.shape[2], trainy.shape[1]
    model = Sequential()
    model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(n_timesteps,n_features)))
    model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
    model.add(Dropout(0.5))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Flatten())
    model.add(Dense(100, activation='relu'))
    model.add(Dense(n_outputs, activation='softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    # fit network
    model.fit(trainX, trainy, epochs=epochs, batch_size=batch_size, verbose=verbose)
    # evaluate model
    _, accuracy = model.evaluate(testX, testy, batch_size=batch_size, verbose=0)
    return accuracy

# summarize scores
def summarize_results(scores):
    print(scores)
    m, s = np.mean(scores), np.std(scores)
    print('Accuracy: %.3f%% (+/-%.3f)' % (m, s))

# run an experiment
def run_experiment(repeats=10):
    # load data
    trainX, trainy, testX, testy = load_dataset()
    # repeat experiment
    scores = list()
    for r in range(repeats):
        #r = tf.convert_to_tensor(r, dtype=tf.int32)
        score = evaluate_model(trainX, trainy, testX, testy)
        score = score * 100.0
        print('>#%d: %.3f' % (r+1, score))
        scores.append(score)
    # summarize results
    summarize_results(scores)

# run the experiment
run_experiment()

from Deep learning to classify a time series of xy spatial coordinates - python