Friday 30 July 2021

Using Grad-Cam for the edge tpu Coral device

I am using the Edge TPU Coral USB device for model inferencing, where I am doing image classification tasks. I have a custom trained tflite model which is being used for the classification. What I am trying to do is to run grad-cam, so I can visualize the activation maps for the image being classified for on an already trained TFLite model. For some reason, I can't find a tutorial. Based on this question, it should be possible: Is it possible to apply GradCam to a TF Lite model. However, there isn't a clear explanation on how to perform the layers check for the inputs and the gradient data. I currently have this grad-cam example:

I am using the grad-cam from this author in Google Colab: Colab Notebook

Additionally, for image classification I have the following code:

import argparse
import time

from PIL import Image
from pycoral.adapters import classify
from pycoral.adapters import common
from pycoral.utils.dataset import read_label_file
from pycoral.utils.edgetpu import make_interpreter
import cv2 as cv
import numpy as np


def main():
  parser = argparse.ArgumentParser(
      formatter_class=argparse.ArgumentDefaultsHelpFormatter)
  parser.add_argument('-m', '--model', required=True,
                      help='File path of .tflite file.')
  parser.add_argument('-i', '--input', required=True,
                      help='Image to be classified.')
  parser.add_argument('-l', '--labels',
                      help='File path of labels file.')
  parser.add_argument('-k', '--top_k', type=int, default=2,
                      help='Max number of classification results')
  parser.add_argument('-t', '--threshold', type=float, default=0.0,
                      help='Classification score threshold')
  parser.add_argument('-c', '--count', type=int, default=5,
                      help='Number of times to run inference')
  args = parser.parse_args()

  labels = read_label_file(args.labels) if args.labels else {}

  interpreter = make_interpreter(*args.model.split('@'))
  interpreter.allocate_tensors()
  print(interpreter)

  size = common.input_size(interpreter)
  image = cv.imread(args.input)
  image = cv.normalize(image, image, 0, 255, cv.NORM_MINMAX)
  common.set_input(interpreter, image)

  print('----INFERENCE TIME----')
  print('Note: The first inference on Edge TPU is slow because it includes',
        'loading the model into Edge TPU memory.')
  for _ in range(args.count):
    start = time.perf_counter()
    interpreter.invoke()
    inference_time = time.perf_counter() - start
    classes = classify.get_classes(interpreter, args.top_k, args.threshold)
    print('%.1fms' % (inference_time * 1000))

  print('-------RESULTS--------')
  for c in classes:
    print('%s: %.5f' % (labels.get(c.id, c.id), c.score))


if __name__ == '__main__':
  main()

I am honestly puzzled on how to access the layers of a TFlite model to check for the gradient values, given that a TFLite model uses tensors, I'd like to know how can i use the pycoral/tflite libraries to use grad-cam, instead of keras/tensorflow models?



from Using Grad-Cam for the edge tpu Coral device

No comments:

Post a Comment