Specify shape for categorical feature columns?

I know that I can use a categorical_column_with_identity to turn a categorical feature into a series of one-hot features.

For instance, if my vocabulary is ["ON", "OFF", "UNKNOWN"]:
"OFF" -> [0, 1, 0]

categorical_column = tf.feature_column.categorical_column_with_identity('column_name', num_buckets=3)
feature_column = tf.feature_column.indicator_column(categorical_column))

---

However, I actually have an 1-dimensional array of categorical features. I would like to turn that into a 2-dimensional series of one-hot features:

["OFF", "ON", "OFF", "UNKNOWN", "ON"]
->
[[0, 1, 0], [1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 0, 0]]

Unlike every other feature column, it doesn't seem like there's a shape attribute on categorical_column_with_identity and I didn't find any help through Google or the docs.

Do I have to give up on categorical_column_with_identity and create the 2D array myself through a numerical_column?

from Specify shape for categorical feature columns?