Autoencoder clustering: What it got right (face size) and wrong (everything else)

A quick experiment clustering Google QuickDraw faces with an RNN autoencoder produced a predictable result: the encoder latent vectors separated faces mostly by overall scale, not by their interior features.

Input format: lines as vector sequences

Each face is encoded as a sequence of vectors — one per stroke, similar to an SVG path. An example face from the dataset:

tensor([[-0.0956, -0.2869,  0.0000],
        [-0.3108, -0.1434,  0.0000],
        [-0.5738,  0.1673,  0.0000],
        [-0.3108,  0.2391,  0.0000],
        [-0.3586,  0.4303,  0.0000],
        ... lots more ...

Architecture: sequence → vector → sequence

The model uses two recurrent networks:

  • Encoder: maps the vector sequence to a single 50-dimensional vector.
  • Decoder: reconstructs the original vector sequence from that one vector.

Training minimizes the mean squared error between the input x and Decoder(Encoder(x)). In practice, loss only dropped from roughly 0.3 to 0.25 — clearly under-trained, but stoppable.

Reconstruction: decent circles, weak interiors

Four training examples with their reconstructions:

The decoder clearly learned to draw a face-sized ellipse and, loosely, that something belongs inside it. Fine details — eyes, mouths, expressions — essentially collapse. Given the modest error reduction, this is reasonable behavior, not a surprise.

Clustering: DBSCAN on 50-D embeddings

The original intent was to cluster the QuickDraw faces by feeding Encoder(x) outputs to DBSCAN from scikit-learn, chosen mainly because it might handle high-dimensional vectors better than alternatives.

Visual inspection of the clusters shows separation controlled almost entirely by face diameter. That aligns directly with the autoencoder’s strengths: it reconstructs the outer contour well but loses interior structure, so the embedding space carries little signal about facial features.

Next step: improve interior reconstruction

The follow-up is to tweak the model so the decoder can faithfully draw eyes and other inner components. Until then, "clustering faces" is really "clustering circles."