Machine learning models are increasingly applied in design and engineering contexts to accompany and inform the process of making design decisions. Understanding how the training data for these models is structured is important to fostering critical engagements with contemporary, data-intensive design practices. Through a case study in urban data comprising thousands of geometric models of buildings in Montréal, Canada, A Walk in a Latent City, is a custom immersive 3-D virtual environment allowing users to leap back and forth between the urban space of the city and a spatial representation of the latent space of the machine learning model.
The 3D Montreal dataset can be placed with two coordinates in different spaces: geographic space of Montreal (Montreal space) and the latent space from feature extraction of the variational autoencoder (VAE). The geospatial coordinates of the dataset allowed us to reconstruct the building models as they would be in the real world. As for representing latent space, we formatted the latent embeddings as we would the geospatial coordinates, constructing both spaces is a similar process.
The first-person experience and game-inspired interface alter the space that data is explored and hence change the way data is explored. I found that movement as your own agent in space encourages an experience that is more fluid and intuition-driven. Identifying points of interest in the dataset became based upon primarily visual cues; choosing to inspect a building is often a result of the building being of particular formal interest, its adjacencies in data space, or by its color. Furthermore, the responsiveness to moving your camera in first-person to highlight and travel through the data samples punctuates that the experience of exploration is about viewership and movement.
Locating clusters is a key part of data visualization and evaluating features in machine learning and latent spaces. Observed, particular to this urban dataset is how clusters of buildings, or perhaps even emergent typologies, correlate to their distribution in the corresponding urban space. This observation is helped by the relationship colors have to clusters of latent embeddings. Qualitative observation of latent space clusterings is as observed: a spread of purple and pink buildings, clusters of red buildings, and outlier buildings of greener and blue buildings.
