`PixPlot is a web-based tool that utilizes machine learning technology to facilitate the visualization of large numbers of images. Developed by Yale, PixPlot allows us at the VRC to offer one new, novel way, to enter the ever expanding collection of digitized works we have on hand. We have implemented two instantiations of PixPlot for demonstration purposes: the first is a 2D constellation of the post-1945 collection of close to 23,000 images, and the second is a 3d constellation of the same collection. We will soon expand this technology to other areas of our collection.
PixPlot allows the user to experience the discovery of images similar to finding adjacent books on the library shelf: in other words, PixPlot finds, using a convolutional neural network, formal and aesthetic similarities between the inputted images. One might expect to find, for example, colour field paintings clustered together. It is an extreme form of an aesthetic formalism that permits the potential discovery of those artists working in a similar style, yet possibly overlooked.
You will note that there are a series of what PixPlot terms "hotspots" on the left-most side of the browser, once you've entered into the visualization space. The program uses Hierarchical density-based spatial clustering of applications with noise (HDBSCAN) to auto-generate these hotspots, or clusters, of formal similarities. These are not exclusive to the visualization, and there may be (many) more that go undetected and not listed among the hotspots. Note that the labels are somewhat arbitrary and subject to revision. It is best to freely explore, and see what you find.
To derive the constellation, PixPlot uses UMAP (Uniform Manifold Approximation and Projection for Dimension Reduction), which allows for an assortment of parameters that affect the density and the dimensions of the visualization.
For a live demonstration, see here: