At the core of the digital, as the etymology suggests, lies the process of dividing everything into clearly distinguishable units. Every process and every being has to be classifiable to survive the digital paradigm, even though analogue matter is not per se made to be compartmentalized and digitally readable. Image analysis software, colloquially referred to as "image recognition" (even though, strictly speaking the software does not recognize anything), is deeply rooted in this digital motivation to calculate, measure and categorize.
“Physiognomy of Russian Female Offenders”from the chapter “Facial and CephalicAnomalies of Female Criminals,” in "TheFemale Offender" by Cæsar Lombroso andWilliam Ferrero (1898)
The obsession with classification and categorization inherent in image analysis software follows a long tradition of authoritarian nation states to count, classify and evaluate the bodies their populations are made of. Images (and above all photographs) of those bodies have hereby served historically to map out the abnormal against the normal, to construct the other against the aligned subject. The pervert, the criminal, the Jewish, the colonized, the racial and sexual other have been dragged into the light of the sovereign's camera, banned on paper, their skulls measured, their eyes read, their body parts examined in an insatiable fascination with and tremendous fear of the unknown. In order to construct the normal (heterosexual, white, European, male) subject as the norm, everybody else had to be neatly classified into different sub-categories of abnormal. Image analysis software picks up these threads and spins a new chapter in the story of body-classification techniques.
Detail from the taxonomy of Google's Open Images Dataset. The category Person contains 4 subcategories: Man, Woman, Girl, Boy. The category Weapon contains 12 subcategories.
Screenshot from Google Open Images Database
Learning from Machine Learning is an investigation into the methodology of algorithmic image and text processing. Ultimately, it is a strategy to embrace the performative character of machine learning procedures. The goal here is not simply to illustrate the pitfalls of algorithmic procedures, but also to speculate about ways of working-with-the-machine, rather than against it.
GIF flip image: an image of the author and an algorithmically generated image with similar features
An image is taken by a webcam and analyzed with Google Cloud VisionThe 10 most probable tags (according to Google Cloud Vision image analysis) are displayed
Google Cloud Vision, a software to analyze images which detects faces and objects and categorizes images.https://cloud.google.com/vision/docs/drag-and-droptested with an image from https://thispersondoesnotexist.com/
The tags created in Step 1 — the 10 labels that ranked highest in the image analysis software's list of tags — are now fed into Dall:e, a text-to-image generator (based on neural networks). Dall:e mini creates new images based on the tags given.https://huggingface.co/spaces/flax-community/dalle-mini
Happy — Nose — White-Collar Worker — Portrait Photography — Smile — Forehead — Event — Makeover — Hair Care — Cosmetic DentistryOutput Images for these 10 tags
The new images are fed back into the image analysis software, where new tags are generated. These tags go back into the text-to-image algorithm and new images are created. Originating in one initial image, layer by layer more images and words populate the screen with parasitic productivity. An eternal dialogue between two machines, learning: over-tagging, over-reading, over-reacting, thereby undermining the authority of unambiguous categorizations.
Together, me and the machine perform the hypersensitivity of the neural networks until all categories become no category at all and all classifications collapse. A small marble is set rolling in the beginning and soon, two algorithmic systems speak to each other in an eternal loop, whispering in their own strange code. (--> Sadie Plant, Zeros + Ones)
