Tomás Laurenzo, Katia Vega: Box

 
  • ©,

  • ©,

Artist Statement:


    Summary

    BOX is a gumball machine augmented by artificial intelligence that locks or unlocks its delivery candy system depending on the interactor’s ethnicity. It highlights potential ethical tensions that arise with the introduction of AI-powered objects.

    Abstract

    BOX is an interactive installation, consisting on an everyday object augmented by artificial intelligence. The piece reflects on the power asymmetries that technology instantiates, aiming at providing with a reflection on the aesthetics of our relationship with it. The artwork also aims to showcasing the advancements and limitations in computer vision and artificial intelligence, allowing the public to experience in person its power as well as its inherent biases.

    Recent advances in computer vision and artificial intelligence, have allowed the creation of systems able to infer (predict) information on a person from camera data, including facial recognition, facial expressions, ethnicity, among others. Nowadays, several companies provide image processing services that include these predictions, among several others.

    In spite of potential benefits that face recognition proposes, its widespread application entails several risks, from privacy breaches to systematic discrimination in areas such as hiring, policing, benefits assignment, marketing, and other purposes BOX consists of a gumball machine that, using computer vision and machine learning, predicts its user’s ethnicity, delivering free candy only to white users.

    The artwork showcases a possible use of computer vision making explicit the fact that every technological implantation crystallises a political worldview, allowing the general public to experience in person the power of these new technologies, while simultaneously providing a tool for participatory observation, as well as ethnographic and technographic research.

    Our project aims to raise awareness on discrimination, ethics, and accountability in AI among practitioners and the general public.

Technical Information:


    The artwork use a standard webcam (HD, 30FPS) to acquire data on its interactors. Two different implementations have been created, both running at interactive rates on a 2018 MacBook Pro laptop.

    The first uses a deep neural network based on David Sandberg’s FaceNet, and trained using Tensorflow on the UTKFace dataset. UTKFace contains 23,708 faces with five labelled races (White, Black, Asian, Indian, and Others (e.g. Hispanic, Latino, Middle Eastern)).

    A second implementation uses Affectiva.ai’s pre-trained model run locally. This service classifies the input images also in five different ethnicities (White, Black, South East Asian, Asian, and Latino).

    Once the interactor’s ethnicity has been predicted by the system, the device is either kept locked or unlocked, using a custom system based on a servomotor, controlled by an Arduino Uno microcontroller.

    In addition to the commercial gumball machine, camera connected and laptop, BOX includes an LED display. When an interactor’s motion is recognized, the LED display shows a message “detecting ethnicity”. If the predictor returns a White ethnicity, BOX unlocks its handle of showing a message on the display: “Ethnicity detected: White.” If it detects any other ethnicity, the machine stays locked and the corresponding message is shown.