LINEAGE : Find what you didn’t know you were looking for, with AI (6)


Friday, April 05, 2019: 1:00pm - 2:00pm - Constitution

Noya Kohavi, Brown Institute for Media Innovation, Columbia University, USA

LINEAGE is an artificial intelligence engine that enables exploration of digitized visual archives in a human-like manner. The user inputs any image, and the engine returns similar images from art and design history. The returned images are not identical to the original but rather gives the user the cultural context in which the input image exists. LINEAGE works without any text keywords, and does not discriminate by medium. Its similarity relies on colors, shapes, patterns and their layered combinations, mimicking the way humans look at objects, and encouraging serendipitous connections across time periods, location of origin, creator and medium. 
LINEAGE was conceived as a research tool for fashion and art journalists, allowing them to find the cultural and visual lineage of new pieces of art and fashion so they can better contextualize them for their audience. But in the basis of this project lies a shift in the way we search and discover visual culture, one that does not presume knowledge and uses the visual as a gateway to the textual rather than the other way around.
Museum digital archives are incredibly well annotated resources, but unless you know precisely what it is you are after (and are proficient in English), they can be difficult to search and thus to utilize. LINEAGE bridges the gap between archivers and the curious public by creating a visual way to search and explore archives without presuming prior knowledge in art history.
This talk will cover the motivation to build LINEAGE, explain the notion of visual search as compared to text search, show use cases and outline the technology behind it. Attendees of this talk should walk away with a new perspective on how AI/computer vision technology can make their institute archives accessible and exciting to explore, and encourage outsiders in. 

When researching the features and design of LINEAGE, I have surveyed major museum's public online archives (notably learning from advances in visual search features offered by Cooper Hewitt, The Barnes Collection, the Rijksmuseum and others ) as well as other visual databases (such as Pinterest similarity and explore features). I used this survey to determine what tools are currently available and where technological and user-interface gaps exist. Some of the features being developed for LINEAGE are a direct result of this survey, such as more human-like color sorting and choosing to use visual data solely rather than relying on textual tags. This research is continuing in parallel to development.