Cognitive (AI) Based Abstraction

Research Collaborators
Steve DiPaola , Graeme McCaig , Vanessa Utz , Nouf Abukhodair , Hanieh Shakeri

What is abstraction? Can you use AI techniques to model the semantics of an idea, object, or entity, where that understanding allows for abstraction of the meaning? We use several AI techniques including genetic programming, Neural Nets and Deep Learning to explore abstraction in its many forms. Mainly here in the visual and narrative arts.

The Research
Our AI thinker-painter system takes on planar (e.g. cubist style) abstraction. Many AI creativity and NPR systems strive for abstraction by simply varying parameters like stroke length to achieve a level of abstraction. This is a worthy way to create rendered abstraction but leaves out cognitive abstraction where the artist is using abstraction in deeper ways for instance to concept blend different thoughts or meanings. Our first attempt at this deeper “thinker” abstraction is with deep or severe planar abstraction. This neural network system uses a regressor based image stylization which has been influenced by blends of genetic algorithms and hill.

* Please click on images to view in full screen *

Steve - Self Portrait - Triptych

Setup and Results
The Python based system allows us to control the level from very severe abstraction (i.e. 3 planes) all the way through mild abstraction with 100’s of planes as in figure 2. It should be noted that this “thinking” AI phase first outputs less painterly low resolution simple planes that have one colour per plane, simulating not a finish painting at this point but thinking model about abstracted planes in the artist’s mind. To make the finished thinker-painter art pieces in the figures, we then take this inner abstraction output and send it through our Painterly NPR system where artistic distortion, human level variation and brush stroking complete the “painting on canvas” look of the final work. Several different levels of first, AI planer abstraction are completed with several different NPR recipes of painterly rendering. Our research lab is now in performing user studies where we testing users reaction to a range of abstractions from this AI output. Our early test results show that users both prefer the abstractions to the original photography they were created from in terms of a work of art, as well are able to identify correct aesthetic descriptors (e.g. cheerful, trust, awe, scary,…) that people see the original source photograph in even severely abstracted versions. This preliminary study data supports that this AI based abstraction technique seems to have some qualities of human based abstract art – that the output holds the qualities inherent to the original sitter but within a simpler aesthetic form.

Downloads and Links
PDF: EVA 2016 Using Artificial Intelligence Techniques to Emulate the Creativity of a Portrait Painter

Additional Media and Code
Images Gallery of Images
More Images  Gallery of Images