We combine our AI work in Empathy based modeling for AI Character Agents with our Deep Learning-based Creativity system (see papers / demo pix) that realizes an empathy-driven fine art portrait from sitters who converse with our AI artist. The interaction emulates a live portrait artist who talks with you and attempts to understand your mood and feelings so they can emphatically create a unique portrait of you in color palette / style / abstraction to match who you are and your mood. We do this using our AI-based cognitive modeling using several cutting edges AI techniques as discussed in our published papers. The system uses AI tools created in our research lab including facial emotion recognition, segmentation of faces, AI salience detection of areas that people tend to look at, empathy modeling using both visual and NLP data ( face recognition, voice stress, semantic NLP, sentiment analysis, a new hybrid creativity based deep dream/deep style system and more). The work was selected and shown as best of AI a demo showcase work at NeurIPS ( NIPS) 2019 – the largest Ai conference in Vancouver Canada. It uses over 6 different AI processes from our research work from facial emotion recognition to Natural Language understanding to CNN based artistic painterly rendering to Semantic creativity modeling.
Talking with our AI avatar. With voice, face, gesture and words emotional empathy modeling.
Here a man chats with our AI artist, and our AI systems, based on the conversation attempts to paint a portrait using an array of techniques it thinks best fits the empathy it feels and the personality of the sitter.
Prof DiPaola with 1 postdoc and 2 PhDs from his lab – demoing live to NeurIPS 19 AI experts who signed up to talk to our empathetic AI avatar and then based on that conversation get their portrait painted.
The work was then taken up by the national press where DIPaola was interview during the event on teh national news. SEE VIDEO AFTER AD: https://www.ctvnews.ca/video?clipId=1850625
Gallery of empathetic portrait styles of that same man, that our AI can build up from its creativity based techniques (subset):
Users see completed portraits in many different fine art abstract styles ( and gally and museum-quality – since our work is typically curated there, (see PDF) in one large projected area that were done live through the demonstration show. Users will be able to sit one at a time and go through an AI-based empathetic conversation with our AI artist painter agent and from that conversational session have their portrait created, which after a few minutes will show up on the completed wall. Many different emotional painterly styles will be demonstrated based on the mood and discussion with each user. The process of emotional modeling and rendering will be demonstrated on another screen. Users will witness and / or take part in this continuous artistic session. And debate the results and process.
Users converse with our Artist which is an AI ECA ( Embodied Conversational Agent with significant research into cognitive empathy modelling with it. See chart: 
With this conversation, an emotional model is created and with other AI sensed information from the user, this emotion profile then is passed to our Deep Learning Painterly System ( also with significant research behind it) to create a unique portrait painting of the user which will be added to the large projected display.
See the ECA system here:
AI Affective Virtual Human
Showcase Title: Empathy based Affective Portrait Painter DiPaola, Yalcin, Abukhodair
Can an AI system using a set of state of the art AI modules, act like a portrait painter who first meets a sitter, chats with them to understand who they are and then using their experiential sense of that person create a portrait that both archives a fine art personalized portrait of both the outer resemblance but also the inner ( resemblance of) mood and gestalt of that sitter. We will combine the two main areas of our research lab, Cognitive based empathy in AI conversational agents and our cognitive-based creativity AI painterly system to have.
For our demo showcase and work:
1) Viewers 1 by 1, talk with our AI empathetic fully 3D character agent system ( our painter)
2) Based on analysis and mood of that conversation (empathy modeling) create a fine art portrait
3) Display that Portrait on the large projected wall with our portraits, newest first
————–
Steve DiPaola – ivizlab.org – sdipaola@sfu.ca
With Postdoc researchers Ozge Nilay Yalcin and PhD Nouf Abukhodair ( and other PhDs at DIPaola’s research lab – ivizlab.org – and the School fo Interactive Art and Tech (SIAT) at Simon Fraser University.
See our papers and corresponding main research areas.
AI Virtual Humans
Deep Learning AI Creativity For Visuals / Words