Steve DiPaola

A Professor at Simon Fraser University, Steve directs the I-Viz Lab (ivizlab.sfu.ca) which strives to make computer systems bend more to the human experience. He came to SFU from Stanford University and before that spent 10 years as a senior researcher at NYIT Computer Graphics Lab, an early pioneering lab in high-end 3D techniques. He has held senior positions at Electronic Arts and Saatchi & Saatchi Innovation and has consulted for HP, Kodak, Macromedia and the Institute for the Future. His art work has been exhibited internationally including the A.I.R. and Tibor de Nagy galleries in NYC as well as the Whitney Museum of Art, and the IBM Gallery of Science and Art. He co-curated the first computer art show in a major NYC gallery in 1988.

Contact: sdipaola@sfu.ca

Position: Professor

https://www.dipaola.org/art/

AI Affective Virtual Human

Our affective real-time 3D AI virtual human project with face emotion recognition, movement recognition and full AI talking, gesture and reasoning.

AI, AvatarsAgents, Character3D, Creativity, EmotionExpression, PainterlyRendering, Vision

BioSensing 2D / 3D / VR Systems

Our lab has extensive experience in using different sensing technology including eye tracking and facial emotion recognition (DiPaola et al 2013), as well as gesture tracking and heart rate and EDA bio sensing (Song & DiPaola, 2015) to affect generative computer graphics systems.

BioVR, Creativity, GenerativeDesign, VRWorlds

AI Anonymization

Bringing together an interdisciplinary team, we created a wholly new AI technique to anonymize interview subjects and scenes in regular and 360 videos to create a technique that would be much better at conveying emotional and knowledge information than current anonymization techniques.

AI, Creativity, EmotionExpression, PainterlyRendering, Vision

Deep Learning AI Creativity For Visuals / Words

Using Cognitive Science as a basis for our work, we attempt to model aspects of human creativity in AI. Specially we are using Neural Networks (and evolutionary systems) in the form of Deep Learning, CNNs, RNNs and other modern techniques to model aspects of human expression and creativity.

AI, CogSci, Creativity, GenerativeDesign, NeuralNets, NPR, PainterlyRendering, Vision

Cognitive (AI) Based Abstraction

What is abstraction? Can you use AI techniques to model the semantics of an idea, object, or entity, where that understanding allows for abstraction of the meaning? We use several AI techniques including genetic programming, Neural Nets and Deep Learning to explore abstraction in its many forms. Mainly here in the visual and narrative arts.

AI, CogSci, Creativity, NeuralNets, NPR, PainterlyRendering, Vision, VRWorlds

Evolving Creative Computer Programs w/Genetic Programming

This research uses creative evolutionary systems to explore computer creativity for various applications (in our first pass – evolving a family of abstract portrait painter programs). We use relatively new form of Genetic Programming (GP) called Cartesian Genetic Programming (CGP) first developed by Julian Miller .

AI, Creativity, GenerativeDesign, GeneticProgramming

Painterly NPR Project

Portrait artists and painters in general have over centuries developed, a little understood, intuitive and open methodology that exploits cognitive mechanisms in the human perception and visual system.

CogSci, Creativity, EmotionExpression, PainterlyRendering, Vision

Rembrandt / Vision Science Work

Using new visual computer modelling techniques, we show that artists use vision based techniques (lost and found edges, center of focus techniques) to guide the eye path of the viewer through their paintings in significant ways.

CogSci, Creativity, FaceResearch, PainterlyRendering, Vision

faceToolKit - A 3D Facial ToolKit

Our long range research project is a visual development system for exploring face space, both in terms of facial types and animated expressions. This development toolkit is based on a hierarchical parametric approach. This approach gives us an additive language of hierarchical expressions, emotions and lip-sync sequences.

Character3D, CogSci, EmotionExpression, FaceResearch, GamesResearch

iFACE - Comprehensive Envir for Interactive Face Anim

Rapid of growth of visual communication systems, from video phones to virtual agents in games and web services, has a brought a new generation of multimedia systems that we refer to as face-centric. Such systems are mainly concerned with multimedia representation of facial activities.

Character3D, CogSci, EmotionExpression, FaceResearch, GamesResearch

Virtual Beluga Project - Vancouver Aquarium

Actual screen shot from our Virtual Beluga Interactive Prototype which shows realistically swimming Beluga in a wild grouping (pod) created via 3d real-time graphics and artificial intelligence systems.

AI, Character3D, NeuralNets

A Social Metaphor-based Virtual Communities (voiceAvatar)

The design goal of this project was to develop avatars and virtual communities where the participants sense a tele-presence – that they are really there in the virtual space with other people. This collective sense of “being-there” does not happen over the phone or with teleconferencing; it is a new and emerging phenomenon, unique to 3D virtual communities.

AvatarsAgents, Character3D, EmotionExpression, GamesResearch, VRWorlds

MusicFace

Can you extract the emotional aspects of a piece of music to animate a face. Music-driven Emotionally Expressive Face (MusicFace) is a early stage project that creates “facial choreography” driven by musical input. In addition to its artistic uses, MusicFace can be used for creating visual effects in movies and animations, and also realistic characters in computer games and virtual worlds.

AI, Character3D, CogSci, EmotionExpression, FaceResearch, PainterlyRendering

Virtual CoLab Project

Virtual Colab and the real CECM Colab located at SFU.
Every smart-board computer display is matched by an in-world browser in 3d
which are able to display the same web based information for distanced collaborators.

AvatarsAgents, Character3D, ColabDesign, KnowledgeViz, VRWorlds

KnowViz

Using open source tools ( python / openGL / … ), our research group is working to create an intuitive, interactive 3D knowledge visualization system that explores a more organic approach to knowledge and data visualization. The hope is by borrowing from our interests in alternative user interface design, AI and aLife systems, visual and interaction design as well as intelligent systems that we can create a more living and intuitive system for exploring date spaces.

KnowledgeViz, VRWorlds

SFU/FIT Collaborative Design Project

Collaboratively created cyber-fashion show, where sketches (white bg)
from FIT fashion designers are turned into 3d avatar models (black bg) by SFU students
– all using distance collaborative tools between two coasts and countries.

AvatarsAgents, Character3D, ColabDesign, KnowledgeViz, VRWorlds

GenFace - Exploring FaceSpace with Genetic Algorithms

Imagine an -dimensional space describing every conceivable humanoid face, where each dimension represents a different facial characteristic. Within this continuous space, it would be possible to traverse a path from any face to any other face, morphing through locally similar faces along that path. We will describe and demonstrate a development system we have created to explore what it means to ‘surf’through face space. We will present our investigation of the relationships between facial types and how this understanding can be used to create new communication and expression systems.

AI, Character3D, FaceResearch, GamesResearch, GenerativeDesign, GeneticProgramming

AI Empathetic Painter

We will combine our AI work in Empathy based modeling for AI Character Agents with our Deep Learning-based Creativity system (see papers and work in PDF) that realizes a fine art portrait from sitters.

https://ivizlab.org/wp-content/uploads/sites/2/2021/06/paper2-300x300.jpg

Artistic style meets Artificial Intelligence
Journal Article: Journal of Perceptual Imaging, 2021
Choi, SK., DiPaola, S., Töyrylä, H. (2021). Artistic Style Meets Artificial Intelligence. Journal of Perceptual Imaging, 4(3).

S. Choi, S. DiPaola

AI, Creativity

External Link

https://ivizlab.org/wp-content/uploads/sites/2/2021/06/paper3-300x300.jpg

Social Prescribing Across the Lifespan with Virtual Humans
Conference Proceedings: IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, 2020

N. Yalcin, S. DiPaola

AvatarsAgents

External Link

https://ivizlab.org/wp-content/uploads/sites/2/2021/06/paper4-300x300.jpg

Empathic AI Painter: A Computational Creativity System with Embodied Conversational Interaction
Conference Proceedings: Proceedings of Machine Learning Research (NeurIPS 2019 Competition and Demonstration Track), 2020
10 Pages

N. Yalcin, N. Abukhodair, S. DiPaola

AI, AvatarsAgents, Creativity, EmotionExpression, PainterlyRendering

External Link

https://ivizlab.org/wp-content/uploads/sites/2/2021/06/paper5-300x300.jpg

A multi-layer Artificial Intelligence and sensing based Affective Conversational Embodied Agent
Conference Proceedings: IEEE Computing and Intelligent Interaction (ACIIW), 2019

N. Yalcin, S. DiPaola

AI, AvatarsAgents, EmotionExpression

External Link

Using an AI creativity system to explore how aesthetic experiences are processed along the brains perceptual neural pathways
Journal Article: Cognitive Systems Research, 2019
19 pages. DOI: https://doi.org/10.1016/j.cogsys.2019.09.012

V. Utz, S. DiPaola

CogSci, Creativity, NeuralNets, Vision

External Link

https://ivizlab.org/wp-content/uploads/sites/2/2019/09/Bica2019-Utz-300x261.png

Aesthetic Judgments, Movement Perception and the Neural Architecture of the Visual System
Conference Proceedings: Biologically Inspired Cognitive Architectures Meeting, August 15, 2019 2019
pp. 538-546. DOI: https://link.springer.com/chapter/10.1007/978-3-030-25719-4_70

V. Utz, S. DiPaola

Seattle, USA

Springer, Cham

Steve DiPaola

Research

Publications