Visual Agent Laboratory

We are developing novel visual applications based on images or graphics technology such as style-based retrieval, edit, and conversion of illustrations and fonts, smart digital lighting including visible light communication, high dynamic range images, and mobile camera communication with visual code for exploring new media environments powered by smart visual agents!

On-Going Projects

Style-Based Manipulations of Graphical Media

<<< Keynote speech with respect to this project >>>

Style-Based Retrieval and Manipulations of Illustrations

Retrieval, classification, and ranking methodologies based on the drawing styles of illustrative images. The intuitive and automatic conversions and transformations of style are also investigated for both raster and vector images.

An Unsupervised Approach for Comparing Styles of Illustrations, CBMI2015 [Best Paper Award]

Editing and Synthesis of Font Styles

Style-based design assistance of expressive fonts is developed, including picture fonts for representing human figures to visually annotate body movements.

Smart Lighting and Illumination

Smart restoration of color lighting scenes

An image synthesis technique is developed for authentically restoring the color of indoor scenes illuminated with color lighting systems. HDR images are smartly converted on the basis of the color appearance model of human vision system.

Restoration of color appearance by combining local adaptations for HDR images, AIC2015

Distributed autonomous lighting controls with color LED

Smart lighting methodologies are developed with sensor-attached digitally-controllable color LED unit. The smart and energy-saving dimming control is implemented by using an algorithm based on distributed adaptive optimization via mutual communication between the lighting units. The image-based monitoring and controls with mobile lighting robot is also developed.
WISISYG Light demo

Completed Projects

Visual Code & Optical Image Communications

Ubiquitous communications via optical spatio-temporal data processing

The pattern recognition technique for optical variations along space and time is explored with image sensors of commercial smartphones or handy mobile devices such as tablets or smart watches. Data communication systems with color LEDs are developed as an application of this optical pattern recognitions.
Virtual Gashapon demo

Data-Embeddable Texture Synthesis

SmartGraphics 2007

A method of synthesizing texture images was developed for embedding arbitrary data. It introduces the smart techniques of generating repetitive texture patterns through feature learning of a sample image. A synthesized image can effectively conceal the embedded pattern, and the pattern can be robustly detected from a photographed image of mobile devices such as smart phones and tablets.
Texture Synthesis for Mobile Data Communications, IEEE Computer Graphics and Applications
Symposium on Smart Graphics 2007
keyword: texture synthesis, code embedding, local binary pattern

Humanoid Animations

Pose-Timeline for Propagating Motion Edits

Symposium on Computer Animation 2009

A motion editing interface was developed for efficiently and flexibly editing the sequence of iterative actions by a few intuitive operations. It can visualize a motion sequence on a summary timeline with editable pose-icons, and drag-and-drop operations on the timeline enable intuitive controls of temporal properties of the motion such as timing, duration, and coordination.
Pose-Timeline for Propagating Motion Edits, Symposium on Computer Animation (SCA 2009)
keyword: pose-timeline, edit propagation, motion retiming, motion style transfer

Multilinear Motion Synthesis with Level of Detail Controls

Pacific Graphics 2007

A hybrid algorithm was developed to optimize the reduction size and computational time, according to the distance from the camera while maintaining visual quality. It can provide a practical tool for creating an interactive animation of many characters while ensuring accurate and flexible controls at a modest level of computational cost.
Multilinear motion synthesis with level-of-detail controls, Pacific Graphics 2007
keyword: motion interpolation, multilinear analysis, level-of-detail control

Geostatistical Motion Interpolation


Motion interpolations were developed with statistical predictions of missing data in an arbitrarily definable parametric space. A practical technique of geostatistics, called universal kriging, was introduced for statistically estimating the correlations between the dissimilarity of motions and the distance in the parametric space. It can statistically optimize interpolation kernels for given parameters at each frame, using a pose distance metric to efficiently analyze the correlation.
Geostatistical Motion Interpolation, ACM Transactions on Graphics (SIGGRAPH 2005)
SIGGRAPH demo movie
keyword: motion interpolation, geostatistics, kriging, variogram

Autonomous, Explorative Motion Generation

IEICE 2004

Explorative synthesis of human motions was developed using hierarchical re-inforcemant learning for searching plausible poses with end-effector's constraints.

Extensive and Efficient Search of Human Movements with Hierarchical Reinforcement Learning, Computer Animation 2002 (CA 2002)

keyword: keyframe animation, hierarchical reinforcement learning

Neural Gait Generator

IEICE 2004

Coupled neural cell models was introduced for generating oscillation signals to synthesize various gait motions that are self-stabilizable for dynamical environment.

Physiological Gaits Controls with a Neural Pattern Generator, The Journal of Visualization and Computer Animation

keyword: neural oscillator, central pattern generator (CPG)

Retrieval of Motion Capture Data

Smart Manipulations of Massive Motion Data

SCA 2004

Self-organizing map was introduced to visualize massive motion capture data while interactively retrieving motion clips with this map.

Motion Map: Image-based Retrieval and Segmentation of Motion Data, Symposium on Computer Animation (SCA2004)

keyword: motion clip retrieval, self organizing map (SOM)

Generating Concise Rules for Retrieving Human Motions from Large Datasets

Computer Animation and Social Agents 2009

A method for retrieving human motion data was developed with concise retrieval rules based on the spatio-temporal features of motion appearance. It converts motion clip into a form of clausal language that represents geometrical relations between body parts and their temporal relationship. A retrieval rule is then learned from the set of manually classified examples using inductive logic programming.
Generating Concise Rules for Retrieving Human Motions from Large Datasets, Computer Animation and Social Agents (CASA2009)
keyword: motion retrieval, inductive logic programming, appearance feature

Behavior Simulations

Psychology-based Crowd Simulation

SCA 2004

A gait simulation was developed based on psychological model of personal space and group intelligence for very crowded environment.

Psychological Model for Animating Crowded, Computer Animation and Virtual Worlds (Special issue CASA 2005)

keyword: crowded pedestrian, psychological model, personal space, virtual memory, locomotion graph

Web-based Behavior Simulation

SIGGRAPH 2003 Web Graphics

A Web-based system for simulating human behaviors was developed with motion capture data and XML-based scenario of behaviors. Java-based middleware was developed with the financial support by Mitou software project of IPA.

Extensible Task Simulation with Motion Archive, Transactions on Information and Systems of IEICE
Extensible Behavior Simulation with Motion Archive, SIGGRAPH2003 Web Graphics

keyword: web3D, xml, simulation middleware