projects | Yate Ge

research

GenFaceUI

Jun 2025 – Sep 2025

GenFaceUI explores generative personalized facial expression interfaces for intelligent agents from a meta-design perspective. Built on the GPFEI framework, it helps designers create facial templates, apply semantic tags, define context-expression rules, and iteratively test run-time generation within coherent, rule-bounded design spaces. A qualitative study with 12 designers showed gains in controllability and consistency while also revealing the need for better visual structure and lightweight explanations; I led the project's system design, prototype development, and paper writing. An early version was presented as a poster at UIST 2025, and the full research was published at CHI 2026.

human-robot interaction human-agent interaction LLM generative UI meta-design

Jokeasy

Dec 2024 – Mar 2025

Jokeasy is a search-enabled human-AI collaboration tool for thematic joke writing. It integrates a dual-role LLM agent as both material scout and prototype writer, providing a visual canvas where retrieved web content becomes editable inspiration blocks within a multistage workflow. A qualitative study with 18 participants (hobbyists, professional comedians, and HCI/AI specialists) showed that the approach enriches ideation while preserving author agency. The interactive prototype was built as a Figma widget plugin for rapid testing. I led research design, system development, and paper writing as a TA for a graduate course on Human-AI Collaborative Intelligence (人机智能协同).

human-AI co-creation LLM visual programming

GenComUI

Jun 2024 – Oct 2024

GenComUI explores generative visual aids as a communication medium to support task-oriented human-robot interaction. Powered by LLMs, the system dynamically generates contextual visual aids (map annotations, path indicators, and animations) on the robot's screen to complement verbal task communication and facilitate end-user robot programming. A formative study of human visual communication practices informed the design, and a within-subjects user study (n=20) demonstrated that generative visual aids enhance communication quality by providing continuous visual feedback and supporting iterative task refinement. I led the project's system design, development, and paper writing.

human-robot interaction generative UI LLM

Cocobo

Sep 2023 – May 2024

Cocobo is an LLM-powered end-user programming system for service robot task customization. Drawing on the theory of multiple representations in end-user development, the system leverages LLMs' code generation, general knowledge, and knowledge representation capabilities to let non-technical users create and modify robot programs through natural language, while providing interactive flowcharts for visual verification and MagicDebug for node-level debugging. A user study with 16 participants validated that non-programmers can independently create and refine robot service programs. I led the system design, development, and paper writing. The work was published at IEEE VL/HCC, the flagship conference in visual languages and end-user development.

visual programming human-robot interaction LLM

Virtual Design Platform for Service Robot

Sep 2019 – Jun 2020

A simulation design platform for service robots, developed as my master's thesis project. The platform leverages virtual reality, mixed reality, and component-based prototyping to support three key phases of service robot interaction design: conceptual character design through virtual bodystorming, interface and flow design through a visual interaction prototype builder, and usability testing through mixed-reality user simulation. Built with Unity and tested with designers, the platform lowers the cost of robot prototyping and enables rapid iteration across the full design cycle. Part of this work was published at AHFE 2023.

VR human-robot interaction Unity

design work

智启新生 / Wisdom Awakens

2025

Opening performance of the 2025 World Design Cities Conference, fusing interactive lighting, embodied intelligence, and new media visuals. The show illustrates a progression from human creativity through machine assistance to algorithm generation and embodied perception. As a member of the robot performance development and execution team, I was responsible for robot motion choreography and development.

robotics art

Cyber Ring / 赛博回环

2021

Cyber Ring is an independent VR game inspired by the classic street ring-toss game. It reimagines the familiar carnival experience in a cyberpunk-styled virtual space, where players use VR controllers to toss glowing neon rings onto targets in an immersive environment. I independently designed and developed the entire project using Unity. The game was showcased at street pop-up events and at Maker Faire, attracting players of all ages.

VR game Unity

Hypercard

Mar 2019 – Jun 2019

Hypercard is a tangible programming education system that combines board games, augmented reality, and tangible programming language to teach computational thinking in a spatial, embodied way. Children place hexagonal programming cards on a table, the mobile app recognizes and configures each card, then renders interactive AR scenes on top. The system supports multiple input modalities (voice, face recognition, screen gestures, UI widgets) and seven card types (operating, user input, logic, variable, signal, object, AI). Built with Unity and Vuforia, prototypes demonstrate scenarios including voice-controlled lighting, multiplayer AR golf, facial expression weather control, and guided programming challenges.

AR visual programming

Visual Analytics for Service Robot Personalized Knowledge Graphs

Oct 2017 – Apr 2018

This project explores user-in-the-loop visual analytics for building and refining personalized knowledge graphs that support service robots in individualized environments. I participated in the visualization interface and interaction design, focusing on how to visually represent working knowledge graphs so that users can inspect potential errors, input domain knowledge, and iteratively refine the graph.

visualization interaction design human-robot interaction

AOI Planet

2017

AOI Planet is a motion-sensing game designed for overweight children and their parents. When playing, kids actually exercise while learning how to eat healthier. It is not only an indoor game, but outdoor activities are also recorded through a wearable device. The project includes a motion-sensing game built with Unity, a companion wearable device powered by Arduino 101 with IMU and machine learning modules for motion recognition, and a parent-child interaction system.

motion-sensing game Arduino

Ninja

2016

Ninja is a smart phone companion that transforms your phone from a tool into a friend. Inspired by the ninja's spirit of endurance and perseverance, the device uses the Pomodoro technique to help users stay focused while building an emotional bond through anthropomorphic expressions, eye contact via face tracking, and contactless gesture interaction. The phone becomes Ninja's face, displayed on a 3D-printed body with servo-driven movements.

interaction design iOS development