Bear Foundation Grant Application

Q3. What kind of creative activities do you engage in as a creator? Please describe in detail, including the reasons for your activities. Up to 500 characters §

There is no objective reality, as reality is constructed based on individual subjective perceptions. Particularly in the “shared reality” facilitated through computers (internet space, AR/VR space, time, language space, etc.), the asymmetry of each individual’s subjective perception can be freely designed.

I am involved in research, product development, and design theory concerning the design of “asymmetry in reality.”

1. Research & Product Development:

• Research on an asynchronous robot remote control VR interface that transcends time and space for human-robot interaction.
• Development of an AI-mediated chat environment where statements are translated into different meanings.
• Product development of an online lecture environment enabling real-time student communication in asynchronous lectures.

2. Building Design Theory: I aim to generalize development insights across various domains like Time, Space, Language, Geography, etc., to explore design theories that reconcile asymmetry and shared reality.

I find fulfillment in the curiosity of shaping the future through abstract contemplation on reality design and practical research and development.

Q4. Through the activities and works mentioned above, what “new values” are you presenting or exploring? Up to 300 characters §

While individual optimization in reality, like noise cancellation in cities with AirPods or displaying only desired content on social media, is advancing, pursuing this optimization blindly can lead to filter bubbles and dystopias like experience machines. I aim to expand the design space and possibilities through research and development, presenting the world with the coexistence of “individual optimization through asymmetry” and “sharing with others through symmetry.”

Q5. What activities would you like to undertake for a year with the support of the Kuma Foundation? Up to 300 characters §

May-August 2024:

• Research on collaborative work mechanisms in asynchronous multiple-person VR with asymmetrical spacetime (Columbia University).
• Evaluation of mutual understanding in AI-mediated chat environments and research on design theory (Tokyo University).

September-December:

• Take a leave of absence for the fall semester and conduct joint research on subjective self with Sony CSL’s Kasahara at OIST.

January-April 2025:

• Return to Columbia for continued studies in information science and philosophy.

Other activities:

• Development of visualization methods for history from multiple perspectives in VR with a group of Stanford students (planned grant application).
• Collaborative artwork with Kuma Foundation members.

I will shuttle between Japan and the US, pursuing a consistent theme while traversing fields through a blend of abstract design theory/philosophy and concrete technical research/product development.

Q6. Do you use AI tools in your creative activities? §

Q7. For those who selected “1 to 4” in Q6, if you use AI tools, please describe specific tools, their usage, and your perception of AI technology. Up to 300 characters §

1. Use of AI tools: Rapidly develop prototypes, introduce externalities from others’ play, and bring them to the market.## Project Development and Research Activities

I developed a prototype of Asymmetric Chat using tools like GH Copilot and Cursor within a few hours and shared it on social media. Various professionals with backgrounds in VR development, communication research, design, and scriptwriting experimented with different ways of using the prototype, allowing me to gain insights from different perspectives. This led me to collaborate on research as a visiting associate professor at the University of Tokyo.

③ Research and Development Related to AI: §

• Asymmetric Chat: Using LLM as a medium for human communication.
• Robot Control VR Interface: Developing path planning AI as a medium for spatiotemporal interaction between humans and robots.

Q1. Academic Achievements and Research Topics §

As an Egleston Scholar at Columbia University, I was recognized as being in the top 1% of applicants. At the CGUI Lab under Professor Steven Feiner, I researched interfaces that allow factory robots to be indirectly controlled by moving objects in VR spaces.

International Conference Publications:

• Aoyama, S. et al. “Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation” presented at IEEE VR, 2024. (Lead author, acceptance rate 28.7%)

International Conference Poster and Demo Presentations:

• Wang, P. et al. “Built to Order: A Virtual Reality Interface for Assigning High-Level Assembly Goals to Remote Robots” at ACM SUI, 2023.
• Aoyama, S. and Asano, K. “Shadow Play Using Ultrasound Levitated Props” at ACM UIST SIC, 2022.

Q2. Extracurricular Activities and Achievements §

• Stanford TreeHacks 2024: Won the Healthcare Division Award, Most Impactful Award, and BGB Award. Our team was the only one to demonstrate a demo at the awards ceremony.
• Code for Japan Civictech Challenge Cup U-22: Won the Grand Prize and PLAID Award in 2021.
• 1st XR Creation Award: Received the Kid’sPlates Award in 2021 for proposing a virtual world where “time” is transformed.
• Information Processing Society of Japan Award: Received the Young Researcher Encouragement Award in 2021.
• National Institute of Informatics “Information Master” Program: Received the Outstanding Student Award in 2021.
• Information Processing Society Contest for Junior and Senior High School Students: Won the Excellence Award in 2021.
• IPA Exploratory IT Human Resources Development Program: Selected in 2020, completed in 2021.
• MEXT Tobitate! Study Abroad JAPAN 5th High School Student Course: Passed in 2019.
• U-22 Programming Contest 2016: Received the Minister of Economy, Trade and Industry Award in October 2016.