- Bluemo
@blu3mo/ Shutaro Aoyama - Columbia University, Junior
- Majoring in Computer Science and Philosophy
I am excited to collaborate with all of you over the next three months!
Yesterday, I managed to rent a car successfully. Although I am not very skilled at driving, the rental agreement stated, “We don’t rent to beginners,” so I acted confidently.
I will be discussing:
- Specific development and research projects
- A narrative connecting them together I hope you find it engaging.
2020-21
- Kineto - A system that merges real-time and on-demand online lectures
- 2020 IPA Exploratory IT Human Resources Project (Inami PM)
- “Real-time” advantage: Student communication
- “On-demand” advantage: Speed optimization through time control
- Our system:
- Comments are saved with the corresponding video timing
- Similar to Nico Nico Douga
- Key feature: “Elastic synchronization” to adjust time discreetly
2022-23
- Research on asynchronous VR teleoperation
- Unlike traditional VR teleoperation, “the robot’s movements are not synchronized with the user’s body movements.”
- Users provide high-level commands like “move this object here” in VR, and the robot autonomously executes the task efficiently.
- Essentially, the robot has increased autonomy.
- Asynchronous in terms of time as well.
- Commands are queued and executed sequentially.
- https://www.youtube.com/watch?v=cChHxkaFB3w
-
Aoyama, S., Liu, J.-S., Wang, P., Jain, S., Wang, X., Xu, J., Song, S., Tversky, B., & Feiner, S. (2024). Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation. 2024 IEEE VR
Special Relativity VR
I developed “Special Relativity in VR” during a university hackathon When moving close to the speed of light, the world seems to contract, and light wavelengths appear to shift. I designed a shader to simulate these effects in VR. It was an exciting experience with potential educational applications. Side note: I created this as a break from reality just before my final exam in a relativity course.
-
My brilliant idea: create a multiplayer VR special relativity simulator at this weekend’s hackathon to study for next week’s physics exam simultaneously.
- Let’s not talk about how the exam went.
- GitHub: https://github.com/Godlysk/SRVR
Livestreaming to My Future Self
“Livestreaming to my future self” is an intriguing and practical concept
- Displays your activities from 10 minutes ago on the bottom right of the screen
- Allows reflection like “past me was focused” or “past me was distracted surfing the web”
- Creates a sense of being observed by someone, but that someone is “future me”
- “Observing your past self as a stranger”
- “Being observed by your future self as a stranger”
- Q: How much time needs to pass for your past/future self to feel like another person?
These are: Eliminating the constraints of universally-shared time
- In the physical world, we assume everyone experiences time uniformly.
- 10:23 AM for me is 10:23 AM for you.
- In computer-mediated environments, this assumption can be challenged.
- Once this assumption is removed, various time structures become feasible:
- Sharing time with individuals from the past —> Kineto (online lectures)
- Sharing time with your past self —> Livestreaming to my future self
- Relativistic time and space structures —> Special Relativity VR
- Asynchronous reality sharing with robots —> Teleoperation
2024-
In computer-mediated environments, we can challenge the notion of universally-shared time.
- This concept can likely extend beyond just “time.”
- In computer-mediated communication and computer-mediated reality, we can challenge the idea of universally-shared reality.
- “My reality differs from your reality.”
Asymmetric Chat
- https://asym-chat.vercel.app
- This platform features two users with individual chat interfaces. They exchange messages, but each person’s message is transformed by GPT-4 into a different topic before reaching the recipient.
- For instance, one person may think they are discussing coffee, while the other perceives it as a conversation about architecture.
Asymmetric Editor
I developed a system for editing documents across language barriers. If Google Docs incorporated this feature, users could collaborate on documents in different languages. For example, during Zoom meetings for English and Japanese notes, participants could edit the document in their native language.
Intersubjective Model of AI-mediated communication
(CHI’25, under review)
AI-mediated Communication applied in Digital Democracy
- Tokyo Governor Election
- I was part of Takahiro Anno’s team
-
Takahiro Anno: 154,638 Votes
-
- AI Anno is available to answer your questions 24 hours a day.
- Watch the video here
- This system goes beyond a simple question and answer platform. It serves as an AI-facilitated communication channel between politicians and citizens.
- Politicians can both broadcast their messages and actively listen to feedback.
- Rather than direct interaction, the AI acts as an intermediary in these exchanges.
How should we shape the “reality” that is shared through computers?
-
Fully “subjective”: Examples include single-player games and the concept of a brain in a vat.
- Each individual experiences a unique reality.
- Drawing parallels to “The Matrix,” where individuals have individually tailored simulated worlds.
-
Completely “objective”: Referring to traditional reality, a concept known as Cluster, and the blue pill option in “The Matrix.”
- Everyone perceives and shares the same objective reality.
- Similar to the blue pill scenario in “The Matrix” where a single reality is accepted as truth.
-
A middle ground can be found:
- Referred to as “Asymmetric Reality.”
- Involves sharing a partially objective reality while allowing for individual optimization as necessary.
- Examples include the individual optimization of “time” (Kineto, etc.), “meaning” (asymmetric chat, Intersubjective Model of AIMC), and “language” (asymmetric editor).
-
Exploring ways to enhance self-subjectivity.
- Introduction of self-synth.