Notes on Immersive Technology and Plurality (WIP)

I translated Thoughts about Plurality and Immersive Technology (WIP) into Japanese in Claude (by blu3mo).

• 

• Many aspects of immersive technology overlap with Plurality.

• For example:

  • From low-bandwidth collaboration to multi-sensor/multi-modal shared experiences
    • This is currently a focal point in the Plurality Book
  • Transition from “one body per individual” to “extended bodies” (e.g., multiple people owning a single avatar), moving towards unconventional individual models
  • Shifting from an objective/singular reality to an inter-subjective/multiple realities
    • This is what this article aims to discuss

• Reality is subjective, and we construct a shared reality by inter-subjectively sharing our perceptions.

  • People’s worldviews are inherently subjective, and a truly objective reality does not exist.
    • This concept of multiple models of reality evokes ideas like Yukusyul’s ambient world or phenomenology, emphasizing how individuals perceive different things even when in the same world.
  • In essence, reality is asymmetric.
    • While experiencing different realities, we can still share a reality.

• The appealing aspect of computer-mediated realities (VR, MR, etc.) is the ability to intentionally design this “asymmetry” mechanism.

  • By leveraging technology, personalized experiences can be created while maintaining a shared reality.

Specific examples of designed asymmetry include:

• Asymmetry in language: Participants in virtual meetings can listen to conversations in their preferred language while sharing the same audio space.
  • Real-time voice translation by Meta
• Asymmetry in non-verbal cues: Adapting gestures and non-verbal cues to each user’s cultural norms can enhance understanding and communication.
  • For example,
  • Real-time Gesture Translation in Intercultural Communication
• Asymmetry in space: Optimizing the layout and placement of virtual environments to align with each user’s physical space and devices ensures a comfortable and efficient experience.
  • For example,
  • Placement Retargeting of Virtual Avatars to Dissimilar Indoor Environments
  • Interaction motion retargeting to highly dissimilar furniture environment
  • Partially Blended Realities: Aligning Dissimilar Spaces for Distributed Mixed Reality Meetings
• Asymmetry in time: In asynchronous collaborations, team members in different time zones can check project updates and deadlines on their own timeline, leading to improved coordination and productivity.
  • For example,
  • Causality-preserving Asynchronous Reality
  • Challenges for Asynchronous Collaboration in Augmented Reality
• Contextual detail levels: Tailoring the amount and complexity of presented information to each user’s needs and preferences provides a personalized learning and working experience.
  • For example,
  • The Two-User Responsive Workbench: Support for Collaboration Through Individual Views of a Shared Space

Part of my work:

• 
• Asymmetry in space and time: Asynchronous remote teleoperation
  • Shifting time and space to suit the user when presenting the environment where the robot is placed.
  • Paper: Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation
• Asymmetry in time: Pseudo-synchronous and adaptive synchronous in asynchronous online lectures
  • Providing lecture videos at an optimized pace for each student while enabling communication between students across time.
  • Details: kineto
• Asymmetric chat: LLM-mediated chat environment with context conversion
  • https://asym-chat.vercel.app/When designing asymmetry for personalization, there is a potential for a more diverse experience, but it raises questions such as:
• How can we design asymmetry to promote collaboration beyond differences, avoiding a dystopian filter bubble where everyone lives in their isolated reality?
• Asymmetrical personalization could potentially lead to a dystopian world where everyone lives in their own reality without cooperation or coexistence.
• How can this be avoided?

The key lies in creating asymmetry in one aspect of reality while enabling sharing in other aspects.

• In other words, making one thing asymmetrical to allow sharing of other things.
• For example:
  • Real-time language translation allows users to hear audio in their preferred language, creating asymmetry in the audio space but enabling sharing of conversation meanings and content.
  • Detailed personalization in presentations provides each participant with the right amount of information, creating asymmetry in the depth of content but allowing sharing of the overall message and context.
  • Culturally adapted gestures and non-verbal cues create asymmetry in visual expressions but enable conveying underlying emotions and intentions.

Another approach is to recognize and accept the coexistence of multiple realities simultaneously.

• While traditional reality is often seen as singular, immersive technology can overlay multiple realities in one space.
• This allows users to experience the world from both their own and others’ perspectives, fostering empathy and understanding.
• By presenting both personalized reality and shared reality simultaneously, users can maintain a sense of connection and collaboration while understanding the diversity of experiences.

In summary, by carefully designing asymmetry in immersive reality, personalized experiences tailored to individual differences can be created while promoting collaboration and shared understanding. By leveraging asymmetry to enhance specific aspects of reality while maintaining a common foundation in other aspects and acknowledging the coexistence of multiple realities, the power of diversity can be harnessed to cultivate a comprehensive, empathetic, and collaborative virtual environment.