https://dl.acm.org/citation.cfm?id=2735828
- One of the methods for learning languages like English is shadowing. It is a practice where you follow the teacher’s spoken words and try to reproduce them yourself.
- In addition to pronunciation, there is also a practice of imitating the teacher’s facial expressions and movements of the muscles in the face.
- Research on a system that supports this.
- It is used in an environment connected by online video chat.
- The teacher’s facial expressions are tracked using ofxCV and the user’s face is fitted onto the obtained 3D mesh.
- This allows the user’s face to synchronize with the movements of the teacher’s face, enabling them to be more conscious of pronunciation.
- Amazon MTurk was used for user studies.
- A service where you can outsource tasks that require human labor online.
- Two user studies were conducted: ① first using this method, then using a conventional method, and ② in the reverse order.
- As a result, improvement in pronunciation was observed, but there was no significant impact on speaking speed.
- The application of advanced movements to oneself using Image Processing was named Futuroid.
- This research is the first in a series, and there are plans to try it in other fields such as soccer.
- Related research was mentioned at the end of the paper.
Thoughts
- It would be interesting to try the same thing with voice using a voice changer.
- I wonder if it would feel weird with my own voice, there might be some psychological research on that.
- I’m curious if there is any discomfort in not being able to control one’s own face.
- It was mentioned in the related research.