- Target
- Is it better to explain to someone like me who has a rough understanding of Neural Networks before starting the investigation?
- Structure
- Show it at the beginning and explain how it works.
- Break down the elements gradually.
- Since I have been researching and summarizing on Scrapbox, it seems good to incorporate that into the presentation.
- Flow: HyperNeRF -> NeRF -> Volume rendering
- HyperNeRF
- SIGGRAPH Asia’s 09 neural rendering.
- After listening to various technical papers in different fields at SIGGRAPH Asia, I became most interested in this research and delved deeper into it.
- Since I didn’t have much prior knowledge in this field, I will try to explain it in a way that can be understood by people without that background.
- Before explaining HyperNeRF, let me explain the non-hyper NeRF, simply called NeRF.
- What can it do?
- https://www.youtube.com/watch?time_continue=92&v=JuH79E8rdKc&feature=emb_logo
- In the previous approach, it was a neural network that outputs images.
- However, when trying to do something like NeRF, there was a problem of inconsistent images.
- When generating images of an object from the top and from the left, they each looked plausible but lacked consistency.
- NeRF solves this problem by changing the fundamental problem.
- It aims to train a neural network that can answer what color and opacity a certain point in space has when viewed from a certain angle.
- For example, when asked about the color and opacity of a specific point on a tea bottle when viewed from the side, the network should answer brown and semi-transparent.
- For the cap, it should answer green and completely opaque.
- For coordinates in empty space, it should answer colorless and transparent.
- In other words, it trains a neural network that can represent a specific three-dimensional scene with a set of colors and densities.
- This set of colors and densities is called a Radiance Field.
- Neural Radiance Field -> NeRF
- What can we do with this?
- Show a video.
- By discretizing it for easier calculations, it can be represented by a function that is differentiable.
- Then various things can be created
- Not good at discontinuous changes.
- HyperNeRF solves this problem.
- Evaluation
- For quantitative evaluation, they used these three evaluation criteria.
- In summary,
- This research is not the first of its kind, but as shown in the video, it can render shape changes like opening and closing the mouth without any problems.
- Furthermore, the remarkable point of this research is that it can render shape changes with topological changes without any problems.
- Impressions
- I should think about it in case I am asked.
- Impressions of HyperNeRF:
- When I read it to a level where I can explain it to others, I felt that there is a kind of genealogy for each topic, as I traced the background of HyperNeRF.
- Also, in 2020, many derivative studies including HyperNeRF have already been published.
- Impressions of SIGGRAPH:
- I feel like I got to see the overall picture of the field.