StylePortraitVideo: Editing Portrait Videos with Expression Optimization

Figure 4. Reconstruction of input videos and their edited reults.
StyleGAN is a pre-trained StyleGAN model with FFHQ, and StyleGAN_va is a video adpated StyleGAN with the given input video. pSp and e4e are image encoders. Show video1, video2, video3, video4, video5, video6, video7, and video8.

Figure 5. Motion blurred videos. Show video1, video2, and video3

Figure 6. Comparison to optimization variations.
Opt. runs optimization per frame to find the corresponding latent code. R + Opt. recurrently use the previous frame's latent code for initilization and optimize to find the latent code for the current frame. Show video1 and video2.

Figure 8. Qualitative results on expression editing.
Videos with Exp. Dyn. Optim. better follow the original input video's lip contact than those without Exp. Dyn. Optim. The eyes in the other hand follow the intented edited results. Note that for all expression directions, the mouth is opening, and our method tries to preserve the original expression dynamics such as the lip contact to follow the original motion while maintaining the edited expression results. Show video1, video2, video3, video4, video5, video6, video7, video8, and video9.

Figure 9. Sequential editing results.
Expression dynamics optimziation can be perfomred after sequential editing for both eye and eyebrow shapes. Show video1 and video2.

Figure 11. Frame interpolation. Show video1 and video2.

Figure 12 (a). Limitations and capacity of StyleGAN3.
Texture sticking in the skin and eyebrows are effectively solved by using StyleGAN3. Show video1, video2, and video2.