Video Inpainting
Diffusion-based Video Inpainting
The goal of this project, which is the PhD work of Nicolas Cherel, in collaboration between Télécom Paris (Yann Gousseau, Alasdair Newson) and Université Paris Cité (Andrés Almansa), is to use diffusion models for video inpainting. It concerns the following publications:
- Infusion: Internal Diffusion for Video Inpainting, N. Cherel, A. Almansa, Y. Gousseau, A. Newson, 2024 — Preprint · Project · Code
- Diffusion-Based Image Inpainting with Internal Learning, N. Cherel, A. Almansa, Y. Gousseau, A. Newson, EUSIPCO 2024 — Paper
For more information, see the webpage of Nicolas Cherel.
Patch-based Video Inpainting
This was part of my PhD work, in collaboration between Technicolor (Matthieu Fradet, Patrick Pérez) and Télécom ParisTech (Andrés Almansa, Yann Gousseau). The goal is to use a patch-based approach to video inpainting. Here is an example of such an inpainting:
Abstract
We propose an automatic video inpainting algorithm which relies on the optimisation of a global, patch-based functional. Our algorithm is able to deal with a variety of challenging situations which naturally arise in video inpainting, such as the correct reconstruction of dynamic textures, multiple moving objects and moving background. Furthermore, we achieve this in an order of magnitude less execution time with respect to the state-of-the-art. We are also able to achieve good quality results on high definition videos. Finally, we provide specific algorithmic details to make implementation of our algorithm as easy as possible. The resulting algorithm requires no segmentation or manual input other than the definition of the inpainting mask, and can deal with a wider variety of situations than is handled by previous work.
Original project webpage: Project webpage
Citing this work
If you wish to use our work or code, please cite the following paper:
Video Inpainting of Complex Scenes
Alasdair Newson, Andrés Almansa, Matthieu Fradet, Yann Gousseau, Patrick Pérez
SIAM Journal on Imaging Sciences 2014 7:4, 1993–2019
Video inpainting examples
Input video — Fontaine, Châtelet
Input video:
Our inpainting result (slowed down by a factor of two for visualisation):
Input video — Les Loulous
Input video:
Our inpainting result (slowed down by a factor of two for visualisation):
Input video — Young Jaws
Input video:
Our result (slowed down by a factor of two for visualisation):
Input video — Museum
This example is from the work of Miguel Granados et al.: How not to be seen: Object removal from videos of crowded scenes, M. Granados, K. Kim, J. Tompkin, O. Grau, J. Kautz and C. Theobalt, Computer Graphics Forum (EUROGRAPHICS), 2012.
To download the input and masks of this work, see How Not to be Seen — Miguel Granados.
Comparison with Granados et al. (their result above, ours below):
Input video — Duo
This example is also from Granados et al. (EUROGRAPHICS 2012). Comparison (their result above, ours below):