Marcos V. Conde

I'm a PhD Researcher in Artificial Intelligence at the University of Würzburg, advised by Prof. Radu Timofte. My work is supported by Sony PlayStation (FTG) where I'm a Computer Vision Research Scientist.
I am also Senior Data Scientist (Kaggle Grandmasters Team) at H2O.ai.

From 2020 to 2022, I worked at Huawei Noah’s Ark Lab (London) supervised by Dr. Eduardo Pérez-Pellitero and Prof. Aleš Leonardis, and received the best intern award for my work on neural camera ISPs.

M.Sc. in Computer Vision from the Autonomous University of Barcelona (UAB) with honours for my work on Real-time Photography Enhancement.

Email  /  Google Scholar  /  GitHub  /  Kaggle  /  Hugging Face 🤗

井の中の蛙大海を知らず  |  井鼃不可以語於海者，拘於虛也
"I learned very early the difference between knowing the name of something and knowing something." - Richard Feynmann

News

[Jan 2024] New paper InstructIR: High-Quality Image Restoration Following Human Instructions is trending on HF 🤗. Read the arXiv preprint | Watch the demo Video | Our demo is available Try it now! (click)

[Jan 2024] We organize the 1st AI for Streaming Workshop at CVPR 2024. Check the awesome speakers and challenges in the website. The workshop is sponsored by Sony PlayStation, Meta and Netflix.

[Jan 2024] I co-organize the New Trends in Image Restoration and Enhancement (NTIRE) Workshop at CVPR 2024.
I organize the RAW SR, Burst ISP and Portrait IQA Challenges.

Research

My current research interests include neural networks, deep learning, low-level computer vision, inverse problems, computational photography and photorealism. How can we improve/change/enhance cameras using AI?
Representative papers are highlighted.

(Past/Present) Collaborators: Radu Timofte  /  Michael S. Brown  /  Tom E. Bishop  /  Javier Vazquez-Corral

InstructIR takes as input a degraded image and a human-written instruction for how to improve that image. The (single) neural model performs all-in-one image restoration. We achieve state-of-the-art results on several restoration tasks including image denoising, deraining, deblurring, dehazing, and (low-light) image enhancement.

NILUTs are neural representations of real 3D LUTs for controllable photo-realistic image enhancement and color manipulation. Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly.

We advance RAW sensor images up-scaling (Super-Resolution). We explore diverse image degradations (e.g. Noise, Blur) to emulate a low-resolution RAW image, and we train a neural network to upsample it.

EBokehNet, an efficient state-of-the-art solution for Bokeh effect transformation and rendering. We can render Bokeh from all-in-focus images, or transform the Bokeh of one lens to the effect of another lens without harming the sharp foreground in the image.

The paper presents an exhaustive study of baseline methods for real-time SR. "Efficient Deep Models for Real-Time 4K Image Super-Resolution. NTIRE 2023 Benchmark and Report" gauges the state-of-the-art methods for real-time 4K upscaling using our new dataset and benchmark protocol. The methods allow 60 FPS and even 120 FPS.

We propose LPIENet, a lightweight network for perceptual image enhancement, with the focus on deploying it on smartphones. The model was tested for image denoising, deblurring, and HDR correction.

Super-resolution of compressed images using transformers. We use the Swin Transformer V2, to improve SwinIR for image super-resolution, and in particular, the compressed input scenario. Using this method we can tackle the major issues in training transformer vision models, such as training instability, resolution gaps between pre-training and fine-tuning.

This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation.

Hybrid model-based and data-driven approach for modelling ISPs using learnable dictionaries. We explore RAW image reconstruction and improve downstream tasks like RAW Image Denoising via raw data augmentation-synthesis.

We were one of the 1st attempts to use CLIP (Contrastive Language-Image Pre-Training) for training a neural network on a variety of art images and descriptions, being able to learn directly from raw descriptions about images, or if available, curated labels.

Academic Service

Reviewer: CVPR 2022/2023, ECCV 2022/2024, ICCV 2023, AAAI 2023, SIGGRAPH 2024, ACCV 2024, IEEE Transactions on Image Processing, IEEE Transactions on Computational Imaging

Teaching: Image Processing and Computational Photography (IPCP), Computer Vision

Workshops: AI4Streaming CVPR  /  NTIRE 2024 CVPR  /  NTIRE 2023 CVPR  /  AIM 2022 ECCV

Other Projects

We introduce a complete open-source ecosystem for developing and testing LLMs. The goal of this project is to boost open alterna- tives to closed-source approaches.
Authors: Arno Candel, Jon McKinney, Philipp Singer, Pascal Pfeiffer, Maximilian Jeblick, Chun Ming Lee, Marcos V. Conde.

I was invited by Japan’s National Institute of Informatics (NII) and ROIS-DS Center for Open Data in the Humanities (CODH) to present a novel solution for the Kuzushiji Recognition Challenge at the Japanese Culture and AI Symposium 2019 in Tokyo.
Hosts: Dr. Asanobu Kitamoto, Tarin Clanuwat, Alex Lamb.

Design and source code from Jon Barron's website.