ESA is challenging ‘citizen scientists’ to apply the latest AI and image processing techniques to upscale images acquired by the Earth-watching Proba-V minisatellite.
Upscaling is a familiar concept from modern HD or 4K TVs, which perform image processing to enhance the appearance of low-resolution, standard definition footage when played on higher-resolution screens.
That is purely cosmetic processing, however; what this new ‘super-resolution’ competition is interested in is enhancing the spatial resolution of Earth observation images. This means you can see smaller objects more clearly by combining multiple lower-resolution images together to extract extra information on detailed features of the target site.
“Proba-V is a very interesting mission because it provides two types of imagery to researchers,” explains Dario Izzo of ESA’s Advanced Concepts Team (ACT), overseeing the competition. “The wide-swath main Vegetation instrument images 90% of the vegetated surface of Earth every day, at 300 m spatial resolution. It also acquires a limited amount of imagery at 100 m resolution.
“So the challenge we’re setting is to take nine or more of these 300 m resolution images of a given view then combine them together using super-resolution techniques to enhance the quality beyond the capability of the original camera, to try and come as close as possible to the equivalent 100 m resolution view.”
The competition was planned with ESA’s Proba-V system engineer Jorg Versluys: “Proba-V’s Vegetation instrument is composed of three telescopes side by side, providing a 100 m resolution in its central field of view, declining to 300 m resolution at the sides – this a unique situation where different resolution images are available of a given scene on the ground using the same instrument.
“These images are also geo-referenced to a sub-pixel level accuracy using startrackers and GPS to precisely fix Proba-V’s orbital position, along with additional fine-tuning performed with known control points on the ground.”
The ACT has gathered a dataset of more than 1 460 Proba-V images covering 74 selected locations around the globe.
“We don’t include 100 m resolution images for 20% of the dataset,” adds ACT fellow Marcus Märtens. “Instead the idea is for competitors to learn from the data how to reconstruct them from multiple 300 m resolution images, plus the 80% of 100 m resolution images that can be employed as a benchmark.
“We’re interested in seeing how traditional processing algorithms compare in performance to modern machine learning, where neural networks can be taught how to interpret the combined images.”
Super-resolution has found various terrestrial applications, such as enhancing the resolution of security video footage to facilitate facial recognition software or medical imaging at hospitals. For Proba-V, super-resolution could help fill in the gaps between 100 m resolution imagery, responding to increasing interest from users in global 100 m resolution coverage.
“For our competition we’ve selected areas that are mostly cloud free, but some pixels will still be lost to cloud cover,” adds Marcus. “The teams will also have to contend with different illumination and atmospheric conditions, as well as shifts in vegetation or the built environment on the ground across a timescale of up to a month between images.”
This is the latest competition hosted at the ACT’s Kelvins website, named after the temperature unit of measurement – with the idea that competitors should aim to reach the lowest possible error, as close as possible to absolute zero. The Proba-V dataset will be available there from Monday 15 October, with the six-month competition starting on 1 November.
