ISPRS Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and Meshes
This benchmark is supported by 2021 ISPRS scientific initiatives project.
Automated extraction of geographic objects from airborne data is an important research topic in photogrammetry and remote sensing since decades. In addition to images, 3D point clouds from airborne LiDAR and Multi-View-Stereo-Image-Matching became more and more important as basic data source. The aim of H3D is to provide state-of-the-art data sets to the community, which can be used by interested researchers to test own methods and algorithms on semantic segmentation for geospatial applications. We propose a benchmark consisting of highly dense LiDAR point clouds captured at three different epochs. The respective point clouds are manually labeled into at least 11 classes and are used to derive labeled textured 3D meshes as an alternative representation. Core features of H3D are:
- UAV-based simultaneous data collection of both LiDAR data and imagery from the same platform
- High density LiDAR data of 800 points/m² enriched by RGB colors of on board cameras incorporating a GSD of 2-3 cm → H3D(PC)
- High resolution 3D textured mesh data generated from both LiDAR data and imagery in an hybrid manner → H3D(Mesh)
- Manually set labels for the LiDAR point cloud, which are automatically transferred to the 3D mesh
- Multi-temporal data set available for 3 different epochs (March 2018, November 2018, and March 2019) captured over the same area with the same sensor configuration (for now only the March 2018 dataset is online, the others will follow in 2021)
More information you will find here »
ISPRS Benchmark on Object Detection in High-Resolution Satellite Images
This benchmark is supported by 2021 ISPRS scientific initiatives project.
Project Goals
This ISPRS benchmark provides an effective way for the evaluation and comparison of object detection and recognition in high-resolution satellite images. Datasets are available from this webpage and the mirror website （http://gaofen-challenge.com/）. Interested participants can test their methods and submit their results for evaluation. The list of submitted evaluation results will be updated on the mirror website.
Activities and Benchmark Datasets
This benchmark provides a large-scale dataset and an evaluation submitting system for applying advanced deep learning technology to remote sensing. Images in the benchmark are mainly collected from the Gaofen satellites. There are more than 1 million instances and more than 15,000 images in this benchmark. As shown in Figure 1, all objects in the dataset are annotated with respect to 5 categories and 37 sub-categories by oriented bounding boxes. Each image is of the size in the range from 1000 × 1000 to 10,000 × 10,000 pixels and contains objects exhibiting a wide variety of scales, orientations, and shapes.
We provide raw data of training set with ground truth for users’ evaluation. We also provide raw data of test set for evaluation by submitting. The evaluation metrics and the format for submitting results can be seen on the mirror website （http://gaofen-challenge.com/）.
More information you will find here »
ISPRS Benchmark on UAVid: A semantic segmentation dataset for UAV imagery
Semantic segmentation has been one of the leading research interests in photogrammetry and computer vision in recent years. The fast development of semantic segmentation attributes enormously to the large scale datasets, especially for the deep learning related methods. We introduce our UAVid dataset, a new high-resolution UAV semantic segmentation dataset, which brings new challenges, including large scale variation, moving object recognition and temporal consistency preservation. UAVid dataset is a high-resolution UAV semantic segmentation dataset focusing on street scenes.
The dataset consists of 42 video sequences (seq1 to seq42), which are captured with 4K high-resolution in oblique views.
In total, 420 images have been densely labeled with 8 classes for the semantic labeling task.
More information you will find here »
ISPRS / EuroSDR Benchmark for Multi-Platform Photogrammetry
The aim of the project is to assess the accuracy and reliability of current methods for calibration and orientation of images acquired by different platforms as well as their integration for image matching and dense point cloud generation. In current research especially the question on how large changes of perspective and scale differences need to be tackled in image orientation and (dense) image matching is not approached systematically.
By providing a new benchmark dataset consisting of state-of-the-art sensor data and covering different relevant tasks and scenarios the current status of research is identified and further works will be stimulated.
The DENSE IMAGE MATCHING (DIM) benchmark is performed in cooperation with the EuroSDR’s Scientific Initiative “Benchmark on High Density Image Matching for DSM Computation”, joining both the two available datasets on Dortmund (Germany) and Zurich (Switzerland). The efforts of both the initiatives are contributing to set up a more complete and challenging dataset, considering flights with different features in terms of GSD size and overlap.
More information you will find
here »
ISPRS Test Project on Urban Classification, 3D Building Reconstruction and Semantic Labeling
ISPRS WG II/4 is running following bechmarks:
- Urban classification and 3D reconstruction (Vaihingen/Germany and Toronto/Canada)
- 2D Semantic Labeling (Vaihingen/Germany and Potsdam/Germany)
- 3D Semantic Labeling (Vaihingen/Germany)
More information you will find here »
Benchmark on High Density Aerial Image Matching
Background and Scope of the project
Innovations in matching algorithms as well as the increasing quality of digital airborne cameras considerably improved the quality of elevation data generated automatically from aerial images. This development motivated the launch of the joint ISPRS/EuroSDR project “Benchmark on High Density Aerial Image Matching” aiming at evaluating the potential of photogrammetric 3D data capture in view of the ongoing developments of software for automatic image matching. Basic scope is the evaluation of 3D point clouds and DSM produced from aerial images with different software systems. Such a comparative evaluation provides a platform for software developers to demonstrate the state-of-the-art of their ongoing developments. Furthermore, it can help potential users like National Mapping and Cadastral agencies (NMCAs), which consider a state-wide-generation of high quality DSMs to understand the applicability of such tools while triggering further developments based on their needs.
As a joint test data set subsets of three aerial image blocks are provided.Two data sets cover nadir imagery, which are captured at different landuse and block geometry, while the third data set includes oblique aerial images.
More information you will find here »
Benchmark on Indoor Modeling
Up-to-date spatial models of indoor environments are needed in a growing number of applications, including navigation, emergency response and a range of location-based services. Automated generation of 3D indoor models from point cloud data has been a topic of intensive research in recent years. While results on various datasets have been reported in the literature, a comparison of the performance of different methods has not been possible due to the lack of benchmark datasets and a common evaluation framework.
The ISPRS Benchmark on Indoor Modelling aims to address this issue by providing a public benchmark dataset and an evaluation framework for performance comparison of indoor modelling methods.
More information you will find here »