Here you get information about the benchmark organized by WG III/4 (2012-2016) and WG II/4 (2016-2022).
NEW 2018 - Full reference data available
As of 2018 all reference data for all benchmarks we are running is available for download - end of the benchmark challenge!
This means that we are stopping evaluating and publishing new results on our website. Instead, participants are encouraged to do an evaluation using the now available independent reference data on their own. A comparison to former results is still possible of course and appreciated. Files with the full reference data are contained in the respective folders on the download site. The naming of files should be clear enough. In order to facilitate a comparison to existing results there are still the old files with reference labels "...for_participants", i.e. a part of the full reference and in addition now the complete reference data.
In addition there is the code we have been using to evaluate the semantic labeling results. The C-code is linked to several external libraries. Please note that we cannot provide support for the software due to restricted resources.
The data, including all reference data, can be downloaded directly from a seafile server. For the formal issues (terms of use), see this doc.
The links to the seafile server are below, the passwords for all download links is
Potsdam: https://seafile.projekt.uni-hannover.de/f/429be50cc79d423ab6c4/ size: 13.3GB
Toronto: https://seafile.projekt.uni-hannover.de/f/fc62f9c20a8c4a34aea1/ size: 3.4GB
Vaihingen: https://seafile.projekt.uni-hannover.de/f/6a06a837b1f349cfa749/ size: 16.0 GB
IMPORTANT technical note: The files are very huge, so it is advised to download one after the other.