Parametric Classification for Generalized Category Discovery: A Baseline Study (ICCV 2023)
By
Xin Wen*,
Bingchen Zhao*, and
Xiaojuan Qi.
Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples.
Previous studies argued that parametric classifiers are prone to overfitting to seen categories, and endorsed using a non-parametric classifier formed with semi-supervised
However, in this study, we investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We demonstrate that two prediction biases exist: the classifier tends to predict seen classes more often, and produces an imbalanced distribution across seen and novel categories. Based on these findings, we propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers. We hope the investigation and proposed simple framework can serve as a strong baseline to facilitate future studies in this field.
pip install -r requirements.txt
Set paths to datasets and desired log directories in config.py
We use fine-grained benchmarks in this paper, including:
We also use generic object recognition datasets, including:
Train the model:
bash scripts/run_${DATASET_NAME}.sh
We found picking the model according to 'Old' class performance could lead to possible over-fitting, and since 'New' class labels on the held-out validation set should be assumed unavailable, we suggest not to perform model selection, and simply use the last-epoch model.
Our results:
| Source | Paper (3 runs) | Current Github (5 runs) | ||||
|---|---|---|---|---|---|---|
| Dataset | All | Old | New | All | Old | New |
| CIFAR10 | 97.1±0.0 | 95.1±0.1 | 98.1±0.1 | 97.0±0.1 | 93.9±0.1 | 98.5±0.1 |
| CIFAR100 | 80.1±0.9 | 81.2±0.4 | 77.8±2.0 | 79.8±0.6 | 81.1±0.5 | 77.4±2.5 |
| ImageNet-100 | 83.0±1.2 | 93.1±0.2 | 77.9±1.9 | 83.6±1.4 | 92.4±0.1 | 79.1±2.2 |
| ImageNet-1K | 57.1±0.1 | 77.3±0.1 | 46.9±0.2 | 57.0±0.4 | 77.1±0.1 | 46.9±0.5 |
| CUB | 60.3±0.1 | 65.6±0.9 | 57.7±0.4 | 61.5±0.5 | 65.7±0.5 | 59.4±0.8 |
| Stanford Cars | 53.8±2.2 | 71.9±1.7 | 45.0±2.4 | 53.4±1.6 | 71.5±1.6 | 44.6±1.7 |
| FGVC-Aircraft | 54.2±1.9 | 59.1±1.2 | 51.8±2.3 | 54.3±0.7 | 59.4±0.4 | 51.7±1.2 |
| Herbarium 19 | 44.0±0.4 | 58.0±0.4 | 36.4±0.8 | 44.2±0.2 | 57.6±0.6 | 37.0±0.4 |
If you find this repo useful for your research, please consider citing our paper:
@inproceedings{wen2023simgcd,
author = {Wen, Xin and Zhao, Bingchen and Qi, Xiaojuan},
title = {Parametric Classification for Generalized Category Discovery: A Baseline Study},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
year = {2023},
pages = {16590-16600}
}
The codebase is largely built on this repo: https://github.com/sgvaze/generalized-category-discovery.
This project is licensed under the MIT License - see the LICENSE file for details.