LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Narratives
Ecosystems
Launchpads

Discover

Search
Sources

Python

RepDistiller

[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods

PythonEmerging

Stars

—

Forks

—

Contributors

4

Last push

9mo ago

Recent commits

Latest commits.

Add poolformerv2, and fix features of convnextv2
28149dcRobin Dong9mo ago
Add missing loss function for AFD
ad9eb8dRobin Dong10mo ago
Reach 81.8% for wrn_40_2->wrn_40_2 distill
8c5149dRobin Dong10mo ago
The missing code for mixup. (Adding mixup for train_student.py seems only improve KD but bad for NST/CRD)
46bbe5fRobin Dong10mo ago
ConvNextV2 should use smaller kernel_size/stride for 32x32 CIFAR100 dataset (reach 73% for teacher)
07f2c72Robin Dong10mo ago

Mixup could at least raise teacher accuracy from about 75% to 83% (resnet32x4)

5ad8222Robin Dong10mo ago

Use 8 repeat for teacher training; 4 repeat for student training

f6d6d9cRobin Dong10mo ago

Add ConvNeXtV2 models

5bbb870Robin Dong10mo ago

Top contributors

Builders behind this project.