DL Challenge 1 Round 1

Round 1 leaderboard

Note that this is NOT the final leaderboard. You may use the following information to improve your models.

Evaluated on test_public.npz (8,000 examples). Ranked by public accuracy, then accuracy at Es/N0 <= 2 dB, then fewer parameters; exact ties share a rank.

Leaderboard

rank submission status params public_acc public_acc_lowsnr
1 Santa_Claude.onnx OK 48472 0.819 0.5523
2 the_tuning_forks.onnx OK 68296 0.8133 0.5474
3 Team7.onnx OK 65960 0.8107 0.5381
4 TokenLimitReached.onnx OK 62840 0.8063 0.5368
5 Team_Transformer.onnx OK 74392 0.8014 0.5169
6 dark_knights.onnx OK 40760 0.7973 0.525
7 tensors.onnx OK 70760 0.7973 0.5132
8 Tensor_Titans.onnx OK 74664 0.7929 0.5193
9 QuadraMinds.onnx OK 74776 0.7913 0.4978
10 ML_Mavericks.onnx OK 47752 0.789 0.4961
11 Akatsuki.onnx OK 68232 0.7566 0.4567
12 Learners.onnx OK 47752 0.7521 0.4595
- Illegal_Activations.onnx FAIL      

Failed submissions (1)

Illegal_Activations.onnx

  • file is empty (0 bytes): the export or the upload failed

Public test set: accuracy per class (all teams)

Rank Team ook bpsk qpsk 16qam 2fsk 4fsk gmsk ofdm
1 Santa_Claude 0.929 0.812 0.658 0.701 0.878 0.882 0.885 0.807
2 the_tuning_forks 0.930 0.824 0.643 0.636 0.877 0.903 0.898 0.795
3 Team7 0.919 0.818 0.682 0.633 0.862 0.892 0.895 0.785
4 TokenLimitReached 0.922 0.817 0.640 0.620 0.877 0.887 0.897 0.790
5 Team_Transformer 0.908 0.791 0.652 0.615 0.863 0.888 0.896 0.798
6 dark_knights 0.920 0.810 0.643 0.603 0.860 0.888 0.898 0.756
7 tensors 0.905 0.815 0.625 0.636 0.857 0.885 0.876 0.779
8 Tensor_Titans 0.921 0.812 0.618 0.584 0.872 0.880 0.888 0.768
9 QuadraMinds 0.906 0.802 0.600 0.635 0.846 0.872 0.880 0.789
10 ML_Mavericks 0.918 0.801 0.617 0.623 0.841 0.872 0.867 0.773
11 Akatsuki 0.878 0.766 0.597 0.538 0.807 0.850 0.864 0.753
12 Learners 0.897 0.786 0.562 0.517 0.831 0.856 0.856 0.712

Public test set: accuracy per Es/N0 band (all teams)

Rank Team -4 to 0 dB 0 to 4 dB 4 to 8 dB 8 to 12 dB 12 to 16 dB
1 Santa_Claude 0.489 0.750 0.907 0.968 0.990
2 the_tuning_forks 0.483 0.737 0.909 0.961 0.985
3 Team7 0.473 0.737 0.897 0.970 0.985
4 TokenLimitReached 0.477 0.719 0.894 0.960 0.990
5 Team_Transformer 0.457 0.710 0.895 0.968 0.986
6 dark_knights 0.464 0.717 0.882 0.951 0.981
7 tensors 0.453 0.709 0.893 0.963 0.978
8 Tensor_Titans 0.464 0.705 0.873 0.951 0.978
9 QuadraMinds 0.444 0.687 0.888 0.960 0.988
10 ML_Mavericks 0.433 0.697 0.881 0.958 0.985
11 Akatsuki 0.393 0.654 0.854 0.932 0.959
12 Learners 0.406 0.646 0.846 0.924 0.947

Per-team breakdown

1. Santa_Claude

Public test set (8,000 examples)

  • Accuracy: 0.8190
  • Accuracy at Es/N0 <= 2 dB: 0.5523
  • Rank: 1 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.929 929 / 1,000
    bpsk 0.812 812 / 1,000
    qpsk 0.658 658 / 1,000
    16qam 0.701 701 / 1,000
    2fsk 0.878 878 / 1,000
    4fsk 0.882 882 / 1,000
    gmsk 0.885 885 / 1,000
    ofdm 0.807 807 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.489 802 / 1,641
    0 to 4 dB 0.750 1,191 / 1,589
    4 to 8 dB 0.907 1,412 / 1,557
    8 to 12 dB 0.968 1,532 / 1,582
    12 to 16 dB 0.990 1,615 / 1,631

2. the_tuning_forks

Public test set (8,000 examples)

  • Accuracy: 0.8133
  • Accuracy at Es/N0 <= 2 dB: 0.5474
  • Rank: 2 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.930 930 / 1,000
    bpsk 0.824 824 / 1,000
    qpsk 0.643 643 / 1,000
    16qam 0.636 636 / 1,000
    2fsk 0.877 877 / 1,000
    4fsk 0.903 903 / 1,000
    gmsk 0.898 898 / 1,000
    ofdm 0.795 795 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.483 793 / 1,641
    0 to 4 dB 0.737 1,171 / 1,589
    4 to 8 dB 0.909 1,416 / 1,557
    8 to 12 dB 0.961 1,520 / 1,582
    12 to 16 dB 0.985 1,606 / 1,631

3. Team7

Public test set (8,000 examples)

  • Accuracy: 0.8107
  • Accuracy at Es/N0 <= 2 dB: 0.5381
  • Rank: 3 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.919 919 / 1,000
    bpsk 0.818 818 / 1,000
    qpsk 0.682 682 / 1,000
    16qam 0.633 633 / 1,000
    2fsk 0.862 862 / 1,000
    4fsk 0.892 892 / 1,000
    gmsk 0.895 895 / 1,000
    ofdm 0.785 785 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.473 777 / 1,641
    0 to 4 dB 0.737 1,171 / 1,589
    4 to 8 dB 0.897 1,397 / 1,557
    8 to 12 dB 0.970 1,534 / 1,582
    12 to 16 dB 0.985 1,607 / 1,631

4. TokenLimitReached

Public test set (8,000 examples)

  • Accuracy: 0.8063
  • Accuracy at Es/N0 <= 2 dB: 0.5368
  • Rank: 4 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.922 922 / 1,000
    bpsk 0.817 817 / 1,000
    qpsk 0.640 640 / 1,000
    16qam 0.620 620 / 1,000
    2fsk 0.877 877 / 1,000
    4fsk 0.887 887 / 1,000
    gmsk 0.897 897 / 1,000
    ofdm 0.790 790 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.477 782 / 1,641
    0 to 4 dB 0.719 1,143 / 1,589
    4 to 8 dB 0.894 1,392 / 1,557
    8 to 12 dB 0.960 1,518 / 1,582
    12 to 16 dB 0.990 1,615 / 1,631

5. Team_Transformer

Public test set (8,000 examples)

  • Accuracy: 0.8014
  • Accuracy at Es/N0 <= 2 dB: 0.5169
  • Rank: 5 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.908 908 / 1,000
    bpsk 0.791 791 / 1,000
    qpsk 0.652 652 / 1,000
    16qam 0.615 615 / 1,000
    2fsk 0.863 863 / 1,000
    4fsk 0.888 888 / 1,000
    gmsk 0.896 896 / 1,000
    ofdm 0.798 798 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.457 750 / 1,641
    0 to 4 dB 0.710 1,128 / 1,589
    4 to 8 dB 0.895 1,394 / 1,557
    8 to 12 dB 0.968 1,531 / 1,582
    12 to 16 dB 0.986 1,608 / 1,631

6. dark_knights

Public test set (8,000 examples)

  • Accuracy: 0.7973
  • Accuracy at Es/N0 <= 2 dB: 0.5250
  • Rank: 6 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.920 920 / 1,000
    bpsk 0.810 810 / 1,000
    qpsk 0.643 643 / 1,000
    16qam 0.603 603 / 1,000
    2fsk 0.860 860 / 1,000
    4fsk 0.888 888 / 1,000
    gmsk 0.898 898 / 1,000
    ofdm 0.756 756 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.464 761 / 1,641
    0 to 4 dB 0.717 1,139 / 1,589
    4 to 8 dB 0.882 1,374 / 1,557
    8 to 12 dB 0.951 1,504 / 1,582
    12 to 16 dB 0.981 1,600 / 1,631

7. tensors

Public test set (8,000 examples)

  • Accuracy: 0.7973
  • Accuracy at Es/N0 <= 2 dB: 0.5132
  • Rank: 7 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.905 905 / 1,000
    bpsk 0.815 815 / 1,000
    qpsk 0.625 625 / 1,000
    16qam 0.636 636 / 1,000
    2fsk 0.857 857 / 1,000
    4fsk 0.885 885 / 1,000
    gmsk 0.876 876 / 1,000
    ofdm 0.779 779 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.453 743 / 1,641
    0 to 4 dB 0.709 1,127 / 1,589
    4 to 8 dB 0.893 1,390 / 1,557
    8 to 12 dB 0.963 1,523 / 1,582
    12 to 16 dB 0.978 1,595 / 1,631

8. Tensor_Titans

Public test set (8,000 examples)

  • Accuracy: 0.7929
  • Accuracy at Es/N0 <= 2 dB: 0.5193
  • Rank: 8 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.921 921 / 1,000
    bpsk 0.812 812 / 1,000
    qpsk 0.618 618 / 1,000
    16qam 0.584 584 / 1,000
    2fsk 0.872 872 / 1,000
    4fsk 0.880 880 / 1,000
    gmsk 0.888 888 / 1,000
    ofdm 0.768 768 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.464 762 / 1,641
    0 to 4 dB 0.705 1,121 / 1,589
    4 to 8 dB 0.873 1,360 / 1,557
    8 to 12 dB 0.951 1,505 / 1,582
    12 to 16 dB 0.978 1,595 / 1,631

9. QuadraMinds

Public test set (8,000 examples)

  • Accuracy: 0.7913
  • Accuracy at Es/N0 <= 2 dB: 0.4978
  • Rank: 9 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.906 906 / 1,000
    bpsk 0.802 802 / 1,000
    qpsk 0.600 600 / 1,000
    16qam 0.635 635 / 1,000
    2fsk 0.846 846 / 1,000
    4fsk 0.872 872 / 1,000
    gmsk 0.880 880 / 1,000
    ofdm 0.789 789 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.444 728 / 1,641
    0 to 4 dB 0.687 1,091 / 1,589
    4 to 8 dB 0.888 1,382 / 1,557
    8 to 12 dB 0.960 1,518 / 1,582
    12 to 16 dB 0.988 1,611 / 1,631

10. ML_Mavericks

Public test set (8,000 examples)

  • Accuracy: 0.7890
  • Accuracy at Es/N0 <= 2 dB: 0.4961
  • Rank: 10 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.918 918 / 1,000
    bpsk 0.801 801 / 1,000
    qpsk 0.617 617 / 1,000
    16qam 0.623 623 / 1,000
    2fsk 0.841 841 / 1,000
    4fsk 0.872 872 / 1,000
    gmsk 0.867 867 / 1,000
    ofdm 0.773 773 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.433 711 / 1,641
    0 to 4 dB 0.697 1,108 / 1,589
    4 to 8 dB 0.881 1,371 / 1,557
    8 to 12 dB 0.958 1,516 / 1,582
    12 to 16 dB 0.985 1,606 / 1,631

11. Akatsuki

Public test set (8,000 examples)

  • Accuracy: 0.7566
  • Accuracy at Es/N0 <= 2 dB: 0.4567
  • Rank: 11 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.878 878 / 1,000
    bpsk 0.766 766 / 1,000
    qpsk 0.597 597 / 1,000
    16qam 0.538 538 / 1,000
    2fsk 0.807 807 / 1,000
    4fsk 0.850 850 / 1,000
    gmsk 0.864 864 / 1,000
    ofdm 0.753 753 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.393 645 / 1,641
    0 to 4 dB 0.654 1,040 / 1,589
    4 to 8 dB 0.854 1,329 / 1,557
    8 to 12 dB 0.932 1,475 / 1,582
    12 to 16 dB 0.959 1,564 / 1,631

12. Learners

Public test set (8,000 examples)

  • Accuracy: 0.7521
  • Accuracy at Es/N0 <= 2 dB: 0.4595
  • Rank: 12 of 12 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.897 897 / 1,000
    bpsk 0.786 786 / 1,000
    qpsk 0.562 562 / 1,000
    16qam 0.517 517 / 1,000
    2fsk 0.831 831 / 1,000
    4fsk 0.856 856 / 1,000
    gmsk 0.856 856 / 1,000
    ofdm 0.712 712 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.406 667 / 1,641
    0 to 4 dB 0.646 1,027 / 1,589
    4 to 8 dB 0.846 1,317 / 1,557
    8 to 12 dB 0.924 1,461 / 1,582
    12 to 16 dB 0.947 1,545 / 1,631

Author: shashank

Created: 2026-10-29 Thu 11:52