DNN Challenge 1: final leaderboard

Evaluated on test_public.npz (8,000 examples), test_private.npz (16,000 examples). Ranked by private accuracy, then accuracy at Es/N0 <= 2 dB, then fewer parameters; exact ties share a rank.

Leaderboard

Here is the final leaderboard. Status OK indicates that the onnx passed the checker script, but the script cannot check the code for all the rules.

rank submission status params public_acc public_acc_lowsnr private_acc private_acc_lowsnr
1 tensors.onnx OK 71368 0.8237 0.5629 0.8261 0.5724
2 Santa_Claude.onnx OK 48472 0.819 0.5523 0.8222 0.563
3 ML_Mavericks.onnx OK 52152 0.8223 0.5625 0.8205 0.5671
4 Team_Transformer.onnx OK 60344 0.8171 0.5442 0.8178 0.5535
5 the_tuning_forks.onnx OK 68296 0.818 0.5535 0.8169 0.5542
6 TokenLimitReached.onnx OK 73832 0.8155 0.5482 0.8161 0.5585
7 Illegal_Activations.onnx OK 70232 0.81 0.5348 0.8151 0.5533
8 QuadraMinds.onnx OK 74776 0.8053 0.5283 0.8095 0.5425
9 dark_knights.onnx OK 41160 0.8039 0.5454 0.8084 0.557
10 Tensor_Titans.onnx OK 74664 0.7993 0.5238 0.8044 0.5441
11 Team7.onnx OK 65960 0.8107 0.5381 0.8041 0.5285
12 Akatsuki.onnx OK 51848 0.7865 0.4856 0.7876 0.4932
13 Learners.onnx OK 47752 0.7861 0.5002 0.7861 0.5047

Submissions with issues

  • ML_Mavericks used test-time augmentations. Gemini told them that including TTA into the model was “100% compliant”, answering a prompt that claimed the rules don’t mention TTA. They do, in FAQ 7. In general, stacking transformed copies of the input along the batch axis counts as weight re-use. This will not constitute an automatic disqualification, but we will look into this more carefully during grading.

Observations

The leaderboard changed after the first round. It looks like the winning team made significant changes to their model after the leaderboard was announced. The final accuracies of most submissions improved, as many teams changed their architecture. I think this is what makes a competition more fun and rewarding: You try different things and hopefully learn a lot during the process.

Most teams (particularly at the top of the leaderboard) used web versions of Chatgpt/Claude while a couple of teams used agents to generate code and modify. However, the ones who used agents performed poorly, and I believe that some of the best performing teams did well because they had a significant number of interactions with the respective models and tried different approaches, which slowed down the process but overall gave a better result.

Public test set: accuracy per class (all teams)

Rank Team ook bpsk qpsk 16qam 2fsk 4fsk gmsk ofdm
1 tensors 0.940 0.831 0.664 0.663 0.885 0.893 0.910 0.804
2 Santa_Claude 0.929 0.812 0.658 0.701 0.878 0.882 0.885 0.807
3 ML_Mavericks 0.930 0.825 0.672 0.664 0.895 0.889 0.899 0.804
4 Team_Transformer 0.929 0.819 0.654 0.651 0.877 0.909 0.892 0.806
5 the_tuning_forks 0.933 0.823 0.645 0.654 0.885 0.903 0.903 0.798
6 TokenLimitReached 0.936 0.814 0.654 0.645 0.892 0.895 0.901 0.787
7 Illegal_Activations 0.921 0.822 0.659 0.652 0.878 0.876 0.896 0.776
8 QuadraMinds 0.920 0.802 0.657 0.625 0.860 0.895 0.884 0.799
9 dark_knights 0.923 0.813 0.663 0.593 0.871 0.890 0.896 0.782
10 Tensor_Titans 0.921 0.816 0.636 0.622 0.862 0.879 0.884 0.774
11 Team7 0.919 0.818 0.682 0.633 0.862 0.892 0.895 0.785
12 Akatsuki 0.904 0.778 0.635 0.603 0.847 0.863 0.877 0.785
13 Learners 0.912 0.798 0.624 0.599 0.850 0.875 0.868 0.763

Public test set: accuracy per Es/N0 band (all teams)

Rank Team -4 to 0 dB 0 to 4 dB 4 to 8 dB 8 to 12 dB 12 to 16 dB
1 tensors 0.500 0.755 0.910 0.975 0.988
2 Santa_Claude 0.489 0.750 0.907 0.968 0.990
3 ML_Mavericks 0.494 0.762 0.900 0.970 0.994
4 Team_Transformer 0.484 0.741 0.911 0.972 0.988
5 the_tuning_forks 0.489 0.744 0.908 0.970 0.988
6 TokenLimitReached 0.486 0.743 0.904 0.968 0.985
7 Illegal_Activations 0.477 0.726 0.901 0.966 0.988
8 QuadraMinds 0.467 0.718 0.896 0.966 0.988
9 dark_knights 0.479 0.736 0.889 0.949 0.975
10 Tensor_Titans 0.467 0.721 0.875 0.958 0.983
11 Team7 0.473 0.737 0.897 0.970 0.985
12 Akatsuki 0.423 0.678 0.890 0.965 0.987
13 Learners 0.439 0.686 0.877 0.958 0.980

Private test set: accuracy per class (all teams)

Rank Team ook bpsk qpsk 16qam 2fsk 4fsk gmsk ofdm
1 tensors 0.931 0.833 0.686 0.660 0.889 0.901 0.913 0.794
2 Santa_Claude 0.922 0.816 0.680 0.691 0.883 0.898 0.898 0.789
3 ML_Mavericks 0.921 0.833 0.678 0.643 0.892 0.903 0.912 0.781
4 Team_Transformer 0.922 0.817 0.678 0.643 0.888 0.931 0.895 0.767
5 the_tuning_forks 0.926 0.820 0.662 0.656 0.882 0.914 0.899 0.775
6 TokenLimitReached 0.926 0.821 0.672 0.640 0.888 0.904 0.904 0.773
7 Illegal_Activations 0.913 0.820 0.675 0.646 0.888 0.906 0.900 0.772
8 QuadraMinds 0.920 0.802 0.666 0.640 0.875 0.909 0.890 0.773
9 dark_knights 0.919 0.820 0.687 0.593 0.882 0.907 0.905 0.754
10 Tensor_Titans 0.914 0.812 0.640 0.625 0.877 0.901 0.900 0.764
11 Team7 0.908 0.820 0.678 0.628 0.864 0.885 0.897 0.752
12 Akatsuki 0.890 0.787 0.659 0.608 0.855 0.877 0.889 0.735
13 Learners 0.903 0.788 0.625 0.602 0.860 0.879 0.891 0.741

Private test set: accuracy per Es/N0 band (all teams)

Rank Team -4 to 0 dB 0 to 4 dB 4 to 8 dB 8 to 12 dB 12 to 16 dB
1 tensors 0.497 0.772 0.910 0.972 0.989
2 Santa_Claude 0.487 0.764 0.908 0.971 0.991
3 ML_Mavericks 0.496 0.760 0.901 0.968 0.988
4 Team_Transformer 0.488 0.749 0.904 0.971 0.988
5 the_tuning_forks 0.486 0.751 0.898 0.971 0.989
6 TokenLimitReached 0.487 0.753 0.895 0.970 0.986
7 Illegal_Activations 0.485 0.744 0.898 0.971 0.987
8 QuadraMinds 0.476 0.735 0.896 0.966 0.986
9 dark_knights 0.492 0.738 0.886 0.956 0.981
10 Tensor_Titans 0.477 0.730 0.885 0.962 0.980
11 Team7 0.456 0.736 0.889 0.965 0.986
12 Akatsuki 0.423 0.698 0.883 0.962 0.984
13 Learners 0.436 0.703 0.870 0.958 0.975

Per-team breakdown

1. tensors

Public test set (8,000 examples)

  • Accuracy: 0.8237
  • Accuracy at Es/N0 <= 2 dB: 0.5629
  • Accuracy per class
    Class Accuracy Correct
    ook 0.940 940 / 1,000
    bpsk 0.831 831 / 1,000
    qpsk 0.664 664 / 1,000
    16qam 0.663 663 / 1,000
    2fsk 0.885 885 / 1,000
    4fsk 0.893 893 / 1,000
    gmsk 0.910 910 / 1,000
    ofdm 0.804 804 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.500 820 / 1,641
    0 to 4 dB 0.755 1,199 / 1,589
    4 to 8 dB 0.910 1,417 / 1,557
    8 to 12 dB 0.975 1,542 / 1,582
    12 to 16 dB 0.988 1,612 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8261
  • Accuracy at Es/N0 <= 2 dB: 0.5724
  • Rank: 1 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.931 1,863 / 2,000
    bpsk 0.833 1,665 / 2,000
    qpsk 0.686 1,372 / 2,000
    16qam 0.660 1,321 / 2,000
    2fsk 0.889 1,778 / 2,000
    4fsk 0.901 1,803 / 2,000
    gmsk 0.913 1,827 / 2,000
    ofdm 0.794 1,588 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.497 1,609 / 3,240
    0 to 4 dB 0.772 2,538 / 3,287
    4 to 8 dB 0.910 2,838 / 3,118
    8 to 12 dB 0.972 3,133 / 3,222
    12 to 16 dB 0.989 3,099 / 3,133

2. Santa_Claude

Public test set (8,000 examples)

  • Accuracy: 0.8190
  • Accuracy at Es/N0 <= 2 dB: 0.5523
  • Accuracy per class
    Class Accuracy Correct
    ook 0.929 929 / 1,000
    bpsk 0.812 812 / 1,000
    qpsk 0.658 658 / 1,000
    16qam 0.701 701 / 1,000
    2fsk 0.878 878 / 1,000
    4fsk 0.882 882 / 1,000
    gmsk 0.885 885 / 1,000
    ofdm 0.807 807 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.489 802 / 1,641
    0 to 4 dB 0.750 1,191 / 1,589
    4 to 8 dB 0.907 1,412 / 1,557
    8 to 12 dB 0.968 1,532 / 1,582
    12 to 16 dB 0.990 1,615 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8222
  • Accuracy at Es/N0 <= 2 dB: 0.5630
  • Rank: 2 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.922 1,844 / 2,000
    bpsk 0.816 1,631 / 2,000
    qpsk 0.680 1,361 / 2,000
    16qam 0.691 1,382 / 2,000
    2fsk 0.883 1,766 / 2,000
    4fsk 0.898 1,796 / 2,000
    gmsk 0.898 1,796 / 2,000
    ofdm 0.789 1,579 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.487 1,577 / 3,240
    0 to 4 dB 0.764 2,511 / 3,287
    4 to 8 dB 0.908 2,832 / 3,118
    8 to 12 dB 0.971 3,130 / 3,222
    12 to 16 dB 0.991 3,105 / 3,133

3. ML_Mavericks

Public test set (8,000 examples)

  • Accuracy: 0.8223
  • Accuracy at Es/N0 <= 2 dB: 0.5625
  • Accuracy per class
    Class Accuracy Correct
    ook 0.930 930 / 1,000
    bpsk 0.825 825 / 1,000
    qpsk 0.672 672 / 1,000
    16qam 0.664 664 / 1,000
    2fsk 0.895 895 / 1,000
    4fsk 0.889 889 / 1,000
    gmsk 0.899 899 / 1,000
    ofdm 0.804 804 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.494 810 / 1,641
    0 to 4 dB 0.762 1,211 / 1,589
    4 to 8 dB 0.900 1,402 / 1,557
    8 to 12 dB 0.970 1,534 / 1,582
    12 to 16 dB 0.994 1,621 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8205
  • Accuracy at Es/N0 <= 2 dB: 0.5671
  • Rank: 3 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.921 1,842 / 2,000
    bpsk 0.833 1,666 / 2,000
    qpsk 0.678 1,357 / 2,000
    16qam 0.643 1,286 / 2,000
    2fsk 0.892 1,785 / 2,000
    4fsk 0.903 1,806 / 2,000
    gmsk 0.912 1,824 / 2,000
    ofdm 0.781 1,562 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.496 1,608 / 3,240
    0 to 4 dB 0.760 2,498 / 3,287
    4 to 8 dB 0.901 2,809 / 3,118
    8 to 12 dB 0.968 3,119 / 3,222
    12 to 16 dB 0.988 3,094 / 3,133

4. Team_Transformer

Public test set (8,000 examples)

  • Accuracy: 0.8171
  • Accuracy at Es/N0 <= 2 dB: 0.5442
  • Accuracy per class
    Class Accuracy Correct
    ook 0.929 929 / 1,000
    bpsk 0.819 819 / 1,000
    qpsk 0.654 654 / 1,000
    16qam 0.651 651 / 1,000
    2fsk 0.877 877 / 1,000
    4fsk 0.909 909 / 1,000
    gmsk 0.892 892 / 1,000
    ofdm 0.806 806 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.484 794 / 1,641
    0 to 4 dB 0.741 1,177 / 1,589
    4 to 8 dB 0.911 1,418 / 1,557
    8 to 12 dB 0.972 1,537 / 1,582
    12 to 16 dB 0.988 1,611 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8178
  • Accuracy at Es/N0 <= 2 dB: 0.5535
  • Rank: 4 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.922 1,844 / 2,000
    bpsk 0.817 1,633 / 2,000
    qpsk 0.678 1,356 / 2,000
    16qam 0.643 1,287 / 2,000
    2fsk 0.888 1,776 / 2,000
    4fsk 0.931 1,863 / 2,000
    gmsk 0.895 1,791 / 2,000
    ofdm 0.767 1,535 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.488 1,580 / 3,240
    0 to 4 dB 0.749 2,462 / 3,287
    4 to 8 dB 0.904 2,818 / 3,118
    8 to 12 dB 0.971 3,130 / 3,222
    12 to 16 dB 0.988 3,095 / 3,133

5. the_tuning_forks

Public test set (8,000 examples)

  • Accuracy: 0.8180
  • Accuracy at Es/N0 <= 2 dB: 0.5535
  • Accuracy per class
    Class Accuracy Correct
    ook 0.933 933 / 1,000
    bpsk 0.823 823 / 1,000
    qpsk 0.645 645 / 1,000
    16qam 0.654 654 / 1,000
    2fsk 0.885 885 / 1,000
    4fsk 0.903 903 / 1,000
    gmsk 0.903 903 / 1,000
    ofdm 0.798 798 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.489 802 / 1,641
    0 to 4 dB 0.744 1,183 / 1,589
    4 to 8 dB 0.908 1,414 / 1,557
    8 to 12 dB 0.970 1,534 / 1,582
    12 to 16 dB 0.988 1,611 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8169
  • Accuracy at Es/N0 <= 2 dB: 0.5542
  • Rank: 5 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.926 1,853 / 2,000
    bpsk 0.820 1,639 / 2,000
    qpsk 0.662 1,324 / 2,000
    16qam 0.656 1,313 / 2,000
    2fsk 0.882 1,765 / 2,000
    4fsk 0.914 1,828 / 2,000
    gmsk 0.899 1,798 / 2,000
    ofdm 0.775 1,551 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.486 1,574 / 3,240
    0 to 4 dB 0.751 2,468 / 3,287
    4 to 8 dB 0.898 2,801 / 3,118
    8 to 12 dB 0.971 3,129 / 3,222
    12 to 16 dB 0.989 3,099 / 3,133

6. TokenLimitReached

Public test set (8,000 examples)

  • Accuracy: 0.8155
  • Accuracy at Es/N0 <= 2 dB: 0.5482
  • Accuracy per class
    Class Accuracy Correct
    ook 0.936 936 / 1,000
    bpsk 0.814 814 / 1,000
    qpsk 0.654 654 / 1,000
    16qam 0.645 645 / 1,000
    2fsk 0.892 892 / 1,000
    4fsk 0.895 895 / 1,000
    gmsk 0.901 901 / 1,000
    ofdm 0.787 787 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.486 798 / 1,641
    0 to 4 dB 0.743 1,180 / 1,589
    4 to 8 dB 0.904 1,407 / 1,557
    8 to 12 dB 0.968 1,532 / 1,582
    12 to 16 dB 0.985 1,607 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8161
  • Accuracy at Es/N0 <= 2 dB: 0.5585
  • Rank: 6 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.926 1,853 / 2,000
    bpsk 0.821 1,641 / 2,000
    qpsk 0.672 1,344 / 2,000
    16qam 0.640 1,281 / 2,000
    2fsk 0.888 1,776 / 2,000
    4fsk 0.904 1,808 / 2,000
    gmsk 0.904 1,808 / 2,000
    ofdm 0.773 1,546 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.487 1,577 / 3,240
    0 to 4 dB 0.753 2,475 / 3,287
    4 to 8 dB 0.895 2,791 / 3,118
    8 to 12 dB 0.970 3,124 / 3,222
    12 to 16 dB 0.986 3,090 / 3,133

7. Illegal_Activations

Public test set (8,000 examples)

  • Accuracy: 0.8100
  • Accuracy at Es/N0 <= 2 dB: 0.5348
  • Accuracy per class
    Class Accuracy Correct
    ook 0.921 921 / 1,000
    bpsk 0.822 822 / 1,000
    qpsk 0.659 659 / 1,000
    16qam 0.652 652 / 1,000
    2fsk 0.878 878 / 1,000
    4fsk 0.876 876 / 1,000
    gmsk 0.896 896 / 1,000
    ofdm 0.776 776 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.477 783 / 1,641
    0 to 4 dB 0.726 1,154 / 1,589
    4 to 8 dB 0.901 1,403 / 1,557
    8 to 12 dB 0.966 1,528 / 1,582
    12 to 16 dB 0.988 1,612 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8151
  • Accuracy at Es/N0 <= 2 dB: 0.5533
  • Rank: 7 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.913 1,826 / 2,000
    bpsk 0.820 1,640 / 2,000
    qpsk 0.675 1,351 / 2,000
    16qam 0.646 1,292 / 2,000
    2fsk 0.888 1,776 / 2,000
    4fsk 0.906 1,812 / 2,000
    gmsk 0.900 1,800 / 2,000
    ofdm 0.772 1,544 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.485 1,573 / 3,240
    0 to 4 dB 0.744 2,446 / 3,287
    4 to 8 dB 0.898 2,800 / 3,118
    8 to 12 dB 0.971 3,129 / 3,222
    12 to 16 dB 0.987 3,093 / 3,133

8. QuadraMinds

Public test set (8,000 examples)

  • Accuracy: 0.8053
  • Accuracy at Es/N0 <= 2 dB: 0.5283
  • Accuracy per class
    Class Accuracy Correct
    ook 0.920 920 / 1,000
    bpsk 0.802 802 / 1,000
    qpsk 0.657 657 / 1,000
    16qam 0.625 625 / 1,000
    2fsk 0.860 860 / 1,000
    4fsk 0.895 895 / 1,000
    gmsk 0.884 884 / 1,000
    ofdm 0.799 799 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.467 766 / 1,641
    0 to 4 dB 0.718 1,141 / 1,589
    4 to 8 dB 0.896 1,395 / 1,557
    8 to 12 dB 0.966 1,528 / 1,582
    12 to 16 dB 0.988 1,612 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8095
  • Accuracy at Es/N0 <= 2 dB: 0.5425
  • Rank: 8 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.920 1,840 / 2,000
    bpsk 0.802 1,604 / 2,000
    qpsk 0.666 1,333 / 2,000
    16qam 0.640 1,280 / 2,000
    2fsk 0.875 1,751 / 2,000
    4fsk 0.909 1,818 / 2,000
    gmsk 0.890 1,780 / 2,000
    ofdm 0.773 1,546 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.476 1,543 / 3,240
    0 to 4 dB 0.735 2,416 / 3,287
    4 to 8 dB 0.896 2,793 / 3,118
    8 to 12 dB 0.966 3,112 / 3,222
    12 to 16 dB 0.986 3,088 / 3,133

9. dark_knights

Public test set (8,000 examples)

  • Accuracy: 0.8039
  • Accuracy at Es/N0 <= 2 dB: 0.5454
  • Accuracy per class
    Class Accuracy Correct
    ook 0.923 923 / 1,000
    bpsk 0.813 813 / 1,000
    qpsk 0.663 663 / 1,000
    16qam 0.593 593 / 1,000
    2fsk 0.871 871 / 1,000
    4fsk 0.890 890 / 1,000
    gmsk 0.896 896 / 1,000
    ofdm 0.782 782 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.479 786 / 1,641
    0 to 4 dB 0.736 1,169 / 1,589
    4 to 8 dB 0.889 1,384 / 1,557
    8 to 12 dB 0.949 1,501 / 1,582
    12 to 16 dB 0.975 1,591 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8084
  • Accuracy at Es/N0 <= 2 dB: 0.5570
  • Rank: 9 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.919 1,839 / 2,000
    bpsk 0.820 1,639 / 2,000
    qpsk 0.687 1,374 / 2,000
    16qam 0.593 1,186 / 2,000
    2fsk 0.882 1,765 / 2,000
    4fsk 0.907 1,814 / 2,000
    gmsk 0.905 1,810 / 2,000
    ofdm 0.754 1,508 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.492 1,595 / 3,240
    0 to 4 dB 0.738 2,427 / 3,287
    4 to 8 dB 0.886 2,761 / 3,118
    8 to 12 dB 0.956 3,079 / 3,222
    12 to 16 dB 0.981 3,073 / 3,133

10. Tensor_Titans

Public test set (8,000 examples)

  • Accuracy: 0.7993
  • Accuracy at Es/N0 <= 2 dB: 0.5238
  • Accuracy per class
    Class Accuracy Correct
    ook 0.921 921 / 1,000
    bpsk 0.816 816 / 1,000
    qpsk 0.636 636 / 1,000
    16qam 0.622 622 / 1,000
    2fsk 0.862 862 / 1,000
    4fsk 0.879 879 / 1,000
    gmsk 0.884 884 / 1,000
    ofdm 0.774 774 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.467 767 / 1,641
    0 to 4 dB 0.721 1,145 / 1,589
    4 to 8 dB 0.875 1,363 / 1,557
    8 to 12 dB 0.958 1,515 / 1,582
    12 to 16 dB 0.983 1,604 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8044
  • Accuracy at Es/N0 <= 2 dB: 0.5441
  • Rank: 10 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.914 1,828 / 2,000
    bpsk 0.812 1,624 / 2,000
    qpsk 0.640 1,281 / 2,000
    16qam 0.625 1,250 / 2,000
    2fsk 0.877 1,755 / 2,000
    4fsk 0.901 1,802 / 2,000
    gmsk 0.900 1,801 / 2,000
    ofdm 0.764 1,529 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.477 1,545 / 3,240
    0 to 4 dB 0.730 2,398 / 3,287
    4 to 8 dB 0.885 2,759 / 3,118
    8 to 12 dB 0.962 3,098 / 3,222
    12 to 16 dB 0.980 3,070 / 3,133

11. Team7

Public test set (8,000 examples)

  • Accuracy: 0.8107
  • Accuracy at Es/N0 <= 2 dB: 0.5381
  • Accuracy per class
    Class Accuracy Correct
    ook 0.919 919 / 1,000
    bpsk 0.818 818 / 1,000
    qpsk 0.682 682 / 1,000
    16qam 0.633 633 / 1,000
    2fsk 0.862 862 / 1,000
    4fsk 0.892 892 / 1,000
    gmsk 0.895 895 / 1,000
    ofdm 0.785 785 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.473 777 / 1,641
    0 to 4 dB 0.737 1,171 / 1,589
    4 to 8 dB 0.897 1,397 / 1,557
    8 to 12 dB 0.970 1,534 / 1,582
    12 to 16 dB 0.985 1,607 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.8041
  • Accuracy at Es/N0 <= 2 dB: 0.5285
  • Rank: 11 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.908 1,817 / 2,000
    bpsk 0.820 1,639 / 2,000
    qpsk 0.678 1,357 / 2,000
    16qam 0.628 1,257 / 2,000
    2fsk 0.864 1,727 / 2,000
    4fsk 0.885 1,770 / 2,000
    gmsk 0.897 1,794 / 2,000
    ofdm 0.752 1,505 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.456 1,478 / 3,240
    0 to 4 dB 0.736 2,418 / 3,287
    4 to 8 dB 0.889 2,772 / 3,118
    8 to 12 dB 0.965 3,108 / 3,222
    12 to 16 dB 0.986 3,090 / 3,133

12. Akatsuki

Public test set (8,000 examples)

  • Accuracy: 0.7865
  • Accuracy at Es/N0 <= 2 dB: 0.4856
  • Accuracy per class
    Class Accuracy Correct
    ook 0.904 904 / 1,000
    bpsk 0.778 778 / 1,000
    qpsk 0.635 635 / 1,000
    16qam 0.603 603 / 1,000
    2fsk 0.847 847 / 1,000
    4fsk 0.863 863 / 1,000
    gmsk 0.877 877 / 1,000
    ofdm 0.785 785 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.423 694 / 1,641
    0 to 4 dB 0.678 1,078 / 1,589
    4 to 8 dB 0.890 1,385 / 1,557
    8 to 12 dB 0.965 1,526 / 1,582
    12 to 16 dB 0.987 1,609 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.7876
  • Accuracy at Es/N0 <= 2 dB: 0.4932
  • Rank: 12 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.890 1,780 / 2,000
    bpsk 0.787 1,575 / 2,000
    qpsk 0.659 1,319 / 2,000
    16qam 0.608 1,216 / 2,000
    2fsk 0.855 1,709 / 2,000
    4fsk 0.877 1,754 / 2,000
    gmsk 0.889 1,778 / 2,000
    ofdm 0.735 1,470 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.423 1,371 / 3,240
    0 to 4 dB 0.698 2,295 / 3,287
    4 to 8 dB 0.883 2,754 / 3,118
    8 to 12 dB 0.962 3,099 / 3,222
    12 to 16 dB 0.984 3,082 / 3,133

13. Learners

Public test set (8,000 examples)

  • Accuracy: 0.7861
  • Accuracy at Es/N0 <= 2 dB: 0.5002
  • Accuracy per class
    Class Accuracy Correct
    ook 0.912 912 / 1,000
    bpsk 0.798 798 / 1,000
    qpsk 0.624 624 / 1,000
    16qam 0.599 599 / 1,000
    2fsk 0.850 850 / 1,000
    4fsk 0.875 875 / 1,000
    gmsk 0.868 868 / 1,000
    ofdm 0.763 763 / 1,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.439 721 / 1,641
    0 to 4 dB 0.686 1,090 / 1,589
    4 to 8 dB 0.877 1,365 / 1,557
    8 to 12 dB 0.958 1,515 / 1,582
    12 to 16 dB 0.980 1,598 / 1,631

Private test set (16,000 examples)

  • Accuracy: 0.7861
  • Accuracy at Es/N0 <= 2 dB: 0.5047
  • Rank: 13 of 13 valid submissions
  • Accuracy per class
    Class Accuracy Correct
    ook 0.903 1,806 / 2,000
    bpsk 0.788 1,576 / 2,000
    qpsk 0.625 1,251 / 2,000
    16qam 0.602 1,204 / 2,000
    2fsk 0.860 1,720 / 2,000
    4fsk 0.879 1,758 / 2,000
    gmsk 0.891 1,782 / 2,000
    ofdm 0.741 1,481 / 2,000
  • Accuracy per Es/N0 band
    Es/N0 Accuracy Correct
    -4 to 0 dB 0.436 1,414 / 3,240
    0 to 4 dB 0.703 2,311 / 3,287
    4 to 8 dB 0.870 2,712 / 3,118
    8 to 12 dB 0.958 3,087 / 3,222
    12 to 16 dB 0.975 3,054 / 3,133

Date: 2026-11-03

Author: shashank

Created: 2026-11-04 Wed 22:52