Neuroprobe

Benchmark for evaluating decoding capabilities of iEEG foundation models.

Paper GitHub Submit
Brain Activity Animation
Rank Model Author Organization Date Overall AUROC Sentence Onset Speech Volume Delta Volume Voice Pitch Word Position Inter-word Gap GPT-2 Surprisal Head Word Position Part of Speech Word Length Global Optical Flow Local Optical Flow Frame Brightness Number of Faces
1 DIVER-1_0.1s_tiny_frozen Yonghyeon Gwon Seoul National University 2026-01-09 0.678 0.930 0.897 0.719 0.803 0.592 0.798 0.636 0.629 0.623 0.627 0.646 0.628 0.616 0.495 0.525
2 Population Transformer (global z-scoring) Geeling Chau Caltech 2026-04-28 0.670 0.930 0.919 0.766 0.781 0.596 0.758 0.620 0.603 0.601 0.604 0.622 0.621 0.618 0.496 0.517
3 CNN_Laplacian_rereferencing_spectrogram Geeling Chau Caltech 2026-04-21 0.669 0.886 0.888 0.757 0.762 0.618 0.734 0.583 0.597 0.614 0.621 0.636 0.638 0.630 0.536 0.530
4 DIVER-1_0.1s_tiny Yonghyeon Gwon Seoul National University 2026-01-09 0.665 0.924 0.897 0.688 0.814 0.561 0.799 0.634 0.618 0.617 0.607 0.638 0.594 0.584 0.486 0.518
5 Linear (Laplacian re-referencing + spectrogram) Andrii Zahorodnii MIT 2026-05-02 0.660 0.888 0.882 0.717 0.756 0.584 0.740 0.613 0.608 0.603 0.601 0.617 0.630 0.615 0.528 0.519
6 MLP_Laplacian_rereferencing_spectrogram Geeling Chau Caltech 2026-04-21 0.656 0.884 0.877 0.717 0.747 0.585 0.728 0.597 0.604 0.602 0.597 0.615 0.629 0.617 0.524 0.521
7 BrainBERT (frozen, global z-scoring) Geeling Chau Caltech 2026-05-02 0.626 0.883 0.887 0.723 0.677 0.564 0.620 0.526 0.557 0.567 0.576 0.586 0.609 0.599 0.497 0.516
8 Linear (spectrogram) Andrii Zahorodnii MIT 2026-05-02 0.625 0.837 0.816 0.719 0.711 0.566 0.651 0.582 0.567 0.564 0.557 0.565 0.604 0.596 0.522 0.522
9 Linear (raw voltage) Andrii Zahorodnii MIT 2026-05-02 0.604 0.792 0.645 0.597 0.742 0.535 0.742 0.598 0.575 0.569 0.579 0.595 0.536 0.541 0.507 0.503
10 BrainBERT (frozen; off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.585 0.754 0.606 0.576 0.699 0.527 0.679 0.581 0.580 0.582 0.556 0.571 0.527 0.520 0.510 0.509
11 BrainBERT (untrained; frozen; off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.581 0.749 0.598 0.566 0.690 0.519 0.677 0.581 0.579 0.581 0.557 0.569 0.525 0.520 0.495 0.506
12 GLIS-GNN (ST-GCN, Functional Graph) Eli Gross Columbia University 2026-04-14 0.553 0.710 0.708 0.601 0.581 0.516 0.534 0.505 0.522 0.523 0.521 0.520 0.524 0.537 0.472 0.515
13 PopulationTransformer (off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.549 0.741 0.672 0.562 0.622 0.527 0.528 0.523 0.517 0.516 0.507 0.521 0.515 0.501 0.489 0.500
Rank Model Author Organization Date Overall AUROC Sentence Onset Speech Volume Delta Volume Voice Pitch Word Position Inter-word Gap GPT-2 Surprisal Head Word Position Part of Speech Word Length Global Optical Flow Local Optical Flow Frame Brightness Number of Faces
1 CNN_Laplacian_rereferencing_spectrogram Geeling Chau Caltech 2026-04-21 0.670 0.919 0.919 0.731 0.753 0.618 0.712 0.581 0.622 0.619 0.638 0.638 0.629 0.608 0.524 0.546
2 Population Transformer (global z-scoring) Geeling Chau Caltech 2026-04-28 0.663 0.924 0.913 0.712 0.759 0.598 0.703 0.602 0.624 0.612 0.611 0.636 0.607 0.594 0.505 0.541
3 MLP_Laplacian_rereferencing_spectrogram Geeling Chau Caltech 2026-04-21 0.655 0.910 0.902 0.705 0.745 0.587 0.688 0.578 0.609 0.603 0.613 0.611 0.617 0.592 0.528 0.536
4 Linear (Laplacian re-referencing + spectrogram) Andrii Zahorodnii MIT 2026-05-02 0.651 0.913 0.908 0.708 0.737 0.588 0.696 0.584 0.596 0.585 0.612 0.608 0.598 0.580 0.532 0.522
5 BrainBERT (frozen, global z-scoring) Geeling Chau Caltech 2026-05-02 0.633 0.905 0.903 0.705 0.674 0.584 0.612 0.526 0.573 0.567 0.592 0.600 0.604 0.590 0.522 0.532
6 Linear (spectrogram) Andrii Zahorodnii MIT 2026-05-02 0.626 0.862 0.860 0.712 0.697 0.563 0.649 0.556 0.564 0.555 0.559 0.566 0.589 0.587 0.545 0.521
7 BrainBERT (frozen; off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.580 0.737 0.632 0.570 0.684 0.513 0.658 0.566 0.578 0.577 0.552 0.559 0.526 0.537 0.509 0.502
8 Linear (raw voltage) Andrii Zahorodnii MIT 2026-05-02 0.574 0.724 0.616 0.565 0.699 0.525 0.654 0.544 0.559 0.538 0.564 0.554 0.527 0.521 0.507 0.510
9 BrainBERT (untrained; frozen; off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.572 0.724 0.609 0.557 0.671 0.506 0.658 0.560 0.576 0.578 0.554 0.558 0.522 0.525 0.493 0.496
10 PopulationTransformer (off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.564 0.754 0.719 0.572 0.637 0.523 0.568 0.526 0.554 0.535 0.503 0.522 0.516 0.518 0.496 0.513
11 GLIS-GNN (ST-GCN, Functional Graph) Eli Gross Columbia University 2026-04-14 0.550 0.679 0.716 0.555 0.563 0.527 0.532 0.496 0.535 0.528 0.530 0.513 0.547 0.505 0.513 0.510
12 RNN (gru) Bennett Stankovits MIT 2026-02-02 0.510 0.531 0.537 0.512 0.518 0.498 0.507 0.494 0.514 0.511 0.514 0.509 0.503 0.506 0.493 0.503
Rank Model Author Organization Date Overall AUROC Sentence Onset Speech Volume Delta Volume Voice Pitch Word Position Inter-word Gap GPT-2 Surprisal Head Word Position Part of Speech Word Length Global Optical Flow Local Optical Flow Frame Brightness Number of Faces
1 CNN_Laplacian_rereferencing_spectrogram Geeling Chau Caltech 2026-04-21 0.578 0.780 0.751 0.587 0.662 0.516 0.617 0.545 0.539 0.555 0.520 0.511 0.531 0.528 0.516 0.505
2 Population Transformer (global z-scoring) Geeling Chau Caltech 2026-04-28 0.575 0.711 0.714 0.589 0.653 0.525 0.630 0.525 0.563 0.553 0.523 0.527 0.548 0.547 0.502 0.516
3 MLP_Laplacian_rereferencing_spectrogram Geeling Chau Caltech 2026-04-21 0.566 0.749 0.729 0.571 0.634 0.514 0.602 0.530 0.532 0.548 0.517 0.521 0.523 0.512 0.502 0.502
4 BrainBERT (frozen, global z-scoring) Geeling Chau Caltech 2026-05-02 0.547 0.712 0.722 0.536 0.564 0.529 0.539 0.499 0.507 0.540 0.515 0.507 0.534 0.514 0.497 0.492
5 Linear (Laplacian re-referencing + spectrogram) Andrii Zahorodnii MIT 2026-05-02 0.539 0.676 0.643 0.534 0.569 0.505 0.566 0.515 0.508 0.525 0.506 0.511 0.515 0.510 0.495 0.511
6 Linear (spectrogram) Andrii Zahorodnii MIT 2026-05-02 0.528 0.616 0.575 0.540 0.548 0.503 0.558 0.509 0.510 0.511 0.509 0.512 0.509 0.501 0.519 0.494
7 BrainBERT (untrained; frozen; off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.527 0.583 0.540 0.521 0.589 0.506 0.570 0.517 0.516 0.531 0.516 0.510 0.503 0.499 0.501 0.498
8 BrainBERT (frozen; off-the-shelf; per-window STFT z-scoring) Andrii Zahorodnii MIT 2026-05-02 0.522 0.584 0.537 0.526 0.571 0.498 0.549 0.510 0.515 0.527 0.505 0.509 0.501 0.497 0.509 0.498
9 GLIS-GNN (ST-GCN, Functional Graph) Eli Gross Columbia University 2026-04-14 0.515 0.608 0.541 0.559 0.519 0.490 0.503 0.498 0.508 0.497 0.500 0.488 0.483 0.538 0.497 0.498
10 Linear (raw voltage) Andrii Zahorodnii MIT 2026-05-02 0.509 0.534 0.504 0.517 0.529 0.501 0.536 0.510 0.506 0.506 0.496 0.504 0.500 0.498 0.496 0.501

Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimuli

By Andrii Zahorodnii¹²*, Christopher Wang¹*, Bennett Stankovits¹*, Charikleia Moraitaki¹, Geeling Chau³, Andrei Barbu¹, Boris Katz¹, Ila R Fiete¹²

¹MIT CSAIL, CBMM | ²MIT McGovern Institute | ³Caltech | *Equal contribution

Download PDF GitHub

Citation

If you use Neuroprobe in your work, please cite our paper:

@misc{neuroprobe,
      title={Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimuli}, 
      author={Andrii Zahorodnii and Christopher Wang and Bennett Stankovits and Charikleia Moraitaki and Geeling Chau and Andrei Barbu and Boris Katz and Ila R Fiete},
      year={2025},
      eprint={2509.21671},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2509.21671}, 
}