Neuroprobe

Benchmark for evaluating decoding capabilities of iEEG foundation models.

Rank	Model	Author	Organization	Date	Overall AUROC	Sentence Onset	Speech	Volume	Delta Volume	Voice Pitch	Word Position	Inter-word Gap	GPT-2 Surprisal	Head Word Position	Part of Speech	Word Length	Global Optical Flow	Local Optical Flow	Frame Brightness	Number of Faces
1	DIVER-1_0.1s_tiny_frozen	Yonghyeon Gwon	Seoul National University	2026-01-08	0.678	0.930	0.897	0.719	0.803	0.592	0.798	0.636	0.629	0.623	0.627	0.646	0.628	0.616	0.495	0.525
2	DIVER-1_0.1s_tiny	Yonghyeon Gwon	Seoul National University	2026-01-08	0.665	0.924	0.897	0.688	0.814	0.561	0.799	0.634	0.618	0.617	0.607	0.638	0.594	0.584	0.486	0.518
3	Linear (Laplacian re-referencing + spectrogram)	Andrii Zahorodnii	MIT	2025-09-25	0.660	0.891	0.883	0.717	0.762	0.578	0.740	0.612	0.613	0.602	0.605	0.618	0.625	0.607	0.521	0.530
4	Linear (spectrogram)	Andrii Zahorodnii	MIT	2025-09-25	0.630	0.851	0.825	0.726	0.718	0.570	0.657	0.579	0.570	0.565	0.559	0.569	0.604	0.593	0.533	0.525
5	Linear (raw voltage)	Andrii Zahorodnii	MIT	2025-09-25	0.606	0.795	0.656	0.595	0.753	0.536	0.742	0.595	0.584	0.570	0.576	0.599	0.535	0.544	0.507	0.499
6	BrainBERT (frozen; Wang et al. 2023)	-	MIT	2025-09-25	0.586	0.757	0.611	0.583	0.706	0.524	0.685	0.584	0.580	0.585	0.556	0.571	0.521	0.525	0.508	0.503
7	BrainBERT (untrained, frozen)	-	MIT	2025-09-25	0.585	0.750	0.603	0.570	0.697	0.524	0.684	0.583	0.581	0.587	0.553	0.571	0.528	0.528	0.504	0.505
8	PopulationTransformer (Chau et al. 2024)	-	MIT	2025-09-25	0.545	0.689	0.677	0.576	0.628	0.509	0.519	0.509	0.523	0.519	0.513	0.505	0.509	0.508	0.499	0.492

Rank	Model	Author	Organization	Date	Overall AUROC	Sentence Onset	Speech	Volume	Delta Volume	Voice Pitch	Word Position	Inter-word Gap	GPT-2 Surprisal	Head Word Position	Part of Speech	Word Length	Global Optical Flow	Local Optical Flow	Frame Brightness	Number of Faces
1	Linear (Laplacian re-referencing + spectrogram)	Andrii Zahorodnii	MIT	2025-09-25	0.648	0.904	0.889	0.714	0.734	0.579	0.691	0.590	0.593	0.580	0.610	0.609	0.595	0.578	0.535	0.525
2	Linear (spectrogram)	Andrii Zahorodnii	MIT	2025-09-25	0.626	0.861	0.849	0.727	0.702	0.564	0.648	0.560	0.567	0.557	0.564	0.564	0.580	0.576	0.546	0.520
3	BrainBERT (frozen; Wang et al. 2023)	-	MIT	2025-09-25	0.581	0.743	0.631	0.572	0.692	0.509	0.661	0.571	0.580	0.572	0.556	0.559	0.527	0.534	0.506	0.497
4	Linear (raw voltage)	Andrii Zahorodnii	MIT	2025-09-25	0.576	0.728	0.611	0.564	0.707	0.529	0.664	0.554	0.561	0.537	0.569	0.558	0.528	0.523	0.494	0.509
5	BrainBERT (untrained, frozen)	-	MIT	2025-09-25	0.574	0.724	0.603	0.560	0.680	0.508	0.664	0.564	0.578	0.573	0.553	0.561	0.521	0.529	0.500	0.498
6	PopulationTransformer (Chau et al. 2024)	-	MIT	2025-09-25	0.566	0.774	0.716	0.574	0.646	0.510	0.559	0.531	0.556	0.524	0.502	0.523	0.529	0.528	0.504	0.512

Rank	Model	Author	Organization	Date	Overall AUROC	Sentence Onset	Speech	Volume	Delta Volume	Voice Pitch	Word Position	Inter-word Gap	GPT-2 Surprisal	Head Word Position	Part of Speech	Word Length	Global Optical Flow	Local Optical Flow	Frame Brightness	Number of Faces
1	Linear (Laplacian re-referencing + spectrogram)	Andrii Zahorodnii	MIT	2025-09-25	0.539	0.673	0.642	0.527	0.568	0.505	0.571	0.515	0.508	0.521	0.508	0.508	0.515	0.513	0.499	0.508
2	Linear (spectrogram)	Andrii Zahorodnii	MIT	2025-09-25	0.528	0.621	0.585	0.530	0.555	0.505	0.552	0.508	0.510	0.511	0.509	0.509	0.508	0.506	0.514	0.496
3	BrainBERT (untrained, frozen)	-	MIT	2025-09-25	0.527	0.585	0.537	0.524	0.590	0.505	0.574	0.513	0.522	0.530	0.517	0.509	0.503	0.501	0.502	0.500
4	PopulationTransformer (Chau et al. 2024)	-	MIT	2025-09-25	0.526	0.638	0.594	0.526	0.573	0.509	0.503	0.519	0.522	0.509	0.498	0.498	0.503	0.500	0.502	0.494
5	BrainBERT (frozen; Wang et al. 2023)	-	MIT	2025-09-25	0.522	0.582	0.537	0.521	0.574	0.507	0.549	0.510	0.511	0.524	0.509	0.504	0.501	0.498	0.506	0.501
6	Linear (raw voltage)	Andrii Zahorodnii	MIT	2025-09-25	0.510	0.539	0.508	0.513	0.533	0.503	0.539	0.511	0.510	0.504	0.495	0.502	0.500	0.500	0.493	0.501

Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimuli

By Andrii Zahorodnii¹²*, Christopher Wang¹*, Bennett Stankovits¹*, Charikleia Moraitaki¹, Geeling Chau³, Andrei Barbu¹, Boris Katz¹, Ila R Fiete¹²

¹MIT CSAIL, CBMM | ²MIT McGovern Institute | ³Caltech | *Equal contribution

Download PDF GitHub

Citation

If you use Neuroprobe in your work, please cite our paper:

@misc{neuroprobe,
      title={Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimuli}, 
      author={Andrii Zahorodnii and Christopher Wang and Bennett Stankovits and Charikleia Moraitaki and Geeling Chau and Andrei Barbu and Boris Katz and Ila R Fiete},
      year={2025},
      eprint={2509.21671},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2509.21671}, 
}