Also, please note that there has been a slight mistake in the evaluation script evaluate_performance.m. The mistake affects the rare case in which only one ROI is estimated, and leads to a wrong measurement of the localization performance when the estimated ROI matches one of the true ones. An corrected version of the script is provided here: evaluate_performance.m. The code package has also been updated to include the corrected version. The corrected script will be used for evaluating the challenge contributions.
The Berlin Brain Connectivity Benchmark (BBCB) is a Matlab based simulation framework for generating electro- and magnetoencephalographic (EEG and MEG) data within a realistic electromagnetic volume conductor (head) model for the purpose of benchmarking EEG/MEG based brain connectivity estimation methodologies. Its development is motivated by our feeling that, while brain connectivity analyses using interaction measures such as Granger Causality are frequently applied to EEG/MEG data, their validity to detect brain interactions rather than more trivial properties of the data is rarely validated in suitable simulations. Problems such as spurious connectivity affecting certain interaction measures are overlooked when essential features of real EEG/MEG data such as mixed brain signals or correlated noise are not reflected in a simulation (see e.g. [5-7]). Their impact might therefore not be entirely clear to all researchers employing such measures. The goal of BCCB framework is therefore to facilitate the assessment of entire source connectivity estimation pipelines, which may be composed of separate pre-processing, inverse source reconstruction and connectivity estimation steps, as a whole, as opposed to benchmarking each analysis step separately under conditions that may be too simplistic.
The BBCB includes head models for both EEG and MEG that allow one to create physically realistic sensor data in an average human head anatomy. It also includes the necessary Matlab scripts for generating time series on the brain source level, mapping them to EEG/MEG sensors, adding noise etc. .
The simulation framework is described in the following paper: Haufe and Ewald, 2016.
The paper also proposes a procedure for benchmarking EEG based brain connectivity estimation. We describe a model for generating EEG data with a well-defined connectivity structure (two alpha band sources that are either linearly and uni-directionally interacting or entirely independent). The task is then to answer the following three questions:
1. Localization: In which two regions (brain octants) are the two alpha sources located? 2. Interaction: Does the data contain interaction (between the two sources) at all? 3. Directionality: If so, which estimated source octant contains the sending and which one the receiving source?
We provide Matlab code for generating data according to the proposed interaction model, and to evaluate the correctness of answers to questions 1-3 for a given dataset. We also provide a script for running an example pipeline involving LCMV beamforming for source reconstruction and the phase-slope index for connectivity estimation. The package also includes many helper functions, most notably routines for plotting scalp topographies and cortical sources distributions.
The package including code and EEG/MEG head models can be downloaded here: BBCB_code.tar.gz (~1.1 GB).
A documentation of the code and data structures used is provided here: BBCB_supplement.pdf.
BBCB_data_EEG1.tar.gz (~1.5 GB). BBCB_data_EEG2.tar.gz (~1.5 GB).
Corresponding MEG data (298 channel CTF system, same brain sources and brain noise, but different sensor noise of the same SNR) are here:
BBCB_data_MEG1.tar.gz (~1.7 GB). BBCB_data_MEG2.tar.gz (~1.7 GB). BBCB_data_MEG3.tar.gz (~1.7 GB). BBCB_data_MEG4.tar.gz (~1.7 GB). BBCB_data_MEG5.tar.gz (~1.7 GB).
Data and code archives should be extracted into the same folder. This can be done by entering the following commands into a Unix shell, where BBCB is an existing target folder.
tar -xzf BBCB_code.tar.gz -C BBCB tar -xzf BBCB_EEG1.tar.gz -C BBCB tar -xzf BBCB_EEG2.tar.gz -C BBCB tar -xzf BBCB_MEG1.tar.gz -C BBCB tar -xzf BBCB_MEG2.tar.gz -C BBCB tar -xzf BBCB_MEG3.tar.gz -C BBCB tar -xzf BBCB_MEG4.tar.gz -C BBCB tar -xzf BBCB_MEG5.tar.gz -C BBCB
Remark: MEG head model: While the EEG head model is described in the paper  and in detail in , this is not the case for the MEG model. The MEG model was computed in Brainstorm  using the same underlying anatomy as for EEG (the ICBM152 v2009 template described in ). We used the three-shell boundary element (BEM) method to compute the MEG lead field for 298 MEG channels of a CTF system, for which channel positions are provided by Brainstorm. All geometrical information such as channel positions are moreover part of the software package.
Submission deadline: 20th September
Evaluation of results: 25th September
Public dissemination of results: 1st October at BIOMAG
Researchers wishing to use data or code from this website in a publication are asked to cite  and . Other papers should be cited as appropriate.