BCI Competition IV

- Final Results -

[ remarks | winners | true labels | organizers ]

The announcement and the data sets of the BCI Competition IV can be found here.

A Kind Request

It would be very helpful for the potential organization of further BCI competitions to get some feedback, criticism and suggestions, about this competition. We are interested to hear from participants about their experiences as well as from non-participants reporting why they refrained from taking part. Feedback may concern general issues or specific data sets (e.g. pros and contras for each data set).
Write your feedback to Benjamin Blankertz <benjamin.blankertz@tu-berlin.de>

[ top ]

Remarks

The results should not be taken too seriously. Of course they do not provide an objective ranking of quality:

There is great variance of how much effort contributors put into preparing their submissions.
Given the time-pressure of an approaching deadline there are a lot of possible pitfalls in processing the test data which may lead to bad or even "random" results. [This is the voice of almost-experience talking, bb.]
When test sets (and the number of classes) are relatively small, luck plays a big role. Let's take a look at a fictitious example. Suppose in a binary problem there are 15 methods that are able to classify correctly 60% of the ideal set of all trials and that they have random output on the remaining 40%. The expected accuracy of all these methods is 80%, but on a fixed test set consisting of 100 trials, the expected difference between the best and the worst result is greater than 10% (assuming independence between methods and test trials).

Nevertheless we hope that

something can be learned from this competition by presenting methods that worked on real world problems.
people who had not much success here can do a failure analysis (we provided labels of the test sets for that purpose) and gain experience for the future.
last but not least it was fun.

Watch out for the special volume devoted to this competition where each winning team will present a detailed description of their algorithms.

[ top ]

Data sets 1 [Berlin: artifically generated vs. real (human) data]

As stated in the description, data sets for some 'subjects' had been artificially generated. Which data sets are real human data and which were artificially generated was secret. Competitors were asked to guess which subjects were real and which artificial.

The solution is: data for subjects c, d, and e have been artificially generated. A guess on this question was submitted by 16 out of 23 competitors. The correct categorization was revealed by two competitors (Astrid Zeman and Manuel Moebius), 8 more competitors revealed 2 of the 3 artificial subjects, but one of those also also considered one real data set as artifical. One competitor made the correct split, but said he could not decide which group is which.

[ top ]

Data sets 1 [Berlin: human data]

The performance measure is the mean squared error with respect to the target vector (-1 for motor imagery class 1, 0 for relax and 1 for motor imagery class 2). Only the segments starting 1s after the starting cue until the stopping cue are considered. First row shows the average across all (human) subjects, rows 2 to 5 show results for individual subjects.
-> Note: The result for a submission with constant output of 0 is around 0.51 (differs for each subject depending on the exact distribution and duration of different mental states; range was 0.49 to 0.54). <-

#.	contributor	mse	a	b	f	g	research lab	co-contributors
1.	Zhang Haihong	0.382	0.40	0.42	0.42	0.29	Institute for Infocomm Research, Singapore	Kai Keng Ang, Guan Cuntai, Wang Chuanchu, Chin Zheng Yang
2.	Dieter Devlaminck	0.383	0.35	0.46	0.41	0.31	University of Ghent; Psychiatric Institute of Guislain; University Hospital Ghent	Willem Waegeman, Bart Wyns, Bruno Bauwens, Luc Boullart, Georges Otte, Patrick Santens
3.	Kai Keng Ang	0.397	0.39	0.43	0.40	0.36	Institute for Infocomm Research, Singapore	Zheng Yang Chin, Cuntai Guan, Haihong Zhang
4.	Jun Lv	0.442	0.48	0.47	0.52	0.30	South China University of Technology, Guangzhou, China	Meichun Liu, Kun Cai
5.	Liu Guangquan	0.455	0.47	0.51	0.46	0.38	Shanghai Jiao Tong University, China	Huang Gan, and Zhu Xiangyang
6.	Abdul Satti	0.459	0.47	0.51	0.49	0.36	University of Ulster	Damien Coyle, Ekta Makhija and Girijesh Prasad
7.	Fabian Bachl	0.466	0.38	0.62	0.51	0.35	Team PhyPa, TU Berlin	Christian Kothe, Thorsten Zander
8.	Jinyi Long	0.475	0.46	0.48	0.49	0.47	South China University of Technology, Guangzhou, China	Yuanqing Li, Zhunliang Yu
9.	Yunjun Nam	0.477	0.54	0.46	0.57	0.35	Pohang University of science and technology, Korea	Seungjin Choi
10.	Emily Mankin	0.499	0.47	0.54	0.48	0.50	University of California, San Diego	Paul Hammon, Walter Talbott
11.	Cedric Gouy-Pailler	0.557	0.57	0.65	0.64	0.37	GIPSA-lab, Grenoble Institute of Technology; Dynamique Cérébrale et Cognition Institut Fédératif des Neurosciences - Université Lyon	Marco Congedo, Chrsitian Jutten, Jérémie Mattout
12.	Jing Jin	0.559	0.55	0.52	0.49	0.67	Institute for Knowledge Discovery, Graz University of Technology	Teodoro Solis Escalante, Clemens Brunner, Gert Pfurtscheller
13.	Michael Buschbeck	0.679	0.57	0.70	0.72	0.73	atip	Klaus Kasper
14.	(*) Teodoro Solis-Escalante	0.692	0.70	0.57	0.54	0.96	Institute for Knowledge Discovery, Graz University of Technology	Jing Jin, Clemens Brunner, Gert Pfurtscheller
15.	Chen Guangming	0.842	0.92	0.82	0.79	0.84	College of Information Science and Technology and National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal Unviersity	Wu Jin, Zhang Jiacai
16.	Yu Huang	0.859	0.89	0.89	0.92	0.75	Perception-Motor Interaction Laboratory, School of Life Science and Technology, University of Electronic Science and Technology of China	Dezhong Yao
17.	Astrid Zeman	0.915	0.79	0.92	0.98	0.96	CSIRO ICT Centre, Australia
18.	Padma Palash	0.928	0.95	0.93	0.91	0.92	City University of Hong Kong	Wing-Ho Howard Leung
19.	Eric Gottschalk	0.930	1.09	0.80	0.89	0.94	Topspin
20.	Sung Wook	0.972	1.10	1.08	0.84	0.86	Gwangju Institute of Science and Technology, Korea	Sung Chan Jun
21.	Manuel Moebius	1.007	0.76	1.13	1.02	1.11	Hochschule Darmstadt
22.	Rui Li	1.150	1.14	1.46	1.47	0.53	Shenyang University of Technology, China	Li Ke, Peipei Pang, Wei He, Yan Huang
23.	Li Ke	1.156	1.05	1.24	1.16	1.17	Shenyang University of Technology, China	Rui Li, Peipei Pang, Wei He, Yan Huang
24.	Yang Banghua	1.312	1.46	1.05	1.39	1.35	Shanghai University, China

(*)Remark: After the true labels have been revealed, competitor Teodoro Solis-Escalante realized that he inaccidentally switched the labels (sign mismatch). Evaluation of his submission with sign corrected labels gives the following result: 0.475 (avg), 0.504 (a) | 0.503 (b) | 0.491 (f) | 0.403 (g) which would mean rank 9.

1. Zhang Haihong, Institute for Infocomm Research, Singapore

with Kai Keng Ang, Guan Cuntai, Wang Chuanchu, Chin Zheng Yang
Clustering after PCA or ICA to get a rest class; CSP on multiple bands; mutual information based feature selection; radial basis function based neural network
some details [ txt ]

2. Dieter Devlaminck, University of Ghent; Psychiatric Institute of Guislain; University Hospital Ghent

with Willem Waegeman, Bart Wyns, Bruno Bauwens, Luc Boullart, Georges Otte, Patrick Santens
Band-pass filtering, multiclass CSP, multiclass SVM with ordinal regression, smoothing
some details [ txt ]

3. Kai Keng Ang, Institute for Infocomm Research, Singapore

with Zheng Yang Chin, Cuntai Guan, Haihong Zhang
CSP on multiple bands; mutual information based feature selection; Naive Bayes Parzen Window classifier
some details [ pdf ]

4. Jun Lv, South China University of Technology, Guangzhou, China

with Meichun Liu, Kun Cai
CSP-based method
some details [ pdf ]

5. Liu Guangquan, Shanghai Jiao Tong University, China

with Huang Gan, and Zhu Xiangyang
CSP in frequency bands that were optimized by cross-validation; log-variance; LDA
some details [ txt ]

6. Abdul Satti, University of Ulster

with Damien Coyle, Ekta Makhija and Girijesh Prasad
Selection of frequency bands by particle swarm optimization; selection of time intervals by corss-validation; CSP with specific filter selection; log-variance; LDA; smoothing
some details [ pdf ]

7. Fabian Bachl, Team PhyPa, TU Berlin

with Christian Kothe, Thorsten Zander
Two CSP-based classifiers: (a) discrimination between the two motor imagery classes, (b) discrimination of motor imagery from idle state; classification with restricted Boltzmann Maschines, logistic regression and LDA
some details [ pdf ]

8. Jinyi Long, South China University of Technology, Guangzhou, China

with Yuanqing Li, Zhunliang Yu
CSP-based method
some details [ pdf ]

9. Yunjun Nam, Pohang University of science and technology, Korea

with Seungjin Choi
CSP-based method
some details [ txt ]

10. Emily Mankin, University of California, San Diego

with Paul Hammon, Walter Talbott
Four classifers combined by a meta classifier; 4 types of features were AR coefficients, Band power, Raw EEG, Discrete Wavelet transform; all features postprocessed by PCA
some details [ txt ]

11. Cedric Gouy-Pailler, GIPSA-lab, Grenoble Institute of Technology; Dynamique Cérébrale et Cognition Institut Fédératif des Neurosciences - Université Lyon

with Marco Congedo, Chrsitian Jutten, Jérémie Mattout
some details [ pdf ]

12. Jing Jin, Institute for Knowledge Discovery, Graz University of Technology

with Teodoro Solis Escalante, Clemens Brunner, Gert Pfurtscheller
some details [ pdf ]

13. Michael Buschbeck, atip

with Klaus Kasper
some details [ txt ]

14. Teodoro Solis-Escalante, Institute for Knowledge Discovery, Graz University of Technology

with Jing Jin, Clemens Brunner, Gert Pfurtscheller
some details [ pdf ]

15. Chen Guangming, College of Information Science and Technology and National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal Unviersity

with Wu Jin, Zhang Jiacai
some details [ txt ]

16. Yu Huang, Perception-Motor Interaction Laboratory, School of Life Science and Technology, University of Electronic Science and Technology of China

with Dezhong Yao
some details [ pdf ]

17. Astrid Zeman, CSIRO ICT Centre, Australia

10REPtrees, bagging and smoothing on 10 frontal channels
some details [ txt ]

18. Padma Palash, City University of Hong Kong

with Wing-Ho Howard Leung
some details [ txt ]

19. Eric Gottschalk, Topspin

L2 Normalization, Support Vector Machines with polynomial kernel degree 3 or 4
some details [ txt ]

20. Sung Wook, Gwangju Institute of Science and Technology, Korea

with Sung Chan Jun
some details [ pdf ]

21. Manuel Moebius, Hochschule Darmstadt

Hidden Markov Model with 3-5 states, using electrodes C5, C3, Cz, C2, C4
some details [ txt ]

22. Rui Li, Shenyang University of Technology, China

with Li Ke, Peipei Pang, Wei He, Yan Huang
PCA
some details [ txt ]

23. Li Ke, Shenyang University of Technology, China

with Rui Li, Peipei Pang, Wei He, Yan Huang
wavelet transform, SVM
some details [ txt ]

24. Yang Banghua, Shanghai University, China

some details [ pdf ]

[ top ]

Data sets 1 [Berlin: artifically generated data]

Without evaluation, we report the results for the artifical data sets. Since the competitors did not know which data sets had been artificially generated, the methods are the same as above.

#.	contributor	mse	c	d	e	research lab	co-contributors
1.	Dieter Devlaminck	0.281	0.33	0.23	0.28	University of Ghent; Psychiatric Institute of Guislain; University Hospital Ghent	Willem Waegeman, Bart Wyns, Bruno Bauwens, Luc Boullart, Georges Otte, Patrick Santens
2.	Fabian Bachl	0.311	0.43	0.27	0.23	Team PhyPa, TU Berlin	Christian Kothe, Thorsten Zander
3.	Abdul Satti	0.348	0.44	0.29	0.31	University of Ulster	Damien Coyle, Ekta Makhija and Girijesh Prasad
4.	Cedric Gouy-Pailler	0.349	0.41	0.48	0.16	GIPSA-lab, Grenoble Institute of Technology; Dynamique Cérébrale et Cognition Institut Fédératif des Neurosciences - Université Lyon	Marco Congedo, Chrsitian Jutten, Jérémie Mattout
5.	Jun Lv	0.368	0.36	0.55	0.20	South China University of Technology, Guangzhou, China	Meichun Liu, Kun Cai
6.	Zhang Haihong	0.373	0.55	0.49	0.08	Institute for Infocomm Research, Singapore	Kai Keng Ang, Guan Cuntai, Wang Chuanchu, Chin Zheng Yang
7.	Jinyi Long	0.391	0.46	0.40	0.32	South China University of Technology, Guangzhou, China	Yuanqing Li, Zhunliang Yu
8.	Liu Guangquan	0.453	0.51	0.57	0.29	Shanghai Jiao Tong University, China	Huang Gan, and Zhu Xiangyang
9.	Kai Keng Ang	0.471	1.01	0.24	0.17	Institute for Infocomm Research, Singapore	Zheng Yang Chin, Cuntai Guan, Haihong Zhang
10.	Emily Mankin	0.496	0.51	0.52	0.45	University of California, San Diego	Paul Hammon, Walter Talbott
11.	Jing Jin	0.539	0.78	0.43	0.40	Institute for Knowledge Discovery, Graz University of Technology	Teodoro Solis Escalante, Clemens Brunner, Gert Pfurtscheller
12.	Michael Buschbeck	0.594	0.63	0.59	0.56	atip	Klaus Kasper
13.	Yunjun Nam	0.649	0.49	0.32	1.14	Pohang University of science and technology, Korea	Seungjin Choi
14.	(*) Teodoro Solis-Escalante	0.660	0.54	0.52	0.92	Institute for Knowledge Discovery, Graz University of Technology	Jing Jin, Clemens Brunner, Gert Pfurtscheller
15.	Padma Palash	0.759	0.83	0.65	0.79	City University of Hong Kong	Wing-Ho Howard Leung
16.	Yu Huang	0.763	0.92	0.65	0.72	Perception-Motor Interaction Laboratory, School of Life Science and Technology, University of Electronic Science and Technology of China	Dezhong Yao
17.	Sung Wook	0.768	0.80	0.71	0.79	Gwangju Institute of Science and Technology, Korea	Sung Chan Jun
18.	Rui Li	0.839	0.54	0.52	1.47	Shenyang University of Technology, China	Li Ke, Peipei Pang, Wei He, Yan Huang
19.	Eric Gottschalk	0.855	0.87	0.67	1.03	Topspin
20.	Chen Guangming	0.875	0.54	1.51	0.58	College of Information Science and Technology and National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal Unviersity	Wu Jin, Zhang Jiacai
21.	Manuel Moebius	0.923	0.91	0.78	1.08	Hochschule Darmstadt
22.	Astrid Zeman	0.933	0.86	1.00	0.94	CSIRO ICT Centre, Australia
23.	Li Ke	1.166	0.58	1.49	1.43	Shenyang University of Technology, China	Rui Li, Peipei Pang, Wei He, Yan Huang
24.	Yang Banghua	1.201	0.98	1.31	1.31	Shanghai University, China

(*)Remark: After the true labels have been revealed, competitor Teodoro Solis-Escalante realized that he inaccidentally switched the labels (sign mismatch). Evaluation of his submission with sign corrected labels gives the following result: 0.448 (avg) | 0.536 (c) | 0.497 (d) | 0.312 (e) which would mean rank 8.

1. Dieter Devlaminck, University of Ghent; Psychiatric Institute of Guislain; University Hospital Ghent

with Willem Waegeman, Bart Wyns, Bruno Bauwens, Luc Boullart, Georges Otte, Patrick Santens
Band-pass filtering, multiclass CSP, multiclass SVM with ordinal regression, smoothing

2. Fabian Bachl, Team PhyPa, TU Berlin

3. Abdul Satti, University of Ulster

4. Cedric Gouy-Pailler, GIPSA-lab, Grenoble Institute of Technology; Dynamique Cérébrale et Cognition Institut Fédératif des Neurosciences - Université Lyon

with Marco Congedo, Chrsitian Jutten, Jérémie Mattout

5. Jun Lv, South China University of Technology, Guangzhou, China

with Meichun Liu, Kun Cai
CSP-based method

6. Zhang Haihong, Institute for Infocomm Research, Singapore

7. Jinyi Long, South China University of Technology, Guangzhou, China

with Yuanqing Li, Zhunliang Yu
CSP-based method

8. Liu Guangquan, Shanghai Jiao Tong University, China

with Huang Gan, and Zhu Xiangyang
CSP in frequency bands that were optimized by cross-validation; log-variance; LDA

9. Kai Keng Ang, Institute for Infocomm Research, Singapore

with Zheng Yang Chin, Cuntai Guan, Haihong Zhang
CSP on multiple bands; mutual information based feature selection; Naive Bayes Parzen Window classifier

10. Emily Mankin, University of California, San Diego

11. Jing Jin, Institute for Knowledge Discovery, Graz University of Technology

with Teodoro Solis Escalante, Clemens Brunner, Gert Pfurtscheller

12. Michael Buschbeck, atip

with Klaus Kasper

13. Yunjun Nam, Pohang University of science and technology, Korea

with Seungjin Choi
CSP-based method

14. Teodoro Solis-Escalante, Institute for Knowledge Discovery, Graz University of Technology

with Jing Jin, Clemens Brunner, Gert Pfurtscheller

15. Padma Palash, City University of Hong Kong

with Wing-Ho Howard Leung

16. Yu Huang, Perception-Motor Interaction Laboratory, School of Life Science and Technology, University of Electronic Science and Technology of China

with Dezhong Yao

17. Sung Wook, Gwangju Institute of Science and Technology, Korea

with Sung Chan Jun

18. Rui Li, Shenyang University of Technology, China

with Li Ke, Peipei Pang, Wei He, Yan Huang
PCA

19. Eric Gottschalk, Topspin

L2 Normalization, Support Vector Machines with polynomial kernel degree 3 or 4

20. Chen Guangming, College of Information Science and Technology and National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal Unviersity

with Wu Jin, Zhang Jiacai

21. Manuel Moebius, Hochschule Darmstadt

Hidden Markov Model with 3-5 states, using electrodes C5, C3, Cz, C2, C4

22. Astrid Zeman, CSIRO ICT Centre, Australia

10REPtrees, bagging and smoothing on 10 frontal channels

23. Li Ke, Shenyang University of Technology, China

with Rui Li, Peipei Pang, Wei He, Yan Huang
wavelet transform, SVM

24. Yang Banghua, Shanghai University, China

[ top ]

Data sets 2a [Graz]

The performance measure is kappa value. The first column shows the average across all subjects, columns 2 to 10 show the results for the individual subjects.
-> Note: The expected kappa value, if classification is made by chance, is 0. <-

#.	contributor	kappa	1	2	3	4	5	6	7	8	9	research lab	co-contributors
1.	Kai Keng Ang	0.57	0.68	0.42	0.75	0.48	0.40	0.27	0.77	0.75	0.61	Institute for Infocomm Research, Agency for Science, Technology and Research Singapore	Zheng Yang Chin, Chuanchu Wang, Cuntai Guan, Haihong Zhang, Kok Soon Phua, Brahim Hamadicharef, Keng Peng Tee
2.	Liu Guangquan	0.52	0.69	0.34	0.71	0.44	0.16	0.21	0.66	0.73	0.69	School of Mechanical Engineeing, Shanghai Jiao Tong University, China	Huang Gan, Zhu Xiangyang
3.	Wei Song	0.31	0.38	0.18	0.48	0.33	0.07	0.14	0.29	0.49	0.44	College of Information Science and Technology, Beijing Normal University, China and National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal University, China	Jin Wu, Jiacai Zhang
4.	Damien Coyle	0.30	0.46	0.25	0.65	0.31	0.12	0.07	0.00	0.46	0.42	Intelligent Systems Research Centre, School of Computing and Intelligent Systems, Faculty of Computing and Engineering, Magee Campus, University of Ulster, UK	Abdul Satti, Martin McGinnity
5.	Jin Wu	0.29	0.41	0.17	0.39	0.25	0.06	0.16	0.34	0.45	0.37	National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal University, China and College of Information Science and Technology, Beijing Normal University, China	Guangming Chen, Wei Song, Jiacai Zhang

1. Kai Keng Ang, Institute for Infocomm Research, Agency for Science, Technology and Research Singapore

with Zheng Yang Chin, Chuanchu Wang, Cuntai Guan, Haihong Zhang, Kok Soon Phua, Brahim Hamadicharef, Keng Peng Tee
Preprocessing: Artifact removal with bandpass filter. Features: Filter Bank CSP (FBCSP) using multiple one-against-the-rest classifiers. Classification: Naive Bayes Parzen Window classifier.
some details [ pdf ]

2. Liu Guangquan, School of Mechanical Engineeing, Shanghai Jiao Tong University, China

with Huang Gan, Zhu Xiangyang
Preprocessing: CSP on bandpass-filtered data (between 8-30Hz). Features: Log variance of best eight components. Classification: LDA and Bayesian classifier.
some details [ txt ]

3. Wei Song, College of Information Science and Technology, Beijing Normal University, China and National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal University, China

with Jin Wu, Jiacai Zhang
Preprocessing: Downsampling, EOG removal with linear regression, bandpass filter between 8-25Hz and select channels based on a recursive channel elimination. Features: CSP. Classification: Ensemble multi-class classifier using three SVM classifiers with two hierarchies and combination through voting strategy.
some details [ txt ]

4. Damien Coyle, Intelligent Systems Research Centre, School of Computing and Intelligent Systems, Faculty of Computing and Engineering, Magee Campus, University of Ulster, UK

with Abdul Satti, Martin McGinnity
Preprocessing: CSP on spectrally filtered Neural Time Series Prediction Preprocessing (NTSPP) signals. Features: Log variance of each filtered channel is calculated with a one second sliding window. Classification: The best classifier among 3 variants of LDA and 2 variants of SVM was chosen for each subject individually.
some details [ pdf ]

5. Jin Wu, National Key Laboratory for Cognitive Neuroscience and Learning, Beijing Normal University, China and College of Information Science and Technology, Beijing Normal University, China

with Guangming Chen, Wei Song, Jiacai Zhang
Preprocessing: Downsampling, selection of electrodes around C3 and C4, bandpass filter between 8-25Hz, EOG removal with regression. Features: CSP. Classification: Ensemble multi-class classifier using three SVM classifiers with two hierarchies.
some details [ txt ]

[ top ]

Data sets 2b [Graz]

#.	contributor	kappa	1	2	3	4	5	6	7	8	9	research lab	co-contributors
1.	Zheng Yang Chin	0.60	0.40	0.21	0.22	0.95	0.86	0.61	0.56	0.85	0.74	Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore	Kai Keng Ang, Chuanchu Wang, Cuntai Guan, Haihong Zhang, Kok Soon Phua, Brahim Hamadicharef, Keng Peng Tee
2.	Huang Gan	0.58	0.42	0.21	0.14	0.94	0.71	0.62	0.61	0.84	0.78	School of Mechanical Engineeing, Shanghai Jiao Tong University, China	Liu Guangquan, Zhu Xiangyang
3.	Damien Coyle	0.46	0.19	0.12	0.12	0.77	0.57	0.49	0.38	0.85	0.61	Intelligent Systems Research Centre, School of Computing and Intelligent Systems, Faculty of Computing and Engineering, Magee Campus, University of Ulster, UK	Abdul Satti, Martin McGinnity
4.	Shaun Lodder	0.43	0.23	0.31	0.07	0.91	0.24	0.42	0.41	0.74	0.53	E+E Engineering, University of Stellenbosch, South Africa	Johan du Preez
5.	Jaime Fernando Delgado Saa	0.37	0.20	0.16	0.16	0.73	0.21	0.19	0.39	0.86	0.44	Robotica y Sistemas Inteligentes, Universidad del Norte, Colombia
6.	Yang Ping	0.25	0.02	0.09	0.07	0.43	0.25	0.00	0.14	0.76	0.47	Perception-Motor Interaction Lab, School of Life Science and Technology, University of Electronic Science and Technology, China	Xu Lei, Yao Dezhong

1. Zheng Yang Chin, Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore

with Kai Keng Ang, Chuanchu Wang, Cuntai Guan, Haihong Zhang, Kok Soon Phua, Brahim Hamadicharef, Keng Peng Tee
Preprocessing: Artifact removal with bandpass filter. Features: Filter Bank CSP (FBCSP) using multiple one-against-the-rest classifiers. Classification: Naive Bayes Parzen Window classifier.
some details [ pdf ]

2. Huang Gan, School of Mechanical Engineeing, Shanghai Jiao Tong University, China

with Liu Guangquan, Zhu Xiangyang
Preprocessing: Bandpass filtering in different frequency band, EOG removal. Features: Common spatial subspace decomposition (CSSD). Classification: LDA.
some details [ pdf ]

3. Damien Coyle, Intelligent Systems Research Centre, School of Computing and Intelligent Systems, Faculty of Computing and Engineering, Magee Campus, University of Ulster, UK

4. Shaun Lodder, E+E Engineering, University of Stellenbosch, South Africa

with Johan du Preez
Preprocessing: N/A. Features: Wavelet packet transform using only C3 and C4. Selected frequency bands were extracted and concatenated to form a multidimensional vector. Classification: LDA.
some details [ txt ]

5. Jaime Fernando Delgado Saa, Robotica y Sistemas Inteligentes, Universidad del Norte, Colombia

Preprocessing: EOG removal with linear regression, highpass filter with 4Hz. Features: Spectral features in the mu and beta bands (from C3 and C4). Classification: Neural network.
some details [ txt ]

6. Yang Ping, Perception-Motor Interaction Lab, School of Life Science and Technology, University of Electronic Science and Technology, China

with Xu Lei, Yao Dezhong
Preprocessing: EOG removal using regression analysis. Features: Band power features in 75 frequency bands for each channel, recursive feature elimination (RFE). Classification: Bayesian LDA.
some details [ txt ]

[ top ]

Data sets 3 [Freiburg, Tübingen]

The performance measure is accuracy. The first column shows the overall accuracy for both subjects, columns 2 and 3 show the results for the individual subjects.
-> Note: The expected accuracy, if classification is made by chance, is 25\%. <-

#.	contributor	acc	S1	S2	research lab	co-contributors
1.	Sepideh Hajipour	46.9	59.5	34.3	Biomedical Signal and Image Processing Laboratory (BiSIPL), School of Electrical Engineering, Sharif University of Technology, Tehran, Iran	Mohammad Bagher Shamsollahi
2.	Jing Li	25.1	31.1	19.2	Institute of Biomedical Engineering of Yanshan University, Qinhuangdao, China	Wenxue Hong, Jialin Song, Yonghong Xu, Xin Li
3.	Nasim Montazeri	23.9	16.2	31.5	Electrical Engineering Department, Sharif University of Technology, Tehran, Iran	Mohammad Bagher Shamsollahi
4.	Jinjia Wang	20.4	23.0	17.8	Institute of Biomedical Engineering of Yanshan University, Qinhuangdao, China	Tao Zhang

1. Sepideh Hajipour, Biomedical Signal and Image Processing Laboratory (BiSIPL), School of Electrical Engineering, Sharif University of Technology, Tehran, Iran

with Mohammad Bagher Shamsollahi
Preprocessing/Features: Feature Extraction: Some statistical features, frequency domain features and wavelet coefficients were extracted from all channels (including 10 real channels and 2 artificial bipolar channels). Feature Reduction: First, the features were reduced using a supervised unrelated to classifier algorithm. Then, genetic algorithm was used to find appropriate features related to the classifier. Classification: combination of linear SVM and LDA

2. Jing Li, Institute of Biomedical Engineering of Yanshan University, Qinhuangdao, China

with Wenxue Hong, Jialin Song, Yonghong Xu, Xin Li
Preprocessing: low-pass filter (0.5-8 Hz), selected time segment: 0-0.5s, subsampling to 20 Hz. Features:time feature were used with FDA for reduced dimension, first three and five pca of the abs and angle of the 128 FFT of each channel and each sample. Then frequency feature were used with FDA for reduced dimension. Classification: fisher discriminant functions

3. Nasim Montazeri, Electrical Engineering Department, Sharif University of Technology, Tehran, Iran

with Mohammad Bagher Shamsollahi
Preprocessing: unknown Features: Feature extraction consist of statistical, temporal, parametric and wavelet coefficients. Feature reduction using PCA and genetic algorithm respectivly. Classification: linear SVM

4. Jinjia Wang, Institute of Biomedical Engineering of Yanshan University, Qinhuangdao, China

with Tao Zhang
Preprocessing: low-pass filter (0.5-8 Hz), selected time segment: 0-0.5s time feature were used with FDA for reduced dimension. Features: first three and five pca of the abs and angle of the 128 FFT of each channel and each sample. Then them were used with FDA for reduced dimension. Classification: fisher discriminant functions

[ top ]

Data sets 4 [Washington, Albany]

The performance measure is the correlation coefficient r between the actual and the predicted finger flexions, averaged across all subject and finger (except for finger 4), see description for more details.
-> Note: The expected correlation, if classification is made by chance, is 0. <-

#.	contributor	r	research lab	co-contributors
1.	Nanying Liang	0.46	Cortex Team, Research Centre INRIA, France	Laurent Bougrain
2.	Remi Flamary	0.42	LITIS INSA de Rouen, France	Alain Rakotomamonjy
3.	Mathew Salvaris	0.27	University of Essex, Colchester, UK
4.	Florin Popescu	0.10	Fraunhofer First, Germany
5.	Hyunjin Yoon	0.05	University of Southern California, Los Angeles, USA	Cyrus Shahabi

1. Nanying Liang, Cortex Team, Research Centre INRIA, France

with Laurent Bougrain
Features: Amplitudes in 1-60Hz, 60-100Hz, and 100-200Hz band; pair-wise feature selection
Classification:Linear regression using Wiener solution and 25 tap delays.

2. Remi Flamary, LITIS INSA de Rouen, France

with Alain Rakotomamonjy
Preprocessing:Downsampling by a factor of 4.Features: Savitsky-Golay filter (0.4 s , 3rd order) and AR coefficients on a moving window of 300 points.
Classification: Ridge regression and sparse linear regression.

3. Mathew Salvaris, University of Essex, Colchester, UK

Preprocessing:Common Average Reference, downsampling to 500 Hz.Features:Bandpower calculated using wavelet packets, and average of time series. CFS/Weka feature selector.
Classification:SVR/LibSVM.

4. Florin Popescu, Fraunhofer First, Germany

Not described

5. Hyunjin Yoon, University of Southern California, Los Angeles, USA

with Cyrus Shahabi
Preprocessing:Rejection of bad channels, Common Average ReferenceFeatures:AR coefficients (order 2); time-domain moving average of 0-3Hz bandpass; PSD (2Hz resolution) of 8-32 Hz bandpass
Classification:DAGSVM.

[ top ]

Competition Winners

data set	research lab	contributor
1	Institute for Infocomm Research, Singapore	Zhang Haihong, Kai Keng Ang, Guan Cuntai, Wang Chuanchu, Chin Zheng Yang
2a	Institute for Infocomm Research, Singapore	Kai Keng Ang, Zheng Yang Chin, Chuanchu Wang, Cuntai Guan, Haihong Zhang, Kok Soon Phua, Brahim Hamadicharef, Keng Peng Tee
2b	Institute for Infocomm Research, Singapore	Zheng Yang Chin, Kai Keng Ang, Chuanchu Wang, Cuntai Guan, Haihong Zhang, Kok Soon Phua, Brahim Hamadicharef, Keng Peng Tee
3	Biomedical Signal and Image Processing Laboratory (BiSIPL), Sharif University of Technology, Tehran, Iran	Sepideh Hajipour, Mohammad Bagher Shamsollahi
4	Cortex Team, Research Centre INRIA, France	Nanying Lian, Laurent Bougrain

Winners are asked to prepare an article for publication in a volume devoted to this competition. We will contact the winners to give them more details on this.

[ top ]

True Labels of Competition's Evaluation Sets

Data sets 1: [ MAT format | [ TXT format ]
Data sets 2a: [ labels ]
Data sets 2b: [ labels ]
Data sets 3: [ labels ]
Data sets 4: [ labels (20 MB!) ]

[ top ]

Organizers

Berlin: Benjamin Blankertz, Carmen Vidaurre, Michael Tangermann, Klaus-Robert Müller

Graz: Clemens Brunner, Robert Leeb, Gernot Müller-Putz, Alois Schlögl, Gert Pfurtscheller

Freiburg/Tübingen: Stephan Waldert, Carsten Mehring, Ad Aertsen, Niels Birbaumer

Washington/Albany: Kai J. Miller, Gerwin Schalk

[ top ]

Contact:
Benjamin Blankertz ‹benjamin.blankertz@tu-berlin.de›
Berlin Institute of Technology
Machine Learning Laboratory
Franklinstr. 28/29
10587 Berlin
and
Fraunhofer FIRST (IDA)
Kekulé Str. 7, D-12489 Berlin, Germany