SUMMARY OF RESULTS ****************** The words obtained from session 12 are: __________________ Block Word __________________ aas012r01: FOOD aas012r02: MOOT aas012r03: HAM aas012r04: PIE aas012r05: CAKE aas012r06: TUNA aas012r07: ZYGOT aas012r08: 4567 __________________ This result is also attached as an ASCII file named results.dat. The same result was also obtained with as few as 6 illuminations: _____________________________________________ Number of illuminations: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 _____________________________________________ Number of errors out of 31 characters: 11 5 2 1 1 0 0 0 0 0 0 0 0 0 0 _____________________________________________ DETAILED DESCRIPTION ******************** ########################################################### Continuous Wavelet Transform (CWT), Scalogram Peak Detection, and Linear Discriminant Analysis (LDA) with Stepwise or Optimal Selection of Variables ########################################################### I. Method ========= I.1. Epochs ----------- One second long overlapping epochs beginning at stimulus onset were extracted from the continuous signal. The signal in each epoch was DC corrected. EpochLength = 1000 ms. I.2. Log Scale -------------- The signal was resampled on a logarithmic scale -- the new sample rate was a linear function of time; the distance between two adjacent time samples was increased linearly. The pertinent parameter was ScaleDilation per second (SDps) and it was set: SDps = 3. I.3. Time Domain Filtering -------------------------- The signal in each trial and each channel was low-pass filtered by convolution with the function: exp( -(4*time/Scale)^2 ) * cos(2*pi*time/Scale). Frequencies higher than 2/Scale vanished, frequencies between 1/Scale and 2/Scale were attenuated, and frequencies lower than 1/Scale remained unchanged. Because of the log sampling, the effect was as if Scale had been dilated linearly with time and the filtering frequency had been reduced SDps times every second. Scale was set: Scale = 100 ms. This means that in the beginning of the epoch the filtering frequency was 10 Hz and at the end of the epoch -- 3.33 Hz. This procedure takes into account the fact that late ERP components usually have longer duration than earlier waves. Thus a more economical presentation of the signal was obtained. I.4. CWT -------- The signal from each channel and each single trial epoch was CWTransformed for scales ranging from Scale/2 to 4*EpochLength. The scalogram was sampled on a logarithmic grid allowing scale invariant resolution of 12*12 points per scale per octave. This means that a peak in the scalogram was always represented by approximately 12*12 time-scale samples regardless of its width. I.5. ERP Component Identification -- Peak Detection --------------------------------------------------- Student's two-sample t-value was calculated for each time-scale point of the CWT in each channel. ERP components were defined as local extrema in the thus obtained t-value scalograms. These were the points of maximal difference (variance taken into account) between the two experimental conditions. I.5.a. Spacial Extrema ...................... In a simplified version of the method, extrema were defined not only in time and scale but also in the spacial dimensions (maximal amplitude over all scalp positions). In this case, the data set was reduced dramatically (by a factor close to the number of channels) and the obtained variables had a straight-forward interpretation as ERP components. This method has also another, practical advantage -- only few selected electrodes are required for further spelling practice. I.6. LDA and Stepwise Selection of Variables -------------------------------------------- The thus extracted components were subject to LDA. Redundant variables were eliminated by a stepwise procedure. First, the discriminant function was calculated with all N components and the HitScore ( = number of correctly classified trials from the training data set) was computed and stored. Then, all (N-1)- component subsets were checked in the same manner and the one that maximized the HitScore was selected. This was repeated until all components were exhausted. The subset with the highest HitScore was selected. Then the eliminated variables were added back one at a time and always the one whose addition maximized the HitScore was selected for inclusion in the subset. Eventually, the subset with the largest HitScore was chosen. The discriminant function corresponding to this subset was finally applied on the test data. I.6.a. Optimal Selection (Checking All Subsets) ............................................... If the number N of original variables was small enough (N<25), e.g. in the case of spacial extrema (see I.5.a.), selection was performed by checking all 2^N - 1 subsets. Again, the subset with the highest HitScore on the training data was selected. I.6.b. No Selection ................... Discrimination with all variables yields very good results as well. It is a reasonable alternative in the case of too many variables when stepwise selection becomes unacceptably time consuming (see II). I.7. Spelling -- Classification ------------------------------- The test data were processed in 180-trial blocks according to the experimental design. The P3-score of each trial was defined as the scalar product of that trial with the coefficient vector of the discriminant function. The P3-score of each row and column was defined as the sum of the P3-scores of all trials in which this particular row or column was illuminated. The row and the column with the highest P3-scores were classified as target row and target column in this block of trials and the corresponding character according to the matrix display was classified as target character in this block. I.7.a Spelling with Fewer Illuminations ....................................... The method was also tested with the first 12, 24, 48, etc. trials of each 180-trial block corresponding to 1, 2, 3, etc. illuminations of each row and column. I.7.b. Correcting Errors in the Training Data ............................................. The first letter of the target word HAT from AAS011R02 was consistently misspelled as "G". Thus the first 180 trials from this file were considered an error and deleted. Indeed, discrimination and classification based on the thus corrected training data set yielded better results, than the procedure using the complete data. I.8. Note --------- The described method does not rely on any assumptions about the kind of the ERP response to target stimuli. In particular, it does not assume a P300 -- ANY significant difference in the ERP waveforms to the two experimental conditions will do. Neither does the method rely on visual inspection of the curves. Indeed, the following results were obtained without such visual inspection. II. Computational Load ====================== All transformation algorithms, especially the CWT, which is the most time consuming transformation, have been optimized for application on large data arrays. On a 704 MHz AMD CPU with 256 MB RAM, the CWT of the 483840 one-second signals ( = 42 characters * 180 trials per character * 64 channels) takes about 20 min (about 2.5 ms per signal). The stepwise procedure with 439 components takes about 13 h; with 547 components it takes 30 h. The HitScore computation for all subsets of variables takes 70 s with 13 components and 320 s with 15 components. The time needed for classification of the test data is negligible, because it involves only matrix multiplication of the data array with the coefficient vector of the discriminant function. III. Results ============ III.1. All Extrema and Stepwise Selection ------------------------------------------ With Scale = 100 ms and SDps = 3, 439 components were detected from which 327 were retained after the stepwise procedure. This is the output from the classification routine p3spell.m: >> p3spell(testlist,discrTeeg); __________________________________________________________________________ Column number: Row number: 1 2 3 4 5 6 1 2 3 4 5 6 __________________________________________________________________________ Normalized column Selected Normalized row Selected Selected scores: column: scores: row: character: aas012r01: 16 8 0 8 23 46 6 52 27 11 0 6 4 1 F 5 15 48 15 0 17 3 6 31 50 14 0 0 3 O 7 22 42 17 11 0 3 17 20 35 20 8 0 3 O 9 1 20 55 16 0 4 47 21 10 9 0 12 1 D aas012r02: 66 26 0 2 7 0 1 1 15 56 18 11 0 3 M 1 13 69 9 7 0 3 9 22 49 15 5 0 3 O 9 18 49 13 0 11 3 9 26 53 9 3 0 3 O 21 50 15 9 5 0 2 0 15 12 50 19 3 4 T aas012r03: 8 68 11 6 0 7 2 23 52 17 1 8 0 2 H 50 23 8 16 0 4 1 62 17 7 11 2 0 1 A 63 13 8 9 7 0 1 22 18 37 19 5 0 3 M aas012r04: 1 9 7 78 5 0 4 20 16 51 13 0 0 3 P 12 18 56 11 0 3 3 30 43 16 10 0 1 2 I 13 0 7 10 51 18 5 50 21 0 11 10 8 1 E aas012r05: 7 20 59 12 2 0 3 47 21 12 10 0 10 1 C 62 12 7 0 11 8 1 70 22 3 5 0 0 1 A 0 5 0 18 61 16 5 21 54 11 0 13 1 2 K 6 0 21 14 48 12 5 55 20 11 8 6 0 1 E aas012r06: 23 51 9 5 0 12 2 2 6 2 71 19 0 4 T 6 32 51 7 0 3 3 18 0 19 54 2 8 4 U 9 66 10 7 0 9 2 6 18 55 20 0 1 3 N 71 13 6 4 6 0 1 70 14 7 0 8 2 1 A aas012r07: 21 66 4 3 6 0 2 5 17 7 20 52 0 5 Z 49 16 17 0 7 12 1 1 13 0 17 53 16 5 Y 76 19 2 0 1 3 1 21 42 20 11 6 0 2 G 14 21 40 16 9 0 3 21 16 40 20 0 2 3 O 29 49 7 9 0 6 2 0 13 19 50 18 0 4 T aas012r08: 1 11 0 20 5 63 6 11 0 6 5 62 16 5 4 46 21 11 9 0 13 1 0 16 2 7 5 70 6 5 24 50 19 5 2 0 2 5 11 0 14 23 48 6 6 20 16 60 4 0 0 3 8 0 7 5 15 64 6 7 __________________________________________________________________________ Block Word __________________ aas012r01: FOOD aas012r02: MOOT aas012r03: HAM aas012r04: PIE aas012r05: CAKE aas012r06: TUNA aas012r07: ZYGOT aas012r08: 4567 __________________ >> This is the output from the function p3spelltest.m, which does the classification with 1, 2, 3, ... 15 illuminations: >> p3spelltest(testlist,discrTeeg); _____________________________________________ Number of illuminations: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 _____________________________________________ Selected character: aas012r01: F F K K L F F F F F F F F F F O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O D D D D D D D D D D D D D D D aas012r02: M M M M M M M M M M M M M M M U O O O O O O O O O O O O O O O O O O O O O O O O O O O O O Z T T T T T T T T T T T T T T aas012r03: B E H H H H H H H H H H H H H A A A A A A A A A A A A A A A M M M M M M M M M M M M M M M aas012r04: P P P P P P P P P P P P P P P I I I I I I I I I I I I I I I D E E E E E E E E E E E E E E aas012r05: C C C C C C C C C C C C C C C A A A A A A A A A A A A A A A G I K K K K K K K K K K K K K E E E E E E E E E E E E E E E aas012r06: T T T T T T T T T T T T T T T C U U U U U U U U U U U U U U H N N N N N N N N N N N N N N A A A A A A A A A A A A A A A aas012r07: Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z T Y Y Y Y Y Y Y Y Y Y Y Y Y B F G G G G G G G G G G G G G N O O O O O O O O O O O O O O T N N T T T T T T T T T T T T aas012r08: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 B 6 6 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 _____________________________________________ Number of errors out of 31 characters: 11 5 2 1 1 0 0 0 0 0 0 0 0 0 0 _____________________________________________ Number of illuminations: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 _____________________________________________ >> III.2. Spacial Exterma and Optimal Selection --------------------------------------------- With Scale = 100 ms and SDps = 3, 13 components were detected from which 11 were retained after checking all subsets: ________________________________________________ Channel Duration [ms] Latency [ms] Polarity ________________________________________________ FC2 80 141 + FC4 156 675 - C1 315 325 + C1 330 144 - C1 917 672 - FP1 273 460 - AF4 61 49 + FT8 143 576 + PO8 133 205 + PO8 232 316 - PO8 368 465 + ________________________________________________ The obtained words were identical with those obtained with the full set of extrema, but the sufficeint minimal number of illuminatain was bigger (10): _____________________________________________ Number of illuminations: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 _____________________________________________ Number of errors out of 31 characters: 15 11 7 3 3 3 1 1 1 0 0 0 0 0 0 _____________________________________________ III.3. Smaller Filtering Scale ------------------------------ The results obtained with filtering Scale = 75 ms were better than the results with Scale = 100 ms, but unfortunately there was no time to perform a test with the corrected training data and Scale = 75 ms. Probably, the minimal sufficient number of illuminations can be reduced further.