The Day We Built the Machine

This day was quite the monument. The entire experimental framework was built from the ground up with MLFlow (for tracking purposes) and Optuna (for hyperparameter tuning in mind). Before this point, we had a collection of somewhat scattered and brittle scripts which needed to be turned into a proper, configurable pipeline, a system that allows for serious and systemic research work.

Configuration is King

The largest issue with the scripts was the hardcoded parameters. The ability to rapidly iterate on hyperparameters, data paths, preprocessing, and model settings is quite important in research settings. Changing values directly in the code is utterly disastrous and does not lend itself well to tracking beyond git, leading to error and lost experiment parameters.

The solution was a centralized config.yaml file that became our single source of truth for the entire pipeline.

A brief look at a section of config.yaml

This meant that reproducing experiments became trivial. Now anyone could tweak the learning rate, change the target dataset, or adjust filtering parameters without ever touching the core Python code. Simply save the config file alongside the results, and anyone could recreate the exact same experimental conditions.

First Successful Architecture

By July 23rd, we already had a working 1D CNN architecture showing promising results, developed by Malakalnawar. The model was relatively simple:

EEGNet v0

The architecture processed 22-channel EEG data with 500 time points per epoch, ultimately classifying segments as either clean or containing artifacts.

The results were quite encouraging, it achieved approximately 85% accuracy on the test set, with balanced precision and recall across both classes.

Results of the first ever EEGNet run

What was particularly of interest was the model's training behavior, as it seemed to only learn effectively between epochs 20-30, suggesting that the optimization landscape required substantial warmup before convergence. This will be important later...

Data Processing Complications

One of the obstacles we came across was the heterogeneity of the TUH dataset. The corpus contains recordings with sampling rates varying from 250 Hz, 256 Hz, 1280 Hz, different montage configurations and inconsistent channels. Some recordings had the full complement of 22 channels we expected, while others were missing electrodes.

Our preprocessing pipeline had to be robust enough to handle these variations automatically. We standardized all recordings to 250 Hz through resampling, using a frequency that captures the relevant content for artifact detection while adhering to dataset limitations.

We resample the EMG to 512 Hz instead of 256 Hz, because the EMG signal is concentrated in the high frequency range, so a higher sampling rate is required (according to the Nyquist sampling theorem).

- IOPscience

The pipeline also included dynamic montage detection to handle different electrode configurations and proper bipolar montage calculation when the necessary reference electrodes were available. The epoch generation process had to carefully handle overlapping annotations to ensure that any epoch containing even a partial artifact received the appropriate label.

Perhaps most importantly data splitting was implemented on a per-file basis to both maintain temporal integrity and prevent data leakage. With time series data, especially EEG, adjacent samples (or in our case, epochs from the same recording) are highly correlated. If the epochs are randomly shuffled and split into training and validation sets, the model essentially gets to "peek" at the validation data. It is conceivable that the model could learn the "fingerprint" of a specific patient rather than generalizable patterns of an artifact albeit I was doubtful that the model was complex enough to learn this. This was solved by partitioning a list of all raw EDF files, then splitting THIS list into train, validation, and test sets.

Finally, Training

The first couple epochs had some interesting dynamics. The model would initially predict only one class for all inputs, essentially just predicting the majority class (even though our noise to clean ratio was close to 50 50). It would often alternate between predicting all clean and all artifact, as if it was trying to cheat the training.

Confusion Matrices for Epochs 0 and 2

This behavior would largely disappear once training extended beyond 20 epochs with proper learning rate scheduling implemented.

Final Results of the Training Run

Looking Forward

By the end of this day we had a solid foundation for future experiments. The codebase refactor resulted in a clean separation between data processing, training, instrumentation, and experiment tracking. But most importantly, we had a working baseline that we could systematically improve on. The 85% accuracy was promising, but there was clear room for architectural improvements, better hyperparameter tuning, perhaps even data improvements...