Optimization of neural network hyperparameters using metaheuristic algorithms

This thesis compares four hyperparameter search methods for tuning a compact convolutional neural network on CIFAR-10 under a uniform setup. The methods are grid search and three population-based metaheuristics, particle swarm optimization, genetic algorithms, and grey wolf optimization. The search space is limited to learning rate and dropout to isolate the effect of the search strategy. Each run has a budget of 50 model evaluations. All evaluations use the same protocol with 5 epochs, batch size 64 and a fixed stratified split. The study assesses three aspects of performance. First, the best validation accuracy reached within budget. Second, evaluation efficiency measured by the index of the first occurrence of the best score. Third, runtime reported as time to best and time to 95 percent of best. The configuration that yields the best validation score is retained on the full training set and then evaluated once on the test set. The methods achieve very similar best validation accuracies. The metaheuristics typically reach near best scores early, which appears as a shorter time to 95 percent of best. Grid search progresses in discrete steps and on average reaches the eventual best earlier and with lower variability. Test accuracy mirrors validation without large gaps. Metaheuristics are a practical choice when fast good solutions are needed, while a compact grid remains a strong baseline when the full budget is used. The study is limited by a narrow search space, short training, and CPU-only execution.