Unleashing the power of hyperparameter optimization for machine and deep learning models

Moustafa, Mohamed (2023) Unleashing the power of hyperparameter optimization for machine and deep learning models. Masters thesis, Memorial University of Newfoundland.

[img] [English] PDF - Accepted Version
Available under License - The author retains copyright ownership and moral rights in this thesis. Neither the thesis nor substantial extracts from it may be printed or otherwise reproduced without the author's permission.

Download (5MB)


Hyperparameter optimization is a crucial aspect for improving the performance of learning models through tuning their hyperparameters. This thesis introduces three effective techniques for tuning hyperparameters in various models: PointNet CNN, regularized and non-regularized standard feedforward neural networks, SVM, and PCA. The techniques employed are random search-based tuning, hybrid random and grid search-based tuning, and hybrid random and manual search-based tuning. The random search is performed through coarse and fine-tuning stages, while hybrid approaches combined the random search’s benefits with grid or manual search. These techniques achieve competitive results while optimizing computation costs regarding time and storage usage, making them valuable for state-of-the-art work. Random search and hybrid random and grid search enhance PointNet to achieve high classification accuracy that surpasses related research with an average F1-score of 93.6%, while the hybrid random and manual search reduces computational time but results in a lower accuracy of 90.97%. The random search-based tuning and hybrid random and grid search achieve an accuracy of 96.67% for the make_moons dataset. Similarly, the hyperparameter tuning techniques lead to a binary classification accuracy of 97% for the SVM model, and they identify the minimum number of PCA components and retain 99.0042% of the original dataset’s variance for a set of human face images.

Item Type: Thesis (Masters)
URI: http://research.library.mun.ca/id/eprint/16213
Item ID: 16213
Additional Information: Includes bibliographical references (pages 178-184)
Keywords: deep learning; random search; hybrid random and grid search; hybrid random and manual search
Department(s): Engineering and Applied Science, Faculty of
Date: August 2023
Date Type: Submission
Digital Object Identifier (DOI): https://doi.org/10.48336/CGMQ-ZY87
Library of Congress Subject Heading: Deep learning (Machine learning); Principal components analysis; Data structures (Computer science)

Actions (login required)

View Item View Item


Downloads per month over the past year

View more statistics