SEMBO: Semantic Memory-Based Warm-Starting for Hyperparameter Optimization with Local Large Language Models SEMBO: Yerel Büyük Dil Modelleri ile Hiperparametre Optimizasyonu Için Anlamsal Bellek Tabanli Sicak Baslangiç
34th Signal Processing and Communications Applications Conference, SIU 2026, İstanbul, Türkiye, 7 - 10 Temmuz 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/siu71813.2026.11636633
- Basıldığı Şehir: İstanbul
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: FAISS, hyperparameter optimization, large language model (LLM), meta-learning, RAG, warm-starting
- Karadeniz Teknik Üniversitesi Adresli: Evet
Özet
In this paper, the SEMBO framework, which accelerates hyperparameter optimization using a large language model-based agent architecture, is proposed. The system is composed of a meta-learning component that stores past experiences in a FAISS vector memory, a Retrieval-Augmented Generation (RAG) module, and a locally running Llama 3.1:8B model. Rather than a fixed point injection, a trust region based on semantic similarities is defined to narrow the search space, and Optuna continues Bayesian optimization within these boundaries. The proposed system is evaluated on three models spanning the gradient boosting and bagging families, using 12 training and 5 test datasets with multiple seeds. It is shown that convergence speedup is model-dependent. On LightGBM and Random Forest, statistically significant speedup is achieved as confirmed by the Wilcoxon signed-rank test, with median speedups determined as 1.40 and 2.02-fold, respectively.