A RAG-enhanced multi-agent LLM/VLM framework for auditable and region-aware structural health monitoring of a historic masonry structure

Creative Commons License

HACIEFENDİOĞLU K., KAHYA V., Aslan T.

Scientific Reports, cilt.16, sa.1, 2026 (SCI-Expanded, Scopus)

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 16 Sayı: 1
  • Basım Tarihi: 2026
  • Doi Numarası: 10.1038/s41598-026-57424-2
  • Dergi Adı: Scientific Reports
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, BIOSIS, Chemical Abstracts Core, EMBASE, MEDLINE, Directory of Open Access Journals, Zoological Record, Academic Search Ultimate (EBSCO), Natural Science Collection (ProQuest), Biological Science Database (ProQuest), Biomedical Reference Collection: Corporate Edition (EBSCO), Health Research Premium Collection (ProQuest)
  • Anahtar Kelimeler: Crack detection, Masonry, Minarets, Multi-agent systems, Multi-camera, Segment Anything Model (SAM), Structural health monitoring, Vision-based monitoring, Vision–language models
  • Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
  • Karadeniz Teknik Üniversitesi Adresli: Evet

Özet

Historic masonry structures require monitoring workflows that are non-destructive, interpretable, and robust to field variability. This paper presents a proof-of-concept, RAG-enhanced structural health monitoring (SHM) workflow demonstrated on a single historic masonry minaret case study, in which a manager–orchestrator coordinates signal analysis, visual analysis, and cross-modal fusion. Vibration data from four sensors installed on this minaret are processed through a reproducible pipeline that includes data cleaning, PSD-based baseline characterisation, adaptive windowing, robust normalisation, and anomaly screening with a 1D convolutional autoencoder trained on healthy reference data (Measurements 1–12), followed by evaluation on Measurements 13–20. Localisation is strengthened through relation-based autoencoders on sensor pairs and sensor-to-zone damage mapping. In parallel, a VLM analyses sensor-linked inspection images from the same case study, while a curated retrieval-augmented structural memory package for the monitored minaret (geometry, materials, baseline dynamics, FE notes, and inspection playbook) supports context-aware interpretation. A three-agent decision layer (Agent A: LLM+RAG, Agent B: VLM+RAG, Agent C: fusion + RAG) reconciles agreements and conflicts between signal- and image-derived evidence to produce case-specific global and zone-level risk assessments. The study compares with-RAG and without-RAG analysis modes within this single deployment and reports structured, traceable outputs, including JSON artefacts, per-image findings, per-sensor summaries, narrative reports, and final risk maps. Its main contribution is to demonstrate how signal-derived anomalies, image-derived evidence, and retrieval-grounded structural memory can be combined within a traceable multi-agent SHM workflow for one monitored historic masonry minaret.