Cantoral Ceballos, José AntonioRomo Chavero, María Andrea2024-12-302024-12Romo Chavero, M. A. (2024). Enhancing BGP security with MAD anomaly detection system and machine learning techniques. [Tesis maestría]. Instituto Tecnológico y de Estudios Superiores de Monterrey.https://hdl.handle.net/11285/702954https://doi.org/10.60473/ritec.30https://orcid.org/0000-0001-5597-939XAnomalies in the Border Gateway Protocol (BGP) represent a signicant vulnerability in the Internet’s infrastructure, as they can cause widespread disruptions, trafc misdirection, and even security breaches. Proactive detection of these anomalies is vital to preserving network stability and preventing potential cyberattacks. In response to this challenge, we present the Median Absolute Deviation (MAD) anomaly detection system, which combines traditional statistical methods with advanced machine learning (ML) techniques for more precise and dynamic detection. Our approach introduces a novel adaptive threshold mechanism, allowing the system to adjust based on the changing conditions of network trafc. This dynamic thresholding signif- icantly improves the accuracy, precision, and F1-score of anomaly detection compared to the previous xed-threshold version. Additionally, we integrate the MAD system with a diverse ML classiers, including Random Forest, XGBoost, LightGBM, CatBoost, and ExtraTrees to enhance the system’s ability to identify complex patterns that indicate unusual BGP behavior.We evaluate our detection system on well-documented BGP anomaly events, such as the Slammer worm, Nimda, Code Red 1 v2, the Moscow blackout, and the Telekom Malaysia misconguration. The results show that our system when combined with ML models achieves an overall accuracy and F1-score of 0.99, demonstrating its effectiveness across various anomaly types. By using both statistical and ML models, the system is able to capture irregularities that could signal security threats, offering a more comprehensive detection solution.This research highlights the importance of combining statistical anomaly detection with ML to obtain a balance between accuracy and computational efciency. The system’s low resource requirements and minimal pre-processing make it highly scalable, allowing it to be potentially deployed in real-time on large-scale networks.TextoengopenAccesshttp://creativecommons.org/licenses/by-nc-sa/4.0INGENIERÍA Y TECNOLOGÍA::CIENCIAS TECNOLÓGICAS::TECNOLOGÍA DE LOS ORDENADORES::SISTEMAS EN TIEMPO REALCIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA::MATEMÁTICAS::CIENCIA DE LOS ORDENADORES::SISTEMAS EN TIEMPO REALTechnologyTesis de Maestría / master ThesisExiste una publicación en revisión.https://orcid.org/0009-0002-5224-1343Border Gateway ProtocolMedian Absolute Deviation (MAD)Anomaly DetectionBGP SecurityMachine LearningDynamic ThresholdingNetwork StabilityRouting ProtocolsCybersecurityTraffic Analysis1276195