<b>Application of machine learning techniques for detection and filtering of fileless malware</b>

Abstract

This work presents a fileless malware filter capable of detecting both normal and perturbed malware programs using machine learning techniques. The study implements two filter algorithms based on back-propagation and gradient descent, respectively. The development methodology employed includes the Top-Down Design Methodology (TDDM) and Object-Oriented Analysis and Design Methodology (OOADM). The system development involved data collection, data extraction, artificial neural networks, activation functions, training, and filter-based classification. Results showed that the average Mean Squared Error (MSE) was 0.002088 and the average Regression (R) was 0.9931 using the Gradient Descent Optimization algorithm. Conversely, the average R was 0.96366, and the MSE was 0.01909 using backpropagation. The results indicate that GDA outperforms backpropagation in both metrics, as it achieved lower MSE and higher R values, demonstrating that the GDA algorithm provides more accurate predictions with a stronger correlation between actual and predicted values.