This article about machine learning in cybersecurity explains the core elements of machine learning, including the definition, types and challenges. It provides an understanding of the role of machine learning in cybersecurity and guidance on evaluating machine learning models. Also covered in this article is a review of benefits and use cases.
See why AI, ML, and automation are needed to proactively identify risks and help IT teams and business stakeholders make more informed decisions.
What is machine learning?
Machine learning is a subset of artificial intelligence (AI) that allows systems to automatically identify features, classify information, find patterns in data, make determinations and predictions, and uncover insights. Historical data is transmitted to systems that use algorithms to create machine learning models that continuously train the systems to increase accuracy.
The quality of a machine learning model depends on two key aspects:
- The quality of the input data (i.e., garbage in, garbage out
- The algorithm’s alignment with the use case
The choice of algorithm for machine learning models depends on the type of data that is available and the specific task.
Examples of how algorithms are used for machine learning in cybersecurity include:
- Decision tree algorithm—for detecting and classifying attacks
- Dimensionality reduction algorithms—for removing noisy and irrelevant data
- K-means clustering—for detecting malware
- K-nearest neighbors classifier (kNN)—for facial recognition used for authentication
- Linear regression—for predicting network security outcomes
- Logistic regression—for fraud detection
- Naïve Bayes algorithm—for intrusion detection
- Random forest algorithm—for classifying phishing attacks
- Support Vector Machine (SVM) algorithm—for classifying, detecting, and predicting blacklisted IP addresses and port addresses
| Origin of the term “machine learning” |
|---|
| An American scientist, Arthur Samuel, coined the term machine learning in 1959. He defined it as “The field of study that gives computers the capability to learn without being explicitly programmed.” He developed one of the world’s first successful machine-learning programs, the Samuel Checkers-playing Program, which was used to play chequers better than the program’s author. |
Source: Some Studies in Machine Learning Using the Game of Chequers
How machine learning transforms cybersecurity
The ability of machine learning models to process and draw inferences from vast amounts of data is the driving force behind the cybersecurity transformations. Traditional security tools primarily rely on predefined rules and known threat signatures, which limits their ability to detect novel or evolving attacks.
Machine learning is reshaping cybersecurity strategies by enabling adaptive, proactive defences. As threats evolve, so do the models trained to combat them, allowing organisations to shift from reactive to predictive security postures.
Machine learning models continuously learn from vast volumes of data to identify patterns and anomalies in real time. These insights allow organisations to identify new threats, including zero-day attacks, before they can cause significant harm.
In addition to detection, machine learning drives automation in incident response and management. Machine learning in cybersecurity systems can initiate predefined actions (e.g., isolating affected systems or blocking malicious internet protocol (IP) addresses) within seconds of detecting a threat. This minimises the potential damage and helps organisations maintain continuity during an attack. It also helps security teams to prioritise alerts more effectively and reduce the time it takes to investigate and contain threats.
What is the role of AI and machine learning in cybersecurity?
Artificial intelligence (AI) and machine learning have become core components of cybersecurity solutions due to their ability to predict, detect, and respond to threats with unprecedented accuracy and speed. Key functions of AI and machine learning in cybersecurity include:
- Analysing patterns and detecting anomalies that may indicate a security breach or an impending attack
- Automating complex security processes and incident responses
- Bolstering the efficacy of existing cybersecurity measures
- Continuously learning from historical and real-time data
- Detecting unknown malware and zero-day attacks
- Enabling adaptive defence mechanisms that evolve based on previous encounters with cyber threats
- Forecasting potential vulnerabilities
- Optimising security policies based on real-world behaviour
- Reducing reliance on manual interventions
- Simulating attack scenarios
Examples of machine learning in cybersecurity tools
Anti-malware and anti-virus
Leverage machine learning to classify and detect malicious software based on code characteristics, behaviour, or execution patterns, including zero-day threats.
Cloud security posture management (CSPM)
Machine learning enhances CSPM tools by identifying misconfigurations, anomalous activity in cloud environments and potential policy violations based on usage trends.
Email security gateways
Employ machine learning to identify phishing , spoofing and business email compromise by analysing content, sender behaviour and URL patterns.
Endpoint detection and response (EDR)
Use machine learning to monitor and analyse endpoint activity and identify threats based on behavioural anomalies rather than just attack signatures.
Intrusion detection and prevention Systems (IDPS)
Use machine learning to detect abnormal network traffic patterns and prevent unauthorised access, even when attack signatures are unknown.
Network traffic analysis
Use machine learning to inspect network flow data for anomalies, helping identify lateral movement, data exfiltration and command-and-control communication.
Security information and event management (SIEM)
Incorporate machine learning for event correlation, anomaly detection and reducing false positives by learning from historical security event data.
Security orchestration, automation and response (SOAR)
Use machine learning to prioritise alerts and recommend or automate response actions based on threat intelligence and behavioural analysis.
User and entity behaviour analytics (UEBA)
Use machine learning to establish baselines of normal behaviour for users and systems, then flag unusual activities that could indicate a threat from external threat actors or malicious insiders.
Types of machine learning
Supervised machine learning in cybersecurity
Supervised machine learning in cybersecurity is used to classify data or predict outcomes. It uses labelled datasets to train algorithms and define the variables to be assessed for correlations, with the input and outputs specified. As part of the cross-validation process, when input data is fed, the model adjusts its weights until it has been fitted appropriately to avoid overfitting or underfitting.
Supervised machine learning in cybersecurity is used in several ways, including:
- Identifying unique labels of network risks, such as scanning and spoofing
- Predicting or classifying a target variable for a specific security threat (e.g., a distributed denial of service or DDOS attack
- Training models on benign and malicious samples to help them predict whether new samples are malicious
In addition to machine learning in cybersecurity, supervised machine learning can be used for:
- Binary classification—dividing data into two categories
- Multi-class classification—choosing between more than two types of answers
- Regression modelling—predicting continuous values
- Ensemble learning—combining the predictions of multiple machine learning models to produce an accurate prediction
Examples of techniques used for supervised machine learning in cybersecurity:
- Adaptive boosting
- Linear regression
- Logistic regression
- Naïve Bayes
- Neural networks
- Random forest
- Support vector machines (SVM
Reinforcement machine learning in cybersecurity
Reinforcement machine learning is a model used for machine learning in cybersecurity that is similar to supervised machine learning. However, reinforcement machine learning trains the algorithm by trial and error rather than using sample data. Positive or negative cues are given and registered along the way, with the algorithm programmed to seek affirmation and avoid penalties.
Reinforcement machine learning is often used to teach a machine to complete a multi-step process where the rules are clearly defined, such as training robots.
Reinforcement machine learning in cybersecurity is used in several ways, including:
- Adversarial simulation to train ML models to identify and respond to attacks in real-time
- Autonomous intrusion detections
- Cyber-physical systems
- Distributed denial of service (DDoS) defences
In addition to machine learning for cybersecurity, reinforcement machine learning is often used in situations where:
- A model of the environment is known, but an analytic solution is unavailable
- Only a simulation model of the environment is given
- The only way to collect environmental information is to interact with it
Examples of techniques used for reinforcement machine learning in cybersecurity:
- Deep Deterministic
- Deep Q Network (DQN)
- Policy Gradient (DDPG)
Unsupervised machine learning in cybersecurity
Unsupervised machine learning in cybersecurity is used to analyse and cluster unlabeled datasets (e.g., photo images, audio and video recordings, articles or social media posts). It can identify hidden patterns or data groupings without human intervention.
The algorithm scans through data sets, looking for patterns that are used to group information into subsets. Unsupervised machine learning is most commonly used for deep learning.
Unsupervised machine learning in cybersecurity can be used in a number of ways, including:
- Detecting unusual behaviour
- Identifying new attack patterns
- Mitigating zero-day attacks
In addition to machine learning for cybersecurity, unsupervised machine learning can be used for:
- Anomaly detection
- Association mining
- Clustering
- Dimensionality reduction (i.e., reducing the number of variables in a data set)
Examples of techniques used for unsupervised machine learning in cybersecurity:
- K-means clustering
- Neural networks
- Principal component analysis (PCA)
- Probabilistic clustering
- Singular value decomposition (SVD)
Semi-supervised machine learning in cybersecurity
Semi-supervised machine learning in cybersecurity blends supervised and unsupervised machine learning. It pulls a small labelled data set from a larger, unlabelled data set for classification and feature extraction when there is not enough labelled data for a supervised learning algorithm. It is also used when labelling a data set is prohibitively expensive.
Semi-supervised machine learning for cybersecurity can be used for:
- Adversarial neural networks
- Malicious and benign bot identification
- Malware detection
- Ransomware detection
In addition to machine learning for cybersecurity, semi-supervised learning can be used for:
- Fraud detection
- Labelling data
- Machine translation
Examples of techniques used for semi-supervised learning in cybersecurity:
- Consistency regularisation
- Label propagation
- Pseudo-labelling
- gSelf-training
Benefits of machine learning in cybersecurity
- Enables BYOD (bring your own device) and CYOD (choose your own device) to be securely implemented
- Automates cybersecurity processes
- Enhances threat detection, finding threats in the early stages
- Enables adaptable and proactive defence systems
- Expedites threat detection and response times
- Identifies hard-to-find network vulnerabilities
- Internalises learnings from previous attacks to prevent future attacks based on similar profiles
- Makes it easier for security analysts to quickly identify, prioritise, and remediate attacks
- Minimises human errors
- Powers sophisticated authentication mechanisms, such as facial recognition, fingerprint recognition, motion tracking, retinal scanners, and voice recognition
- Helps prevent security threats against endpoints
- Provides insights into advanced threats
- Reduces workloads
- Scans massive amounts of data to identify malware
- Detects nuances of normal behaviour to enable identification of even the smallest deviances
Machine learning in cybersecurity use cases
Preventing DDoS attacks and botnets
Models can be trained to analyse the large volumes of traffic between different endpoints to proactively identify and predict DDoS attacks (e.g., application, protocol and volumetric attacks) and botnets.
Identifying web shells
Machine learning models can be trained to identify web shells despite sophisticated evasion techniques.
Web shell detection has been proven far more accurate with machine learning than other systems because the models are able to improve complete predictions for unknown pages significantly.
Threat detection and classification
Machine learning is used in applications to facilitate and expedite detection and responses to attacks. Large datasets of security events are analysed to identify patterns of malicious activities.
When an incident is detected, the machine learning model automatically takes action. Datasets are drawn from a number of sources, such as indicators of compromise (IOCs) and security system log files.
Detecting malware
Models can be trained to help anti-virus solutions fight all types of malware, such as adware, backdoors, ransomware, spyware and trojans. Machine learning is also effective in detecting zero-day malware that traditional signature-based systems miss.
Network risk scoring
Machine learning can be used to analyse previous cyber attack datasets to determine areas targeted by particular attacks and assign accurate risk scores that quantify an attack’s location, likelihood, and impact. This data helps organisations prioritise the allocation of resources and directs responses in the event of a pervasive attack.
Protecting against application attacks
Machine learning can be utilised to train models to detect anomalies in HTTP/S, SQL, and XSS attacks to protect applications prone to different Layer 7 attacks.
Securing mobile endpoints
Machine learning is used in a number of detection and response applications to address threats to mobile devices. Another use of sophisticated machine learning is to protect against attacks using voice-based commands by training models to differentiate between the owner’s voice and hackers’ voices.
Security operation centres (SOCs)
This use case for machine learning supports the monitoring and detection of and response to security threats by automating the analysis of a large amount of data generated at high volumes.
Preventing phishing attacks
Machine learning can be used to analyse data in real-time and to identify and stop phishing emails. By training machine learning models on email headers, body copy, and punctuation patterns, they can learn to delineate between harmful and harmless emails, identifying patterns to classify and reveal possible phishing attacks. The models can also be trained to identify malicious URLs embedded in emails that appear benign.
Task automation
Machine learning excels at automating time-consuming, repetitive and error-prone security tasks, such as network log analysis, threat analysis, triaging intelligence and vulnerability assessment. In addition to providing automation, machine learning can identify threats and anomalies at a rate that is faster and far more effective than if performed by humans.
User and entity behaviour analytics (UEBA)
UEBA leverages machine learning to provide complete visibility of users and entities, detect account compromises and mitigate and detect malicious or anomalous insider activity. By using ML algorithms, baselines for normal behaviour patterns are established and used to identify unusual activity, such as an employee login late at night, inconsistent remote access or an unusually high number of downloads.
Email monitoring and security
Natural Language Processing (NLP), a type of machine learning, is highly effective for monitoring and assessing email for malware and viruses without opening the message. Machine learning is also adept at detecting phishing by analysing email headers, body content, links and sending patterns to flag phishing attempts.
Insider threat detection
By learning baseline user behaviours (e.g., login times, file access patterns), machine learning models can detect unusual activity that could be the sign of a malicious insider, such as a user downloading large volumes of sensitive information.
Firewall tuning
Machine learning models help firewall tools distinguish between legitimate traffic and malicious requests (e.g., SQL injection or cross-site scripting) by learning patterns in real traffic over time.
Behavioural biometrics for authentication
Machine learning continuously analyses human behaviour, such as how a user types, moves the mouse or interacts with a device. This helps detect imposters even if the correct login credentials are used.
Threat hunting and forensics
Machine learning facilitates threat hunting and forensics work by processing the massive volume of information in log files and telemetry data to uncover hidden threats, correlate indicators of compromise (IOCs) and surface attack patterns.
Supply chain attack detection
Machine learning models can identify unusual patterns in software updates, third-party tool behaviour, or vendor access to flag potential supply chain compromises.
Evaluating machine learning models
In cases where a machine learning model is not pre-built into a solution, care must be taken when evaluating and selecting models for machine learning in cybersecurity. Considerations when searching for a machine learning model that suits the use case and data include:
- Determine what resources are available to support machine learning models (e.g., training, monitoring, maintenance and measuring success)
- Establish the objective and identify potential data inputs
- Evaluate outcomes of machine learning models for similar use cases
- Understand how much data the model requires to be effective
Machine learning challenges and considerations
Machine learning in cybersecurity is indisputably a powerful and effective advancement. However, machine learning in cybersecurity does have challenges.
Some of the most commonly cited challenges related to machine learning include:
- Algorithms trained on data sets that exclude certain information or contain errors can lead to inaccurate models
- Monitoring and maintenance are required to keep machine learning models performing optimally
- Overly sensitive machine learning models can generate false positives, which can lead to alert fatigue and reduced trust in the system
- The vast amounts of data needed to power machine learning in cybersecurity require costly computational and data processing resources
- Poor data quality can degrade the performance of models used for machine learning in cybersecurity
- Machine learning in cybersecurity has difficulty identifying zero-day threats because they lack known signatures or patterns that can be used as examples to direct models
Overfitting and underfitting also create challenges resulting from degradation in machine learning models.
- Overfitting occurs when a machine learning model is trained with too much data and starts capturing noise and inaccurate data into the training data set, negatively affecting its performance
- Underfitting occurs when a model cannot fully learn the patterns in the training data and cannot deliver accurate results
Machine learning myths
| Machine learning myth | Machine learning reality |
|---|---|
| Machine learning in cybersecurity can fully replace human experts. | While powerful, machine learning cannot replace skilled cybersecurity professionals who offer contextual knowledge, creativity, critical thinking, intuition, and a nuanced understanding of complex attack vectors and cybercriminals’ thinking. |
| Machine learning can address all threats and vulnerabilities. | Certain types of attacks, such as zero-day exploits or highly targeted and sophisticated attacks, can be missed by machine learning models that lack training in that area. |
| Machine learning models in cybersecurity do not make mistakes. | Machine learning models are only as good as the datasets they are fed. The results will be subpar or incorrect if the data is incomplete or inaccurate. |
| Machine learning renders attacks ineffective. | While machine learning models can adjust defences to counter cyberattack vectors, criminals continuously adjust their approaches with a high degree of efficacy. |
| Machine learning in cybersecurity is impervious to adversarial attacks. | Unfortunately, machine learning is susceptible to adversarial attacks. If an attacker is able to inject misleading or incorrect data into a training dataset, the machine learning model will generate inaccurate results or make erroneous predictions. |
| Machine learning is only available to large organisations. | Machine learning is available and in wide use. Any organisation can use and benefit from machine learning at some level by leveraging user-friendly security tools, cloud-based security services, and pre-built models. |
| Machine learning in cybersecurity requires large datasets to provide value. | The efficacy of machine learning improves with the volume of data provided, but models can be used and trained with smaller quantities of quality data. |
Machine learning in cybersecurity bolsters solutions that fight threats
Machine learning in cybersecurity gives solutions a special edge that allows them to adjust and become more effective with time and experience. Threat intelligence produced by machine learning not only supports proactive threat protection, but helps make the solutions even better. Machine learning is pervasive and is expected to be a standard part of many solutions.
DISCLAIMER: THE INFORMATION CONTAINED IN THIS ARTICLE IS FOR INFORMATIONAL PURPOSES ONLY, AND NOTHING CONVEYED IN THIS ARTICLE IS INTENDED TO CONSTITUTE ANY FORM OF LEGAL ADVICE. SAILPOINT CANNOT GIVE SUCH ADVICE AND RECOMMENDS THAT YOU CONTACT LEGAL COUNSEL REGARDING APPLICABLE LEGAL ISSUES.
