Some reasons for the need for IDS/IPS are to deal with losses of intellectual property, data integrity, process control, industrial espionage, etc. They enable the recording of event sequences.

Daily life is increasingly influenced by the internet. For example, it simplifies tasks such as bank transfers, sales, travel, and interactions with government agencies and healthcare providers. However, the benefits of the internet are accompanied by the dangers of fraud and misuse. For instance, phishing, a form of online fraud, allows the theft of valuable information such as credit card numbers, social security numbers, usernames, and passwords, causing losses of three trillion dollars in the USA alone in 2007 (see the December 2007 report at http://www.gartner.com).
Alongside the growing dependence on Information Technologies (specifically ICT, Information and Communication Technology) by the global economy, government structures, communications, industry, businesses, and society in general, there has been a significant increase in the risk associated with pervasive intrusions in the electronic space. It is now evident that malicious intruders, using various threat vectors, are able to bypass protection systems (firewalls, antivirus software, identity management systems, access control systems, etc.) designed to restrict access to computer network resources in institutions such as banks, businesses, and organizations in general. To reduce risk and potential consequences, it is crucial to identify intrusions at an early stage and respond appropriately. This requires the implementation of intrusion detection and prevention systems, with intrusion management systems (IMS) being the most advanced example. A properly configured and updated IDS/IPS is an integral component of a defense-in-depth solution.
An IDS (Intrusion Detection System) is a protection system designed to identify and respond to malicious activities targeting computers and network resources. It is important that the IDS can process all transmitted packets regardless of the level of network usage; therefore, the minimum number of packets excluded from detection is needed. This can be achieved with load balancing schemes and redundancy mechanisms with high availability functionalities.

Some commonly used network traffic parameters are: (i) Number of IP datagram packets, for example, 7.5 million, classified by encapsulated protocol type, indicating their proportion, for example: TCP-86%, UDP-11%, ICMP-1%, IGMP, etc. (ii) Average data transmission rate measured in Mbps. (iii) Average packet size measured in bytes, for example, 720 bytes. (iv) Average number of packets per second, for example, 114,765 pps. (v) Duration, measured in time units, in the International System of Units (SI) in seconds. Information security is the process of protecting information (data, programs, knowledge) from a broad (and ever-growing) range of threats in order to ensure business continuity, minimize business damage, and maximize ROI and business opportunities and objectives. Therefore, information security is not a fad or a trend; it is a necessity and a profitable investment.

Security problem. Categories and methods of breaches. Levels of security measures and threats

Information security must consider the entire environment, including both the network and its end systems (PCs, PDAs, servers, etc.), and protect its resources and information. Intruders attempt to breach security. A threat is a potential security violation exploiting a vulnerability (the number of vulnerabilities is infinite, although the number of known vulnerabilities increases daily). An attack is an attempt to breach security. An attack can be accidental or malicious. It is easier to protect against accidental attacks than against malicious misuse. Security breaches can be classified into categories such as: breaches of confidentiality, breaches of integrity, breaches of availability, theft of services, and denial-of-service attacks.

The main methods of security breaches are: masquerading (breaking authentication), replay attacks (message modification), MITM/Man-In-The-Middle attacks (for entity impersonation), and session hijacking. The four main levels of security measures are: physical, human (preventing social engineering, phishing, and dumpster diving), operating system, and network. We must remember that security is only as strong as its weakest link. Some of the main threats to software are Trojans, backdoors, logic bombs, stack and buffer overflows, and viruses (file viruses, boot sector viruses, macro viruses, source code viruses, polymorphic viruses, encrypted viruses, stealthy viruses, tunneling viruses, multipartite viruses, and shielded viruses). Some relevant threats to the system and network are worms, port scanning, and DoS/DDoS attacks.

Network Security Threats.
Network security refers to the protection of network resources, particularly computer systems (PCs, servers, PDAs, etc.) and information/data/knowledge. Regarding the different risks to network resources, those related to computer systems can be identified as those concerning availability/fault tolerance and those related to unauthorized access. Regarding information/data, those related to the acronym CIA (Confidentiality, Integrity, Availability) can be identified. Regarding where these risks lie, three areas can be defined: those related to internal versus external factors, those related to the network itself (e.g., eavesdropping), and those related to computers, such as system vulnerabilities, access control, and physical security.

The main current security threats are: (i) Those used to compromise system security: scanning/exploring other systems to find backdoors for unauthorized access and Denial-of-Service (DoS) attacks. (ii) Malware, such as computer viruses, spyware, Trojans, worms, etc. (iii) Spamming: Systems used to send unsolicited emails.

Intrusion detection and prevention. Functions, technologies, and methods of an IDS. Intrusion prevention:
An intrusion is any intentional event through which an intruder gains access and compromises the confidentiality, integrity, or availability of computers, networks, or the data, information, and knowledge residing on them. An attacker, adversary, intruder, or hacker can bypass a firewall and steal sensitive files located on a server. The solution is to use some type of intrusion detection, prevention, and management system, which can be viewed as a security camera acting as an informant.

An Intrusion Detection System (IDS) detects intrusive behavior automatically. The following functions can be identified in an IDS: (i) Monitoring events in network traffic and on computers or hosts. (ii) Real-time event analysis for signs of intrusion. (iii) Recording information in an audit log. (iv) Alert notifications, for example, using email, SMS, or by triggering audible or visual alarms. (v) Generating customized reports of events of interest.

Basically, two technologies can be identified in relation to IDSs: HIDS (Host-based IDS) and NIDS (Network-based IDS). Likewise, two detection methods can be identified: Signature-based detection (identifies known sequences of events that indicate intrusive behavior) and Anomaly-based detection (looks for abnormal behavior, thus potentially detecting unknown attacks).
Intrusion prevention is the ability to detect an event and attempt to stop potential incidents. An Intrusion Prevention System (IPS) prevents intrusion attempts detected by the IDS. Therefore: IPS = (IDS + prevention module). The IPS acts as a "network traffic police" that automates the response. An IPS uses various response techniques: (i) It stops the attack itself by blocking access to the victim and terminating the network connection. (ii) It changes the security environment by reconfiguring network devices such as firewalls, routers (L3), and switches/bridges (L2). (iii) Change the attack content, for example by deleting an infected file attached to an email, modifying the content of a UDP datagram inline, etc. (iv) Redirect traffic to a Honeypot/PaddleCell and send SNMP traps.

IDS/IPS Control Strategies. NIDS/NIPS Deployment Types.
An IDS can be implemented using one of three basic control strategies: (1) Centralized. All IDS control functions are implemented and managed from a central location or console. (2) Fully Distributed. All control functions are applied at the physical location of each IDS component. (3) Partially Distributed. This combines the two previous strategies; while individual agents can analyze and report local threats, they report to a hierarchical central station to enable the organization to detect widespread attacks.

The NIST (National Institute of Standards and Technology, see http://www.nist.gov ) recommends four locations for placing NIDS sensors: (i) Behind each external firewall in the DMZ. (ii) Outside the external firewall. (iii) On the main network backbones. (iv) On critical subnets. Other possible locations include: in the DMZ (Demilitarized Zone), in front of and behind the firewall (this combination allows for the detection of firewall attacks and can help refine the firewall rule set), on server network segments, on network segments with high-power users, behind the RAS (Remote Access Server), between business units, and between the corporate network and corporate partner networks.

Measuring the Effectiveness of IDS/IPS
Two dominant metrics used to evaluate IDS/IPS are: (1) Administrators assess the number of attacks detected on a known set of tests. (2) Administrators examine the utilization level of each failing IDS. The evaluation of an IDS can be expressed as follows: for example, at 1000 Mbps, an IDS can detect 97% of targeted attacks. Because developing this data collection can be tedious, most IDS vendors provide testing mechanisms to verify that the systems are performing as expected. Some of these testing processes will allow the administrator to: (i) Log and retransmit packets from real virus or worm scanners. (ii) Log and retransmit packets from real virus and worm scanners with incomplete TCP session connections (losing SYN packets). (iii) Perform a real virus or worm scan against an invulnerable system.

Classification and Drawbacks of IDS/IPS
Three criteria used to classify IDS/IPS are:
(1) According to the technologies or methods used to detect intrusions. These can be identified as:
(i) Signature-based attack systems. These are used by NIDS that employ pre-identified attack signatures.
(ii) Anomaly-based systems. These use statistical methods and are based on finding irregular activities that differ from the normal baseline pattern.
(iii) System protocol misuse systems. These monitor deviations from the normal protocol. This method is useful for detecting attempts by a user or application to gain unauthorized access to a system.
(2) According to the practical implementation of the system, that is, according to the monitored target. Three main groups can be identified:
(i) Host-based or HIDS (examples include: Tripwire, ISS RealSecure OS Sensor, and Axent Intruder Alert) and all related IDS, such as application-based or AIDS and protocol-based or PIDS. HIDS are small software programs called agents that reside on a computer; they monitor metrics such as file access and modification, insertion of removable disk drives (USB/CD/DVD), CPU performance, etc. AIDS reside on a computer and monitor metrics such as process terminations, anomalous exits, message queues, etc.
(ii) Network-based IDS or NIDS (e.g., Snort 2.8.0, Axent's Netpowler, NAI's Cybercop Monitor, ISS's RealSecure Network Sensor, Cisco's Secure IDS) and all related IDS, such as protocol-based IDS or PIDS. An NIDS resides on a separate appliance and monitors network traffic, anomalous devices, and insider attacks. A PIDS can reside on top of an NIDS or on a computer and is responsible for monitoring the use of the communication protocol between systems.
(iii) Hybrid IDS or distributed IDS. They combine the two operational locations, being able to detect both internal intrusions in end-user computing systems (HIDS) and external intrusions in network traffic with or without switches (NIDS).
(3) Functional classification. According to their functionality: (i) Packet capture and pattern or signature comparator. (ii) Log analyzer. (iii) File integrity checker. (iv) Activity monitor. (v) Host firewall.
Some intrusion detection styles are based on: known signatures (syntax), known misuse (may include semantics), anomaly detection (must operate within the domain), specification (what is allowed is defined, and the rest is prohibited), and behavior (contextual evidence, for example, “unset HISTFILE”).
The main drawbacks of IDS are: (i) The volume of false positives is enormous. (ii) The number of unreported events. (iii) Most packet sniffing solutions are context-free. They have no idea if an attack is relevant. (iv) They are only reporting entities; they are reactive agents. (v) High workload for the IDS administrator. (vi) Manual response to events. (vii) No preventative capability.

Comparison between HIDS and NIDS: (a) HIDS. Two common implementation methods are agent-based and log analyzer. Agent-based deployment requires installing specialized software on the host system. Log analysis implementation relies on host logs being sent to or extracted from a logging system. HIDS are better than NIDS at handling encrypted information. (b) NIDS. These are deployed at one or more points along the network. The sniffer agent analyzes network packets containing data, including the message and header that identifies the sender and receiver. Network attacks such as IP spoofing, packet flooding, and denial-of-service attacks are better detected through NIDS packet examination than using an HIDS.

Criteria for Choosing an IDS/IPS. Potential Problems.
When choosing an IDS, the following guidelines or evaluation criteria can be considered: (1) Attack signatures used. Their quality and the organization that creates them should be assessed; the false positive rate, false negative rate, and BRF (Base Rate Fallacy) should be examined. Likewise, the update frequency should be considered (more than every eight hours can be dangerous). It is also advisable to evaluate the update mechanism (whether it is protected and how it works). (2) Scalability. The traffic management and handling capacity should be evaluated, and it should be noted whether it has load balancing functionalities for peak hours. Similarly, in HIDSs, the supported platforms should be considered. Another aspect to evaluate is the type of shutdown mechanism used.

(3) The type of hardware platform used. For example, on a PC or on a switch appliance with general-purpose processors such as Intel-based processors (e.g., Intel Core 2 Quad), or based on SPARC processors, or based on ASIC hardware (Application-Specific Integrated Circuits, integrated circuits that execute a set of hardware-coded instructions on the passing data), or based on FPGA hardware (Field Programmable Gate Arrays, integrated circuits that can be programmed to perform certain operations on the passing data). (4) Manageability. Consider whether it incorporates features such as: log review capabilities, cross-referencing, archiving capabilities, and the existence of a centralized console. Some of the potential problems presented by IDS/IPS include: the quality of attack signatures; the management of encrypted SSL traffic, IPsec/PPTP tunnels, and PGP attachments in email; The use of LANs with switches (multiple collision domains) versus LANs with hubs (one collision domain) requires making connections through the switch's spanning/mirroring port to monitor all traffic, which may degrade performance; deployment in very high-speed networks may result in not monitoring all traffic.

Response Actions of an IDS/IPS:
An IDS/IPS monitors and detects malicious behavior and identifies suspicious patterns that could compromise the security of a network/computer system. An IDS is a passive system; it detects a potential security breach, logs the information, and issues an alert. An IPS is both reactive, responding to suspicious activity by reconfiguring firewalls to block network traffic or dropping traffic from the network, and proactive, taking measures in advance based on detected indicators. Possible response actions that an intrusion management system can perform include: (1) Logging: IP packet headers and encapsulated protocols, relevant application data, and raw IP packets. (2) Alerting: Via a console to administrators, email, SMS, or by sending SNMP trap messages to a network management system. (3) Terminating intrusive connections: Using the `kill TCP` command or kernel drop. (4) Notifying or interacting with third-party devices in an integrated environment: reconfiguring firewalls or L3 routers or L2 switches. (5) Using user scripts. Increasing the log level, sending a message to a modem to forward it to a pager, sending emails to trigger SMS messages, redirecting suspicious traffic to a honeypot or paddle cell, placing the infected file or device in a quarantine zone, for example, using VLANs. Two intrusion detection strategies used: (i) Policy detection. This involves deciding in advance what type of behavior is undesirable and detecting intrusions through the use of a permit or deny policy or default settings. (ii) Anomaly detection. This involves reporting anything unusual for the subject (computer, user, etc.) that is suspected and warrants further investigation.

Standards Associated with IDS/IPS
Two standards used in IDS/IPS are: (1) IDMEF (Intrusion Detection Message Exchange Format) from the IETF. The goal of the IDWG (Intrusion Detection Working Group) is to define the data format, define the exchange procedure, and specify a common intrusion language. IDMEF is a standard and interoperable data format that uses XML. Typical deployments include communication between the sensor and the manager, database storage, interaction with the centralized console, and the event correlation system. (2) CVE (Common Vulnerabilities and Exposures). This enables interoperability between tools. An example of a nomenclature is CVE-2000-0809, which is an entry in the CVE list that standardizes a security issue, in this case, a buffer overflow in Check Point's VPN-1/Firewall-1 v4.1 tool.

Defense-in-depth approach to securing an environment.
The need for a set of different levels of defense is evident in order to reduce risk to the levels demanded by an organization's senior management. These levels are: (L1) Blocking network-based attacks. This includes the use of firewalls and corporate intrusion detection/prevention systems, Network Admission Control (NAC) tools, and antispam/antivirus security appliances. (L2) Blocking computer-based attacks. This includes the use of personal firewalls, host-based intrusion detection/prevention systems, antivirus, and antispam/antiphishing software. (L3) Eliminating security vulnerabilities. This includes the use of vulnerability management tools, vulnerability patch/remediation management tools, security configuration compliance tools, and tools for ensuring compliance with laws, standards, and regulations, as well as application security testing tools. (L4) Secure support for authorized users. Access and identity management tools, identity federation, file encryption systems, client-side VPN tools, and clientless SSL VPNs are used. (N5) Minimize business losses and maximize effectiveness. Security information management tools with dashboards, integrity monitoring tools, backup tools, business recovery tools, forensic tools, and security skills development tools are used.

Final Considerations
Our research group has been actively working for over fifteen years on the development, analysis, and evaluation of intrusion management mechanisms, schemes, and systems at the end-user computing and network resource levels. We have deployed IDS/IPS systems across a wide variety of topologies in environments with both low and high electromagnetic interference, achieving highly satisfactory results in terms of effectiveness. We have analyzed performance and the problems associated with errors. We have provided solutions in the area of ​​delayed, stealthy attacks, attacks involving fragmented and manipulated packets at the header and L2/L3/L4/L5 data field levels.

More information or a quote

This article is part of the activities developed within the

LEFIS-APTICE project (funded by Socrates 2005-2007. European Commission ).