๐ Lecture 39 โ Overview of todayโs lecture
๐ Overview: This lecture introduces fundamental concepts of security and protection in operating systems, covering security issues, the distinction between policy and mechanism, design principles for security, and basic terminology. It also provides an introduction to user authentication methods. Understanding these concepts is critical for designing systems that can resist both accidental misuse and malicious attacks.
๐๏ธ Topics Covered
The lecture covers the introduction to security and protection, including security issues such as isolation, authentication, access control, and protection problems. It then discusses policy versus mechanism, the reality that no perfect protection system exists, design principles for security, how to determine security requirements, key terminology (principals, objects, rights), and activities (authentication, authorization, auditing), concluding with an introduction to user authentication.
๐ Lecture Summary
Security & Protection
The purpose of a protection system is to prevent accidental or intentional misuse of a system. Accidental misuse, such as a program mistakenly deleting the root directory, is relatively easy to solve by making the likelihood small. Malicious abuse, such as a hacker breaking a password and transferring money, is very hard to completely eliminate because loopholes cannot be fully sealed and probability-based solutions are insufficient.
๐ก Why this matters: Understanding the difference between accidental and malicious threats helps engineers design appropriate defense mechanisms for each type.
Security issues
Isolation ensures separate processes execute in separate memory space, so a process can only manipulate allocated pages. Authentication determines who can access the system by proving identities. Access control governs when a process can create or access a file, create or read/write to a socket, or make a specific system call. The protection problem is to ensure that each object is accessed correctly and only by those processes that are allowed to do so. Comparison between different operating systems involves evaluating which protection models support least privilege most effectively and which system best enforces its protection model.
Policy versus mechanism
A good way to approach security is to separate policy (what) from mechanism (how). A protection system is the mechanism to enforce a security policy, with roughly the same set of choices regardless of the specific policy. A security policy delineates what is acceptable and unacceptable behavior. Example security policies include: each user can only allocate 40MB of disk, no one but root can write to the password file, and you cannot read my mail.
There is no perfect protection system
A very simple but easily missed point: protection can only increase the work factorโthe effort needed to do something badโbut it cannot prevent it entirely. Even assuming a technically perfect system, there are always the four Bs:
- Burglary: if you cannot break into the system, you can always steal it (physical security)
- Bribery: find whoever has access to what you want and bribe them
- Blackmail
- Bludgeoning: beat someone until they tell you
Design Principles for Security
- System design should be public
- Default should be no access
- Check for current authority
- Give each process least privilege possible
- Protection mechanism should be simple, uniform, and in lowest layers of system
- Scheme should be psychologically acceptable
First: What are Your Security Requirements?
To determine security requirements, first assess your security environment: what threats exist and how severe are they, who is not trusted, what assumptions are made, what platforms and network environment exist, what organizational policies apply, and what assets need protection. Then identify your product's security objectives:
- Confidentiality ("can't read")
- Integrity ("can't change")
- Availability ("works continuously")
- Other: Privacy ("doesn't reveal"), Audit, etc.
Finally, determine what functions and assurance measures are needed, using the Common Criteria as a useful checklist of requirements.
Terminology I: the entities
- Principals โ who is acting? (user/process creator, code author)
- Objects โ what is that principal acting on? (file, network connection)
- Rights โ what actions might you take? (read, write)
Familiar UNIX file system example: owner/group/world with read/write/execute permissions.
Terminology II: the activities
- Authentication โ who are you? Identifying principals (users/programs)
- Authorization โ what are you allowed to do? Determining what access users and programs have to specific objects
- Auditing โ what happened? Recording what users and programs are doing for later analysis/prosecution
User Authentication
User authentication must identify the user through one of three basic principles:
- Something the user knows (e.g., password)
- Something the user has (e.g., security token)
- Something the user is (e.g., biometric characteristic)
This is done before the user can use the system.
โญ Key Takeaways
The most critical point is that no protection system is perfectโit can only increase the work factor required to breach security, and attackers may bypass technical controls through physical means (the four Bs). Students must understand the crucial distinction between policy (what behavior is allowed) and mechanism (how to enforce it), and that a protection system is just the mechanism implementing a policy. The six design principles, especially "default no access" and "least privilege," are foundational for building secure systems. Finally, security requirements must be evaluated before design, considering confidentiality, integrity, availability, and privacy, and the three authentication factors (something you know, have, or are) must be clearly distinguished.
๐ง Quick Revision Questions
- What is the fundamental difference between accidental misuse and malicious abuse in terms of difficulty of prevention?
- What are the four Bs that can defeat even a technically perfect protection system?
- List all six design principles for security as presented in the lecture.
- What are the three security objectives (confidentiality, integrity, availability) and how do they differ from privacy?
- What three categories of information are used for user authentication, and provide an example of each?
๐ Lecture 40 โ Overview of todayโs lecture
๐ Overview: This lecture covers the fundamental concepts of user authentication and access control in operating systems. It examines various authentication methods ranging from password-based schemes to biometrics, and introduces the access control matrix model along with its practical implementations (ACLs and capabilities). Understanding these mechanisms is crucial for designing secure systems that correctly identify users and control their access to resources.
๐๏ธ Topics Covered
This lecture covers user authentication, password-based authentication, the UNIX password scheme, one-time password schemes (including Lamportโs scheme), challenge-response authentication (used in PPP), biometrics and other authentication alternatives like badges/keys, access control and authorization, and the access control matrix model with its two implementation concepts: access control lists (ACLs) and capabilities.
๐ Lecture Summary
Overview of todayโs lecture
The lecture begins with an outline of topics including user authentication, password based authentication, the UNIX password scheme, one-time password schemes, challenge response authentication, biometrics and other authentication schemes, and access control and authorization culminating in the access control matrix.
Authentication
Authentication is usually done with passwords, which is considered a relatively weak form because it relies on something people must remember. Empirically, passwords are often based on easily guessed personal information like a spouse's or child's name or a favorite movie name.
Passwords should not be stored in a directly-readable form. Instead, use some sort of one-way-transformation (a secure hash) and store that hash. For example, if you look in /etc/passwords, you will see a bunch of gibberish associated with each name โ that is the hashed password.
A key problem is preventing dictionary attacks: passwords should be long and obscure to prevent guessing, but unfortunately, such passwords are easily forgotten and are usually written down.
๐ Definition โ Dictionary attack: An attack where an adversary tries a list of common passwords or dictionary words against a hashed password database.
๐ Example: A user chooses "password123" as their password. An attacker runs a program that hashes thousands of common passwords (e.g., "password", "123456", "qwerty", "letmein") and compares the resulting hashes to the stored hash in /etc/passwords. If any match, the password is cracked.
๐ก Why this matters: Dictionary attacks are the most common method for breaking password-based systems; understanding this motivates the need for strong, random passwords and hashing with salt.
UNIX password security: Uses encryption of passwords.
One time passwords: A clever scheme was developed by Lamport (read Tanenbaum for details).
Challenge-Response based authentication: Used in PPP (Point-to-Point Protocol) and many other applications.
Authentication alternatives
An alternative to passwords is a badge or key. This does not have to be kept secret and is usually some sort of picture ID worn on a jacket (e.g., at military bases). It should not be forgeable or copy-able. It can be stolen, but the owner should know if it is.
๐ Definition โ Capability: In this context, a badge or key is similar to the notion of a "capability" that will be seen later โ a token that grants access to a resource.
Biometrics
Biometrics refers to authentication of a person based on a physiological or behavioral characteristic. Example features include face, fingerprints, hand geometry, handwriting, iris, retinal, vein, and voice. This provides strong authentication but still requires a trusted path.
๐ Definition โ Trusted Path: A mechanism that ensures a user is communicating directly with the trusted operating system, not with a malicious program (e.g., a fake login screen) that might intercept credentials.
Access control
The context for access control is that the system knows who the user is (e.g., user has entered a name and password, or other info). Access requests pass through a gatekeeper, and the OS must be designed so that the monitor cannot be bypassed.
Access control matrix [Lampson]
The access control matrix is a conceptual model where:
- Rows represent subjects (users or processes)
- Columns represent objects (resources like files, devices)
- Cells contain the access rights (e.g., read, write, execute) that a subject has to an object
๐ Formula: AccessControlMatrix[Subject, Object] = {set of access rights} โ Plain-English meaning: For each user and each resource, the matrix specifies exactly what operations that user is allowed to perform.
Two implementation concepts
Two main implementations of the access control matrix exist:
-
Access control list (ACL): Store the column of the matrix with the resource. Each resource (object) maintains a list of users and their access rights. ๐ Example: A file "report.txt" has an ACL specifying: "Alice: read, write; Bob: read; Charlie: no access".
-
Capability: The user holds a ticket for each resource. Each user (subject) possesses a list of tickets that grant access to specific objects. ๐ Example: Alice holds a capability token that allows her to open "report.txt" with read and write access; Bob holds a different capability token for the same file giving only read access.
Access control lists are widely used, often with groups. Some aspects of the capability concept are used in Kerberos.
๐ก Why this matters: ACLs and capabilities represent the two fundamental approaches to implementing authorization โ one stores permissions with the object (easier for revocation), the other with the user (easier for delegation). Choosing between them affects system performance and security properties.
โญ Key Takeaways
- Passwords should never be stored in plain text; always use one-way transformations (secure hashes) to prevent direct exposure, and long obscure passwords are needed to resist dictionary attacks despite being hard to remember. Biometrics (face, fingerprints, iris, etc.) offer stronger authentication but still require a trusted path to prevent interception. The access control matrix is the foundational model for authorization, where rows represent subjects, columns represent objects, and cells define access rights. This matrix is implemented in two main ways: access control lists (ACLs) store permissions with the resource (column-oriented), while capabilities store them with the user (row-oriented). ACLs are the more widely used approach, often with groups, while capability concepts appear in systems like Kerberos.
๐ง Quick Revision Questions
- What is the main weakness of password-based authentication as described in this lecture, and what attack does it enable?
- How does the UNIX password scheme store passwords securely, and what should you find in
/etc/passwords? - What is Lamport's scheme used for, and what is the main advantage of one-time passwords?
- List four different biometric features that can be used for authentication.
- Explain the difference between an access control list (ACL) and a capability in terms of where the access permissions are stored.
๐ Lecture 41 โ Overview of todayโs lecture
๐ Overview: This lecture contrasts Access Control Lists (ACLs) and capabilities as two fundamental access control mechanisms, covering delegation, revocation, and operations on capabilities. It then explores advanced access control models including roles, groups, confidentiality (Bell-LaPadula), integrity (Biba), and other policy concepts like separation of duty and the Chinese Wall policy, which are crucial for designing secure operating systems.
๐๏ธ Topics Covered
The lecture begins with a comparison of ACLs versus capabilities, focusing on delegation and revocation. It then details operations on capabilities (copy, copy object, remove, destroy), discusses sandboxing mobile code, and introduces roles and groups for organizing permissions. Finally, it covers Multi-Level Security (MLS) concepts, the Confidentiality Model (Bell-LaPadula), the Integrity Model (Biba), and other policy concepts like separation of duty and the Chinese Wall policy.
๐ Lecture Summary
ACL vs Capabilities
Access Control Lists (ACLs) associate a list with each object and check the user/group against that list. This relies on authentication because the system needs to know the user's identity. In contrast, capabilities are unforgeable ticketsโeither a random bit sequence or managed by the OSโthat can be passed from one process to another. A reference monitor checks the ticket and does not need to know the identity of the user or process.
For delegation, with capabilities, a process can pass a capability to another at run time. With ACLs, you must try to get the owner to add the permission to the list. For revocation, ACLs simply remove the user or group from the list. With capabilities, revocation is harder because you have to try to get the capability back from the process. Revocation is possible in some systems only if the OS performs appropriate bookkeeping and knows what data is a capability. If a capability is used for multiple resources, you have to revoke all or none.
Operations on Capabilities
Four operations are defined on capabilities:
- Copy: create a new capability for the same object
- Copy object: create a duplicate object with a new capability
- Remove capability: delete an entry from the capability list; the object remains unaffected
- Destroy object: permanently remove an object and its capability
Sandboxing mobile code involves starting a foreign program in a process and giving that process a specific set of capabilities, such as read and write on the monitor and read and write a scratch directory. This follows the principle of least privilege.
๐ก Why this matters: Capabilities have been described as an operating system concept "of the future (and always will be?)" but are implemented in systems like Hydra, StarOS, Intel iAPX 432, Eros, and Amoeba (distributed, unforgeable tickets).
๐ Definition โ Capability: An unforgeable ticket that grants a process access rights to an object, managed either as a random bit sequence or by the operating system.
Roles (also called Groups)
A role is a set of users, such as Administrator, PowerUser, User, and Guest. Permissions are assigned to roles, and each user gets the permissions of their assigned role. A role hierarchy is a partial order of roles where each role automatically inherits the permissions of roles below it. You only need to list new permissions given to each role.
Groups for resources, rights
A permission is defined as a pair โจright, resourceโฉ. Permission hierarchies allow that if a user has right r and r > s, then the user also has right s. Similarly, if a user has read access to a directory, the user has read access to every file in that directory. The big problem in access control is that complex mechanisms require complex input, making them difficult to configure and maintain. Roles and other organizing ideas try to simplify this problem.
Multi-Level Security (MLS) Concepts
MLS originates from military security policy where classification involves sensitivity levels and compartments. The goal is to prevent classified information from leaking to unclassified files. Grouping individuals and resources uses some form of hierarchy to organize policy. Other policy concepts include separation of duty and the "Chinese Wall" Policy.
Confidentiality Model
The Confidentiality Model (Bell-LaPadula) answers: When is it OK to release information? It has two properties:
- Simple security property: A subject S may read object O only if
C(O) โค C(S) - *-Property (star property): A subject S with read access to object O may write object P only if
C(O) โค C(P)
In words: You may only read below your classification and only write above your classification.
๐ Definition โ Simple security property: A subject may read an object only if the object's classification is less than or equal to the subject's classification (no read-up). ๐ Definition โ *-Property (star property): A subject that has read access to an object may write to another object only if the first object's classification is less than or equal to the second object's classification (no write-down).
Integrity Model
The Integrity Model (Biba) preserves the integrity of information. It also has two properties:
- Simple integrity property: A subject S may write object O only if
C(S) โฅ C(O). Only trust S to modify O if S has higher rank. - *-Property (star property): A subject S with read access to O may write object P only if
C(O) โฅ C(P). Only move info from O to P if O is more trusted than P.
In words: You may only write below your classification and only read above your classification.
The problem is that these models appear contradictory. Confidentiality says "read down, write up," while integrity says "read up, write down." To have both confidentiality and integrity, this contradiction is partly an illusion. You may use confidentiality for one classification of personnel/data and integrity for another. Otherwise, the only way to satisfy both models is to allow read and write only at the same classification. In practice, confidentiality is used more than the integrity model (e.g., in Common Criteria).
๐ Definition โ Simple integrity property: A subject may write an object only if the subject's classification is greater than or equal to the object's classification (no write-up). ๐ Definition โ *-Property (integrity): A subject with read access to an object may write another object only if the first object's classification is greater than or equal to the second's (no read-down).
Other policy concepts
Separation of duty: If an amount is over $10,000, a check is only valid if signed by two authorized people. The two people must be different. Policy involves role membership and inequality.
Chinese Wall Policy: Lawyers L1 and L2 in Firm F are experts in banking. If bank B1 sues bank B2, L1 and L2 can each work for either B1 or B2, but no lawyer can work for opposite sides in any case. Permission depends on the use of other permissions.
โญ Key Takeaways
The critical lesson is the fundamental contrast between ACLs (which require identity) and capabilities (which rely on unforgeable tickets and support run-time delegation but complicate revocation). You must know the two pairs of properties: for confidentiality (Bell-LaPadula) โ Simple security property (read down) and *-Property (write up) โ and for integrity (Biba) โ Simple integrity property (write down) and *-Property (read up). Remember that these models are contradictory if applied to the same data, but can be used for different classifications. Finally, be able to define capabilities, roles, role hierarchies, separation of duty, and the Chinese Wall policy.
๐ง Quick Revision Questions
- How does delegation differ between ACL-based and capability-based systems?
- What are the two properties of the Confidentiality (Bell-LaPadula) model, and what do they prevent (in terms of reading up/down and writing up/down)?
- What are the two properties of the Integrity (Biba) model, and how do they differ from the Confidentiality model's rules?
- Explain the revocation problem with capabilities and what OS bookkeeping is needed to make it possible.
- Describe the Chinese Wall policy and explain why it differs from standard access control models.
๐ Lecture 42 โ Overview of todayโs lecture
๐ Overview: This lecture introduces several common types of malicious software attacks, including Trojan horses, login spoofing, logic bombs, and trap doors. It then focuses extensively on buffer and stack overflow attacksโone of the most prevalent and dangerous security vulnerabilitiesโexplaining how they work and how they can be exploited using unsafe C library functions.
๐๏ธ Topics Covered
The lecture covers Trojan horses as deceptive programs containing hidden harmful code, login spoofing attacks that trick users into revealing credentials, logic bombs that activate upon specific conditions, and trap doors providing unauthorized access. The main focus is on buffer and stack overflow attacks, including their history, mechanics using stack diagrams, and the role of unsafe C library functions (strcpy, strcat, gets, scanf, printf) in enabling these exploits.
๐ Lecture Summary
Trojan Horses (from Tanenbaumโs book)
A Trojan horse is a free program made available to an unsuspecting user that actually contains code to do harm. The attacker places an altered version of a utility program on the victim's computer, tricking the user into running that program. This is similar to the ancient Greek story where a seemingly harmless gift concealed enemy soldiers.
Login Spoofing
Login spoofing involves displaying a phony login screen that looks identical to the correct one. The user types their username and password, which are captured by the attacker. The fake screen might then show an error message and present the real login screen, making the user believe they mistyped their credentials.
๐ Definition โ Login spoofing: A technique where a fake login interface is presented to trick users into entering their credentials, which are then stolen by the attacker.
Logic Bombs
A logic bomb is malicious code written by a company programmer that has the potential to do harm. It remains harmless as long as the programmer enters a password daily. If the programmer is fired and stops entering the password, the bomb "explodes" and executes the harmful payload.
๐ Definition โ Logic bomb: Malicious code inserted into a system that activates upon a specific logical condition (e.g., missing a daily password entry).
Trap Doors
A trap door (also called a backdoor) is secret entry point inserted into code that bypasses normal authentication. The lecture shows normal code and code with a trapdoor inserted, allowing the attacker to gain unauthorized access to the system.
๐ Definition โ Trap door: A hidden mechanism in software that grants unauthorized access to a system, bypassing normal security controls.
Buffer overflows
Buffer overflows are an extremely common bug. The first major exploit was the 1988 Internet Worm targeting the fingerd service. Ten years later, over 50% of all CERT advisories involved buffer overflows: 1997 had 16 out of 28 (57%), 1998 had 9 out of 13 (69%), and 1999 had 6 out of 12 (50%). Buffer overflows often lead to total compromise of the host. Fortunately, exploiting them requires expertise and patience, involving two steps: locating a buffer overflow within an application, and designing an exploit.
The lecture shows stack diagrams: (a) situation when main program is running, (b) after program A is called, and (c) buffer overflow shown in gray, illustrating how the overflow overwrites adjacent memory.
๐ก Why this matters: Buffer overflows remain one of the most critical security vulnerabilities despite being known for decades. They allow attackers to execute arbitrary code and take complete control of systems.
What are buffer overflows?
Suppose a web server contains a function:
void func(char *str) {
char buf[128];
strcpy(buf, str);
do-something(buf);
}
When the function is invoked, the stack looks like a structure containing local variables (buf with 128 bytes), the saved frame pointer, and the return address. If *str is 136 bytes long, after strcpy() the extra 8 bytes overwrite the return address and potentially the saved frame pointer, corrupting the stack.
๐ Definition โ Buffer overflow: Occurs when a program writes more data to a buffer than it can hold, overwriting adjacent memory and potentially causing crashes or enabling code execution.
Basic stack exploit
The main problem is no range checking in strcpy(). Suppose *str is crafted so that after strcpy, the stack contains the attacker's shellcode in the buffer area, and the overwritten return address points to that shellcode. When func() exits, the return address is used to transfer controlโand instead of returning normally, the user will be given a shell. Note that attack code runs in the stack. To determine the correct return address, the attacker guesses the position of the stack when func() is called and uses a stream of NOPs (no-operation instructions) to create a "NOP sled" that increases the chance of landing in the exploit code.
๐ Definition โ NOP sled: A sequence of NOP (no-operation) instructions placed before shellcode to increase the probability that the overwritten return address lands somewhere in the sled, sliding execution into the actual exploit code.
๐ Formula: Buffer overflow exploit formula: Overflow buffer + overwrite return address + NOP sled + shellcode = arbitrary code execution
๐ Example: A 128-byte buffer receives 136 bytes via strcpy. The first 128 bytes fill the buffer (possibly containing NOP sled and shellcode), bytes 129-132 overwrite the saved frame pointer, and bytes 133-136 overwrite the return address to point to the shellcode in the buffer.
Some unsafe C lib functions
The following C library functions are unsafe because they do not perform bounds checking:
strcpy (char *dest, const char *src)โ copies string without size limitstrcat (char *dest, const char *src)โ appends string without size limitgets (char *s)โ reads input without size limitscanf ( const char *format, ... )โ may overflow if format doesn't limit inputprintf (const char *format, ... )โ can be exploited via format string vulnerabilities
๐ Definition โ Unsafe C library functions: Standard C functions that do not check buffer boundaries, making them vulnerable to buffer overflow attacks when used with untrusted input.
How does an attacker actually launch this attack
An attacker launches a buffer overflow attack through inspection of source code (if available), help of debuggers to examine memory layout and stack positions, and cramming a lot of data into a program to trigger the overflow.
Exploiting buffer overflows
Suppose a web server calls func() with a given URL. The attacker can create a 200-byte URL to obtain a shell on the web server. Some complications arise: the program should not contain the '\0' character (which would terminate string operations early), and the overflow should not crash the program before func() exits. Sample buffer overflows of this type include: overflow in MIME type field in MS Outlook, and overflow in ISAPI in IIS.
๐ Example: A web server processes a URL parameter and passes it to a function with a 128-byte buffer. The attacker crafts a 200-byte URL containing NOP sled + shellcode + overwritten return address. When processed, the function returns to the shellcode, giving the attacker a command shell with the server's privileges.
โญ Key Takeaways
The lecture identifies five major attack types: Trojan horses, login spoofing, logic bombs, trap doors, and buffer/stack overflows. Buffer overflows are historically the most dangerous, responsible for over 50% of security advisories for years. The core mechanism involves overwriting a fixed-size buffer using unsafe C functions like strcpy, which lack bounds checking, allowing attackers to corrupt the return address on the stack and redirect execution to malicious shellcode. The classic exploit uses a NOP sled to increase reliability and is launched by sending input data (like a crafted URL) longer than the expected buffer size. Understanding these vulnerabilities requires knowledge of memory layout, stack organization, and the specific functions that enable these attacks.
๐ง Quick Revision Questions
- What is the difference between a Trojan horse and a logic bomb in terms of how they are activated?
- How does a login spoofing attack work, and what makes it effective against users?
- When a buffer is 128 bytes and the input is 136 bytes, what specific memory locations are overwritten on the stack?
- Why is the strcpy() function considered unsafe in the context of buffer overflow attacks?
- What is the purpose of using a NOP sled in a buffer overflow exploit, and how does it increase the attacker's success rate?
๐ Lecture 43 โ Overview of todayโs lecture
๐ Overview: This lecture covers buffer overflow attacks and their prevention mechanisms, then transitions to broader security topics including viruses, worms, and mobile code security. Understanding these concepts is critical for building secure systems and defending against common exploits.
๐๏ธ Topics Covered
The lecture covers types of buffer overflow attacks including stack smashing, function pointer and longjmp buffer attacks, followed by methods for finding buffer overflows. Prevention techniques are discussed including marking stack as non-execute, StackGuard with canary types, PointGuard, Libsafe, and address obfuscation. The lecture then covers viruses and worms, their operation and spread, antivirus techniques, and concludes with mobile code security including sandboxing and Java security mechanisms.
๐ Lecture Summary
Types of buffer overflow attacks
Buffer overflow attacks exploit programs that use unsafe string functions without bounds checking. The stack smashing attack overrides the return address in a stack activation record by overflowing a local buffer variable, redirecting execution to attacker-controlled code. Attackers can also target function pointers โ overflowing a buffer can override a function pointer to redirect execution, as was used in the attack on Linux superprobe. Similarly, longjmp buffers โ overflowing a buffer next to the jump position overrides the value of pos, as used in the attack on Perl 5.003.
Causing program to exec attack code
The core goal of buffer overflow attacks is to cause the program to execute attack code. The stack smashing attack overrides the return address in the stack activation record by overflowing a local buffer variable. Function pointers can be targeted โ overflowing a buffer will override a function pointer to point to malicious code. Longjmp buffers like longjmp(pos) โ overflowing a buffer next to the jump position overrides the value of pos, redirecting control flow.
Finding buffer overflows
Hackers find buffer overflows through systematic testing. One method involves running the web server on a local machine and issuing requests with long tags, where all long tags end with "$$$$$". If the web server crashes, the attacker searches the core dump for "$$$$$" to find the overflow location. Some automated tools exist for finding buffer overflows, including eEye Retina and ISIC.
Preventing buf overflow attacks
The main problem is that unsafe C functions like strcpy(), strcat(), and sprintf() have no range checking. The "safe" versions strncpy() and strncat() are misleading because strncpy() may leave the buffer unterminated. Defenses include using type safe languages like Java or ML (but legacy code remains problematic), marking the stack as non-execute, static source code analysis, run time checking using tools like StackGuard, Libsafe, SafeC, or Purify, and black box testing using tools like eEye Retina or ISIC.
Marking stack as non-execute
The basic stack exploit can be prevented by marking the stack segment as non-executable or randomizing the stack location. Code patches exist for Linux and Solaris. However, this approach has significant problems: it does not block more general overflow exploits like heap overflows where the buffer next to a function pointer is overflowed. Some applications need an executable stack, such as LISP interpreters. The patch is not shipped by default for Linux and Solaris.
Run time checking: StackGuard
Many run-time checking techniques exist. StackGuard (by WireX) performs run time tests for stack integrity. It embeds canaries in stack frames and verifies their integrity prior to function return. Two types of canaries exist: Random canary โ a random string is chosen at program startup, inserted into every stack frame, and verified before returning from a function. To corrupt a random canary, the attacker must learn the current random string. Terminator canary โ the canary consists of characters like 0, newline, linefeed, and EOF. String functions will not copy beyond these terminators, so the attacker cannot use string functions to corrupt the stack. StackGuard is implemented as a GCC patch, requiring the program to be recompiled, with minimal performance effects (8% for Apache). The newer version PointGuard protects function pointers and setjmp buffers by placing canaries next to them, with more noticeable performance effects. However, canaries don't offer full protection โ some stack smashing attacks can leave canaries untouched.
๐ Definition โ Canary: A random or terminator value placed in a stack frame to detect buffer overflows by verifying its integrity before function return.
๐ Example: With a terminator canary containing 0, newline, linefeed, and EOF characters, if an attacker attempts to overwrite the stack using strcpy() to copy a string containing these characters, the copy will stop at the first terminator character, preventing corruption of the return address.
Run time checking: Libsafe
Libsafe (by Avaya Labs) is a dynamically loaded library that intercepts calls to strcpy(dest, src). It validates sufficient space in the current stack frame by checking |frame-pointer โ dest| > strlen(src). If there is enough space, it performs the strcpy operation; otherwise, it terminates the application.
More methods
Address obfuscation (Stony Brook, 2003) encrypts the return address on the stack by XORing it with a random string, then decrypts just before returning from the function. The attacker needs the decryption key to set the return address to a desired value. Another method is to randomize the location of functions in libc, so the attacker cannot jump directly to an exec function.
๐ก Why this matters: Run-time checking and obfuscation techniques provide defense-in-depth, even if system-level protections fail, but no single method is foolproof.
Viruses and worms
An external threat involves code transmitted to a target machine where it is executed, causing damage. The goals of a virus writer include creating a quickly spreading virus, making it difficult to detect, and making it hard to get rid of. A virus is a program that can reproduce itself by attaching its code to another program and additionally causes harm.
How Viruses Work (1)
A virus is typically written in assembly language and inserted into another program using a tool called a dropper. The virus remains dormant until the program is executed, at which point it infects other programs and eventually executes its payload. A virus can be placed at the front of an executable program, at the end, or spread over free space within the program.
How Viruses Spread
Viruses are placed where they are likely to be copied. When copied, they infect programs on the hard drive or floppy disks and may try to spread over a LAN. They can attach to innocent-looking emails โ when the email attachment runs, the virus uses the mailing list to replicate itself.
Antivirus and Anti-Antivirus Techniques
Antivirus techniques include signature-based detection, integrity checkers, and behavioral checkers. Virus avoidance strategies include using a good operating system, installing only shrink-wrapped software, using antivirus software, not clicking on attachments to email, and performing frequent backups. Recovery from a virus attack involves halting the computer, rebooting from a safe disk, and running antivirus software.
Mobile Code (1) Sandboxing
Sandboxing divides memory into 1-MB sandboxes. Instructions are checked for validity by verifying that each memory access stays within its allocated sandbox. For example, an OR instruction that computes the effective address is checked with a guard to ensure it falls within the sandbox boundaries.
Mobile Code (2) and (3)
In mobile code execution, applets can be interpreted by a Web browser. Code signing is a security mechanism where a developer signs the code with a digital signature. The signature verifies the code's origin and integrity. If the signature is valid, the code is given more privileges (e.g., full access); if not, it runs with restricted privileges in the sandbox.
๐ Definition โ Sandboxing: A security mechanism that restricts mobile code to a limited set of resources and memory regions, preventing it from accessing system resources without permission.
Java Security (1)
Java is a type safe language โ the compiler rejects attempts to misuse variables. Security checks in Java include: attempts to forge pointers, violation of access restrictions on private class members, misuse of variables by type, generation of stack over/underflows, and illegal conversion of variables to another type. These compile-time checks prevent many buffer overflow and memory corruption attacks.
โญ Key Takeaways
Buffer overflow attacks exploit the lack of bounds checking in C functions like strcpy() to overwrite return addresses, function pointers, or longjmp buffers. Multiple defense layers exist: marking the stack as non-executable prevents basic exploits but not heap overflows; StackGuard uses canaries to detect stack corruption at runtime with minimal performance overhead; and Libsafe intercepts unsafe function calls to validate buffer sizes. Viruses and worms are self-replicating programs that spread and execute payloads, countered by signature-based, integrity, and behavioral detection methods. Mobile code security relies on sandboxing to restrict untrusted applets and code signing to grant privileges based on verified origins.
๐ง Quick Revision Questions
- What are the three types of buffer overflow attack targets discussed, and what do they each overwrite?
- How does StackGuard's random canary differ from its terminator canary in implementation and defense mechanism?
- Why does marking the stack as non-executable fail to prevent all buffer overflow attacks?
- What are the three main categories of antivirus detection techniques, and what does each monitor?
- How does Java's type safety prevent buffer overflow attacks, and what specific violations does the compiler check?