π Lecture 5 β Overview of todayβs lecture
π Overview: This lecture continues the discussion on process management models and state machines, introducing the concept of process suspension. It details the contents of a Process Control Block (PCB) and examines the operating system control structures (memory, I/O, file tables). The lecture also provides a comprehensive overview of UNIX system calls for process management, including
fork,exit,wait, andexec, with practical examples.
ποΈ Topics Covered
This lecture begins with a review of previous process states and then introduces a four-state model that includes suspend states (Ready Suspend and Blocked Suspend). It covers the detailed components of a Process Control Block, including process identification, processor state information (registers, stack pointers), and process control information (scheduling, data structuring, IPC, privileges). The lecture then shifts to UNIX SVR4 process states and modes of execution (user mode vs. kernel mode). Finally, it presents a series of example programs that demonstrate the use of key UNIX system calls: fork for creating processes, exit for destroying them, wait/waitpid for synchronization, and exec for running new programs, along with explanations of zombie processes and reaping.
π Lecture Summary
Re-view of the previous lecture
The lecture begins with a brief review of foundational concepts from the previous session, including process management models and state machines, setting the stage for the introduction of more advanced states.
Process management models and state machines (contβd from the previous lecture)
Suspending Processes
Suspending a process involves swapping part or all of it to disk. This is most useful when waiting for an event that will not arrive soon (e.g., a printer or keyboard). If not done well, it can slow the system down by increasing disk I/O activity. The state transition diagram includes four key states:
- Ready β In memory, ready to execute
- Blocked β In memory, waiting for an event
- Blocked Suspend β On disk, waiting for an event
- Ready Suspend β On disk, ready to execute
π‘ Why this matters: The suspend states allow the operating system to free up main memory for other processes by temporarily moving blocked or ready processes to disk, improving overall system utilization.
Unix SVR4 Processes
UNIX SVR4 uses 9 process states. The "Preempted" and "Ready to run, in memory" states are nearly identical; a process may be preempted for a higher-priority process at the end of a system call. A Zombie state saves information to be passed to the parent of this process. Process 0 is the Swapper, created at boot. Process 1 is Init, which creates other processes.
What is in a process control block
Modes of Execution
There are two modes of execution:
- User mode: A less-privileged mode where user programs typically execute.
- System mode, control mode, or kernel mode: A more-privileged mode where the kernel of the operating system executes.
Operating System Control Structures
The operating system maintains information about the current status of each process and resource through tables constructed for each entity it manages:
- Memory Tables: Manage allocation of main memory and secondary memory to processes, protection attributes for shared memory, and information for virtual memory.
- I/O Tables: Track whether an I/O device is available or assigned, the status of an I/O operation, and the location in main memory used as source/destination for the I/O transfer.
- File Tables: Maintain information about the existence of files, their location on secondary memory, current status, and attributes (sometimes managed by a file management system).
Process Control Block
The Process Control Block (PCB) contains three main categories of information:
1. Process Identification Includes numeric identifiers such as:
- Identifier of this process
- Identifier of the process that created this process (parent process)
- User identifier
2. Processor State Information Includes:
- User-Visible Registers: Registers that may be referenced by the machine language while in user mode (typically 8 to 32, but some RISC implementations have over 100).
- Control and Status Registers: These control the operation of the processor, including:
- Program counter: Contains the address of the next instruction to be fetched
- Condition codes: Result of the most recent arithmetic or logical operation (e.g., sign, zero, carry, equal, overflow)
- Status information: Includes interrupt enabled/disabled flags, execution mode
- Stack Pointers: Each process has one or more LIFO system stacks. A stack stores parameters and calling addresses for procedure and system calls. The stack pointer points to the top of the stack.
3. Process Control Information Includes:
- Scheduling and State Information: Needed by the OS to perform scheduling. Typical items include:
- Process state: Defines readiness (e.g., running, ready, waiting, halted)
- Priority: One or more fields for scheduling priority (e.g., default, current, highest-allowable)
- Scheduling-related information: Depends on the scheduling algorithm (e.g., time waited, time executed)
- Event: Identity of event the process awaits before resuming
- Data Structuring: A process may be linked to others in a queue, ring, or parent-child relationship. The PCB may contain pointers to other processes.
- Inter-process Communication: Various flags, signals, and messages associated with communication between two independent processes may be maintained in the PCB.
- Process Privileges: Processes are granted privileges regarding memory access, instruction types, and use of system utilities and services.
π Definition β Process Control Block (PCB): A data structure in the operating system kernel that contains all information needed to manage a particular process, including its identification, processor state, and control information.
Operating system calls for process management in UNIX family of systems
Example programs invoking OS services for process management
fork: Creating new processes
The fork system call creates a new process (child process) that is identical to the calling process (parent process). It returns 0 to the child process and returns the child's PID to the parent process.
π Formula:
int fork(void) β Creates a new process. Returns 0 to child, child's PID to parent, or -1 on error.
Fork Example #1: Parent and child both run the same code. They distinguish parent from child by the return value from fork. Both start with the same state, but each has a private copy, including a shared output file descriptor. The relative ordering of their print statements is undefined.
Fork Examples #2β5: In each of these examples, both parent and child can continue forking, demonstrating the recursive nature of process creation.
exit: Destroying Process
The exit system call terminates a process. It normally returns with status 0. The atexit() function registers functions to be executed upon exit.
π Formula:
void exit(int status) β Terminates the calling process. Status 0 indicates normal termination.
Zombies
When a process terminates, it still consumes system resources (various tables maintained by the OS). This state is called a zombieβa living corpse, half alive and half dead. Reaping is performed by the parent on a terminated child. The parent is given exit status information, and the kernel then discards the process. If any parent terminates without reaping a child, the child will be reaped by the init process. Explicit reaping is only needed for long-running processes (e.g., shells and servers).
Zombie Example: The ps command shows a child process as "defunct." Killing the parent allows the child to be reaped.
Non-terminating Child Example: A child process can still be active even though the parent has terminated. It must be killed explicitly, or it will keep running indefinitely.
wait: Synchronizing with children
The wait system call suspends the current process until one of its children terminates. The return value is the PID of the child process that terminated. If child_status != NULL, the object it points to will be set to a status indicating why the child terminated.
π Formula:
int wait(int *child_status) β Suspends calling process until a child terminates. Returns child's PID. Status information is stored in child_status.
Wait Example: If multiple children complete, they are taken in arbitrary order. Macros WIFEXITED and WEXITSTATUS can be used to get information about the exit status.
Waitpid
The waitpid system call can wait for a specific process and supports various options.
π Formula:
waitpid(pid, &status, options) β Waits for a specific child process with the given PID. Allows more control than wait.
Wait/Waitpid Example Outputs: Examples fork10 (using wait) and fork11 (using waitpid) demonstrate the different behaviors.
exec: Running new programs
The exec system call loads and runs an executable at a given path with specified arguments. The path is the complete path of an executable. arg0 becomes the name of the process (typically identical to path or just the executable filename). "Real" arguments start with arg1. The list of args is terminated by a (char *)0 argument. It returns -1 if error, otherwise it doesn't return.
π Formula:
int execl(char *path, char *arg0, char *arg1, ..., 0) β Replaces the current process image with a new program. Does not return on success.
π‘ Why this matters: exec is how UNIX systems run entirely new programs within the same process. After fork, a child process typically calls exec to replace its copy of the parent's code with a new program.
β Key Takeaways
For the exam, you must understand the four-state suspension model and differentiate between Ready, Blocked, Ready Suspend, and Blocked Suspend states. Master the three main categories of the Process Control Block: process identification, processor state information (user-visible, control/status registers, stack pointers), and process control information (scheduling, data structuring, IPC, privileges). Be able to explain zombie processes, including when they occur and how they are reaped through parent-child relationships or by init. Crucially, you must know the exact behavior of fork (returns PID to parent, 0 to child), wait/waitpid (suspends parent until child terminates), and exec (replaces the process image). Finally, be prepared to trace through fork example programs to determine how many processes are created and identify the output.
π§ Quick Revision Questions
- What are the four key states in the process suspension model, and what distinguishes "Blocked Suspend" from "Blocked"?
- List the three main categories of information stored in a Process Control Block.
- What does the
fork()system call return to the parent process and to the child process? - Explain what a zombie process is and describe two ways it can be reaped (cleaned up).
- What is the fundamental difference between how
wait()andexec()affect a process's execution?
π Lecture 6 β Overview of todayβs lecture
π Overview: This lecture continues the exploration of process management in operating systems, covering fork examples, process termination and zombie processes, synchronization via wait/waitpid, and introduces the critical concept of threads for achieving concurrency. Understanding these mechanisms is essential for building efficient, concurrent systems like web servers and parallel programs.
ποΈ Topics Covered
The lecture covers continuation of fork examples from the previous lecture, the concept of zombies and reaping processes, the wait and waitpid system calls in Linux, the need for threads within processes for concurrency, an introduction to threads, and a recap of the lecture covering exceptions, processes, spawning, terminating, reaping, and replacing processes.
π Lecture Summary
Fork Example #3
- Both parent and child can continue forking
Fork Example #4
- Both parent and child can continue forking
Fork Example #5
- Both parent and child can continue forking
exit: Destroying Process
- void exit(int status) exits a process. Normally, a process returns with status 0.
- atexit() registers functions to be executed upon exit.
Zombies
- When a process terminates, it still consumes system resources (various tables maintained by the OS). This is called a "zombie" β a living corpse, half alive and half dead.
- Reaping is performed by a parent on a terminated child. The parent is given exit status information, and then the kernel discards the process.
- What if Parent Doesn't Reap? If any parent terminates without reaping a child, then the child will be reaped by the init process. Only explicit reaping is needed for long-running processes, e.g., shells and servers.
π Definition β Zombie: A process that has terminated but still consumes system resources because its parent has not yet reaped it. π‘ Why this matters: Zombie processes waste kernel resources (like process table entries). If too many accumulate, they can prevent new processes from being created.
π Zombie Example: The ps command shows the child process as "defunct". Killing the parent allows the orphaned child to be reaped by init.
π Non-terminating Child Example: A child process can still be active even though its parent has terminated. It must be killed explicitly, or else it will keep running indefinitely.
wait: Synchronizing with children
- *int wait(int child_status) suspends the current process until one of its children terminates. The return value is the pid of the child process that terminated.
- If
child_status != NULL, then the object it points to will be set to a status indicating why the child process terminated.
π Wait Example: If multiple children have completed, wait will take them in an arbitrary order. Macros like WIFEXITED and WEXITSTATUS can be used to get information about the exit status.
Waitpid
- waitpid(pid, &status, options) can wait for a specific process. It has various options.
exec: Running new programs
- **int execl(char *path, char arg0, char arg1, ..., 0) loads and runs the executable at
pathwith argsarg0,arg1, ...pathis the complete path of an executable.arg0becomes the name of the process (typically identical topathor contains only the executable filename).- "Real" arguments to the executable start with
arg1, etc. - The list of args is terminated by a
(char *)0argument.
- Returns -1 if error, otherwise doesn't return!
Summarizing
- Exceptions: Events that require nonstandard control flow, generated externally (interrupts) or internally (traps and faults).
- Processes: At any given time, the system has multiple active processes, but only one can execute at a time. Each process appears to have total control of the processor + private memory space.
- Spawning Processes: Call to
forkβ one call, two returns. - Terminating Processes: Call
exitβ one call, no return. - Reaping Processes: Call
waitorwaitpid. - Replacing Program Executed by Process: Call
execl(or variant) β one call, (normally) no return.
Concurrency
- Imagine a web server, which might like to handle multiple requests concurrently: while waiting for the credit card server to approve a purchase for one client, it could be retrieving the data requested by another client from disk, and assembling the response for a third client from cached information.
- Imagine a web client (browser), which might like to initiate multiple requests concurrently.
- Imagine a parallel program running on a multiprocessor, which might like to employ "physical concurrency": for example, multiplying a large matrix β split the output matrix into k regions and compute the entries in each region concurrently using k processors.
Whatβs in a process?
- A process consists of (at least):
- an address space
- the code for the running program
- the data for the running program
- an execution stack and stack pointer (SP) β traces state of procedure calls made
- the program counter (PC), indicating the next instruction
- a set of general-purpose processor registers and their values
- a set of OS resources (open files, network connections, sound channels, ...)
- That's a lot of concepts bundled together! Decompose into:
- an address space
- threads of control
- (other resources...)
Whatβs needed?
- In each of these examples of concurrency (web server, web client, parallel program):
- Everybody wants to run the same code
- Everybody wants to access the same data
- Everybody has the same privileges
- Everybody uses the same resources (open files, network connections, etc.)
- But you'd like to have multiple hardware execution states:
- an execution stack and stack pointer (SP)
- the program counter (PC)
- a set of general-purpose processor registers and their values
How could we achieve this?
- Given the process abstraction as we know it:
forkseveral processes and cause each to map to the same physical memory to share data. - It's really inefficient:
- space: PCB, page tables, etc.
- time: creating OS structures, fork and copy address space, etc.
Can we do better?
- Key idea: separate the concept of a process (address space, etc.) from that of a minimal "thread of control" (execution state: PC, etc.).
- This execution state is usually called a thread, or sometimes, a lightweight process.
Threads and processes
- Most modern OS's (Mach, Chorus, NT, modern UNIX) therefore support two entities:
- the process, which defines the address space and general process attributes (such as open files, etc.)
- the thread, which defines a sequential execution stream within a process
- A thread is bound to a single process/address space. Address spaces, however, can have multiple threads executing within them. Sharing data between threads is cheap: all see the same address space. Creating threads is cheap too!
- Threads become the unit of scheduling. Processes/address spaces are just containers in which threads execute.
β Key Takeaways
The fork() system call creates a child process that shares code with the parent and both can continue forking. A terminated process becomes a zombie if its parent does not reap it, consuming kernel resources until the init process adopts it. The wait() system call blocks the parent until a child terminates, returning the child's PID and exit status, while waitpid() allows waiting for a specific child with options. Concurrency is essential for modern systems like web servers, but creating multiple processes for each task is inefficient due to overhead from PCBs, page tables, and memory copying. Threads solve this by separating the execution state (PC, stack, registers) from the process (address space, resources), allowing multiple lightweight execution streams within a single process that share code, data, and resources cheaply.
π§ Quick Revision Questions
- What system resources does a zombie process still consume after termination?
- What happens to a child process if its parent terminates without reaping it?
- How does the wait() system call differ from waitpid() in terms of which child is waited for?
- What are the three components of a process that are separated when introducing threads?
- Why is using multiple processes (via fork) inefficient for achieving concurrency compared to using multiple threads?
π Lecture 7 β Overview of todayβs lecture
π Overview: This lecture explores the design space for threads, comparing user-level and kernel-level thread implementations. It explains how threads are illustrated within an address space and examines the trade-offs between the two approaches, including their respective problems and advantages, which is crucial for understanding modern operating system design for concurrency.
ποΈ Topics Covered
The lecture covers the design space for threads, including the evolution from a traditional process address space to one with threads. It explains the process/thread separation and the benefits of multithreading even on uniprocessors. The lecture details kernel threads and user-level threads, including their implementations, scheduling strategies, context switching, and the problems that arise with I/O operations and lock preemption. Finally, it summarizes the pros and cons of both user-level and kernel-level thread implementations.
π Lecture Summary
The design space
The lecture begins by contrasting an old process address space (single process with a single thread of control) with a new process address space that includes threads, showing how multiple threads of execution can coexist within a single address space.
Process/thread separation
Concurrency (multithreading) is useful for handling concurrent events (e.g., web servers and clients), building parallel programs (e.g., matrix multiply, ray tracing), and improving program structure (the Java argument). Multithreading is useful even on a uniprocessor, even though only one thread can run at a time. Supporting multithreading by separating the concept of a process (address space, files, etc.) from that of a minimal thread of control (execution state) is a big win because creating concurrency does not require creating new processes, making it "faster / better / cheaper".
Kernel threads
With kernel threads, the OS manages threads and processes, meaning all thread operations are implemented in the kernel. The OS schedules all threads in a system. If one thread in a process blocks (e.g., on I/O), the OS knows about it and can run other threads from that process, making it possible to overlap I/O and computation inside a process.
Kernel threads are cheaper than processes because there is less state to allocate and initialize. However, they're still pretty expensive for fine-grained use, being orders of magnitude more expensive than a procedure call. Thread operations are all system calls, involving context switch and argument checks, and the kernel must maintain state for each thread.
User-level threads
To make threads cheap and fast, they need to be implemented at the user level, managed entirely by a user-level library (e.g., libpthreads.a). User-level threads are small and fast, with each thread represented simply by a PC, registers, a stack, and a small thread control block (TCB). Creating a thread, switching between threads, and synchronizing threads are done via procedure calls with no kernel involvement necessary, making user-level thread operations 10-100x faster than kernel threads.
User-level thread implementation
The kernel believes the user-level process is just a normal process running code, but this code includes the thread support library and its associated thread scheduler. The thread scheduler determines when a thread runs, using queues to keep track of what threads are doing (run, ready, wait), just like the OS and processes, but implemented at user-level as a library.
How to keep a user-level thread from hogging the CPU?
- Strategy 1: Cooperation β A thread willingly gives up the CPU by calling
yield().yield()calls into the scheduler, which context switches to another ready thread. What happens if a thread never callsyield()? It hogs the CPU. - Strategy 2: Preemption β The scheduler requests a timer interrupt be delivered by the OS periodically, usually delivered as a UNIX signal. Signals are like software interrupts, but delivered to user-level by the OS. At each timer interrupt, the scheduler gains control and context switches as appropriate.
Thread context switch
For user-level threads, context switching is very simple:
- Save context of currently running thread: push machine state onto thread stack
- Restore context of the next thread: pop machine state from next thread's stack
- Return as the new thread: execution resumes at PC of next thread
This is done by assembly language and works at the level of the procedure calling convention. It cannot be implemented using procedure calls because a thread might be preempted (and then resumed) in the middle of a procedure call. C commands setjmp and longjmp are one way of doing it.
What if a thread tries to do I/O?
The kernel thread "powering" the user-level thread is lost for the duration of the synchronous I/O operation. One could have one kernel thread powering each user-level thread (no real difference from kernel threads), or have a limited-size "pool" of kernel threads powering all user-level threads in the address space, where the kernel schedules these threads obliviously to what's going on at user-level.
π‘ Why this matters: The blocking I/O problem is the main limitation of user-level threads, requiring creative solutions to maintain performance.
What if the kernel preempts a thread holding a lock?
Other threads will be unable to enter the critical section and will block (stall). Solving this requires coordination between the kernel and the user-level thread manager through "scheduler activations". Each process can request one or more kernel threads, with the process given responsibility for mapping user-level threads onto kernel threads. The kernel promises to notify the user-level before it suspends or destroys a kernel thread.
Pros and Cons of User Level threads
Pros:
- Procedure invocation instead of system calls results in fast scheduling and much better performance.
- Could run on existing OSes that don't support threads.
- Customized scheduling is useful in many cases, e.g., for garbage collection.
- Kernel space not required for thread-specific data, scaling better for large numbers of threads.
Cons:
- Blocking system calls affect all threads. Solution 1: Make the system calls non-blocking (is this a good solution?). Solution 2: Write jacket or wrapper routines with
select, but this requires re-writing parts of the system call library. - Similar problem occurs with page faults; the whole process blocks.
- Voluntary
yieldto the run-time system is necessary for context switch. Solution: Run time system may request a clock signal. - Programs that use multi-threading need to make system calls quite often; if they don't, there is usually no need to be multi-threaded.
Pros and cons of kernel level threads
Pros:
- Blocking system calls cause no problem.
- Page faults can be handled by scheduling another thread from the same process (if one is ready).
Cons:
- Cost of a system call is substantially greater. Solution: Thread recycling β don't destroy thread data structures when the thread is destroyed.
β Key Takeaways
The fundamental trade-off in thread implementation is between speed and functionality: user-level threads are 10-100x faster than kernel threads because operations use procedure calls instead of system calls, but they suffer from blocking problems when a thread performs I/O or encounters a page fault. Kernel threads solve these blocking issues because the OS can schedule other threads from the same process, but they are more expensive due to system call overhead. The key decision factors include whether the application is I/O-bound or CPU-bound, the need for customized scheduling, and the scalability requirements for large numbers of threads. Understanding these trade-offs is essential for designing efficient concurrent systems, with modern solutions often employing a hybrid approach using scheduler activations or thread pools.
π§ Quick Revision Questions
- What is the main advantage of user-level threads over kernel-level threads in terms of performance?
- What happens to a user-level thread's process when one of its threads performs a blocking I/O system call?
- How does thread recycling solve one of the disadvantages of kernel-level threads?
- What is the "scheduler activation" approach, and which problem does it solve?
- Compare how page faults are handled in user-level threads versus kernel-level threads.
π Lecture 8 β POSIX Threads (pthreads) Standard Interface and Linux Processes/Threads
π Overview: This lecture bridges the gap between the theoretical concept of threads and their practical implementation in Linux. It introduces the POSIX threads (pthreads) standard interface for C programming and explores how Linux internally represents and manages processes and threads using the same underlying data structures. Understanding these concepts is crucial for writing efficient multi-threaded programs and for comprehending how modern operating systems handle concurrency.
ποΈ Topics Covered
The lecture begins with an overview of the POSIX threads (pthreads) standard interface, covering functions for creating, terminating, and synchronizing threads. It includes a simple "Hello, World" pthreads program to illustrate thread execution. The focus then shifts to how Linux treats processes and threads, explaining the clone() system call as the core mechanism for creating threads. The fields within the task_struct process descriptor are detailed, including the process/thread states and their finite state machine (FSM) in Linux. The lecture concludes with a look ahead and a recap.
π Lecture Summary
POSIX Threads (Pthreads) Interface
The Pthreads standard defines a portable interface for approximately 60 functions that manipulate threads from C programs. The key categories of these functions include:
- Creating and reaping threads:
pthread_createis used to spawn a new thread, andpthread_joinis used to wait for a specific thread to terminate and clean up its resources. - Determining your thread ID: A thread can obtain its own unique identifier using the
pthread_selffunction. - Terminating threads: A thread can be terminated voluntarily by calling
pthread_exit, or another thread can request its termination withpthread_cancel. Theexitsystem call will terminate the entire process (all threads), while a simpleret(return from the thread's start function) terminates only the current thread. - Synchronizing access to shared variables: The interface includes functions for mutexes (e.g.,
pthread_mutex_init,pthread_mutex_lock,pthread_mutex_unlock) and condition variables (e.g.,pthread_cond_init,pthread_cond_wait,pthread_cond_timedwait).
π Definition β Pthreads: A standardized C programming interface (POSIX.1c, Threads extensions) for creating and manipulating threads.
The Pthreads "hello, world" Program
A simple "Hello, World" program demonstrates the basic usage of Pthreads. The main program uses pthread_create to create a new thread, passing it a function to execute. The main thread then calls pthread_join to wait for the newly created thread to finish its execution before the program exits. This illustrates the fundamental pattern of creating and reaping threads.
Processes and threads in Linux
Linux uses the same internal representation for both processes and threads. A thread is simply a new process that happens to share the same address space as its parent. This distinction is made when a new thread is created by the clone system call, as opposed to fork.
- fork(): Creates a new process with its own entirely new process context (address space, file descriptors, signal handlers, etc.). The child process is a complete, independent copy.
- clone(): Creates a new process with its own identity (a new PID) but is allowed to share various data structures of its parent. This fine-grained control is achieved through flags passed to the
clone()system call.
π Definition β clone(): A Linux-specific system call that creates a new process (task) with the ability to selectively share resources (like address space, file descriptors, etc.) with its parent, forming the basis for thread creation in Linux.
Main Flags for Linux clone
The clone() system call uses a set of flags to determine which parts of the parent's context are shared with the new child process. Key flags include:
- CLONE_FILES: Shares the table that identifies the open files. Both processes use the same set of file descriptors.
- CLONE_FS: Shares the table that identifies the root directory and the current working directory, as well as the value of the bit mask used to mask the initial file permissions of a new file (umask).
- CLONE_SIGHAND: Shares the table that identifies the signal handlers.
- CLONE_THREAD: Inserts this process into the same thread group as the parent. If this flag is true, it implicitly enforces
CLONE_PARENT, making the new and parent tasks siblings. This is the key flag for creating a standard POSIX thread. - CLONE_VM: Shares the address space (memory descriptor and all page tables). This is the fundamental flag for creating a thread that shares memory with its parent. π‘ Why this matters: Without
CLONE_VM,clone()behaves more likefork(), creating a separate address space.
clone()
The clone() function is the core system call for creating both processes and threads in Linux.
- fork() is actually implemented as a wrapper around
clone()with specific parameters. - The general signature is
clone(fp, data, flags, stack). (The asterisks indicate it's an internal kernel function not meant to be called directly from user space). fpis the function pointer for the thread's start routine.datais the pointer to the argument to be passed to the start routine.flagsis a bitmask of theCLONE_*flags (e.g.,CLONE_VM | CLONE_THREAD).stackis the address of the user stack that the new thread will use.- Internally,
clone()calls thedo_fork()function, which is the common workhorse for creating new tasks in the kernel.
π Formula: fork() = clone(flags = SIGCHLD, stack = 0); β A standard fork() is equivalent to calling clone() with only the SIGCHLD flag (meaning the parent is notified of child termination) and a zero for the stack (indicating a new, separate stack should be created).
Internal Kernel threads
Linux has a small number of kernel threads that run continuously in the kernel space. These are also known as daemons.
- They do not have a user address space; they only have the kernel-mapped portion of the address space.
- They are created using the
kernel_thread()function. - Process 0 (idle process): The first process, created during boot, which runs when no other tasks are runnable.
- Process 1: Spawns several kernel threads before transitioning to user mode as
/sbin/init. Key kernel threads include:- kflushd (bdflush): Flushes dirty buffers to disk under "memory pressure."
- kupdate: Periodically flushes old buffers to disk.
- kswapd: The swapping daemon.
Linux processes
In Linux terminology, processes are called tasks. The kernel maintains a list of process descriptors, which are data structures of type task_struct (defined in include/linux/sched.h).
- The maximum number of threads/processes allowed is dependent upon the amount of memory in the system.
- A user can check the current limit by viewing
/proc/sys/kernel/threads_max. By writing to that file (as superuser), the limit can be changed on the fly.
Linux Process Identity
A process is identified externally by its PID (Process ID). Internally, the kernel uses the address of the task_struct descriptor to quickly access process information.
- PIDs are dynamically allocated and reused. They are 16-bit values (up to 32767) and the system avoids immediate reuse to prevent confusion.
- To efficiently find a
task_structfrom a PID, the kernel uses a hash table. - The
currentmacro gives a pointer to thetask_structof the currently executing process. Socurrent->pidprovides the PID of the current process.
do_fork()
The function do_fork() is the core kernel routine that performs the heavy lifting of creating a new task. Its key steps are:
alloc_task_struct(): Allocates a newtask_structfor the new process.find_empty_process(): Finds an available slot in the task list for the new process.get_pid(): Allocates a unique PID for the new process.- Update ancestry: Sets up links between the new task and its parent, establishing the process tree.
- Copy components based on flags: Duplicates or shares resources (memory, files, etc.) based on the flags passed to
clone(). copy_thread(): Copies the processor-specific context (registers, kernel stack pointer) from the parent to the child and sets up the initial execution state for the new process.- Link into task list, update nr_tasks: Adds the new process's descriptor to the global list of tasks and increments the count of running tasks.
- Set TASK_RUNNING: Sets the state of the new process to
TASK_RUNNING, making it eligible to be scheduled for execution. wake_up_process(): Wakes up the new process so the scheduler can consider it.
Linux data structures for processes
The process control blocks (or descriptors) are kept in circular linked lists. The implementation in the kernel uses the structure struct list_head as the node for these lists:
struct list_head {
struct list_head *next, *prev;
};
These linked lists are circular, meaning there is no explicit head or tail node. One can start at any node and traverse the entire list. The task_struct contains a list_head member to link itself into this main task list.
Task struct contents
The task_struct contains a vast amount of information about a process. Key categories of fields include:
- Scheduling information: Information needed by the Linux scheduler, such as process priority (normal or real-time), scheduling policy, and a
counterthat tracks the remaining time quanta for the process. - Identifiers: The PID, user ID (UID), group ID (GID), and other identifiers used for resource access privileges.
- Interprocess communication: Structures to support IPC mechanisms like those found in UNIX SVR4.
- Links: Pointers to the parent, siblings (processes with the same parent), and all children of this process.
- Times and timers: The process creation time, the amount of processor time consumed, and pointers to any associated interval timers. A timer can be single-use or periodic, and a signal is sent when it expires.
- File system: Pointers to the files opened by the process, as well as pointers to the current and root directories.
- Address space: The mm_struct which defines the virtual address space assigned to the process (memory descriptor and page tables).
- Processor-specific context: The registers and stack information that constitute the context of the process, saved during context switches.
- State: The execution state of the process.
Process State
The state of a process in Linux is stored in a field within the task_struct. The primary states are:
- Running (TASK_RUNNING): This state value corresponds to two states: A process is either executing on a CPU or it is ready to execute and waiting in a run queue. π‘ Why this matters: The scheduler only operates on processes in the
TASK_RUNNINGstate. - Interruptible (TASK_INTERRUPTIBLE): This is a blocked state. The process is waiting for an event (e.g., completion of I/O, a signal, or availability of a resource). It can be woken up by receiving a signal, which allows it to handle the signal before or instead of the event it was waiting for.
- Uninterruptible (TASK_UNINTERRUPTIBLE): This is another blocked state. The difference from
TASK_INTERRUPTIBLEis that a process in this state is waiting directly on hardware conditions (e.g., waiting for a disk drive to become ready) and therefore will not handle any signals. It must wait for the specific hardware event to complete. - Stopped (TASK_STOPPED): The process has been halted, typically by a signal (e.g.,
SIGSTOPorSIGTSTP). It can only resume execution by a positive action from another process (e.g., aSIGCONTsignal). This is often used for debugging. - Zombie (EXIT_ZOMBIE): The process has terminated (finished execution) but its
task_structstill remains in the process table. This is because the parent process has not yet calledwait()orwaitpid()to read the child's exit status. Once the parent does, the zombie is reaped (itstask_structis deallocated). π‘ Why this matters: A zombie process uses up a process table slot but consumes no other system resources. If the parent fails to reap its children, a large number of zombies can accumulate.
π Definition β Zombie: A process state where the process has finished execution but still has an entry in the process table (its task_struct), waiting for its parent to read its exit status.
β Key Takeaways
The most critical concepts from this lecture are the Linux-specific approach to threads and the central role of the task_struct. Linux treats threads as lightweight processes that share resources via the clone() system call, which provides fine-grained control through flags like CLONE_VM and CLONE_THREAD. The task_struct is the master data structure for every process and thread, containing everything from scheduling information and memory maps to its execution state. Understanding the process state machineβespecially the TASK_RUNNING, TASK_INTERRUPTIBLE, TASK_UNINTERRUPTIBLE, and EXIT_ZOMBIE statesβis fundamental to grasping process lifecycles and scheduling behavior.
π§ Quick Revision Questions
- What is the difference between
fork()andclone()in Linux? - Name the key fields within a Linux
task_structthat relate to file system information, memory management, and scheduling. - What does
CLONE_VMdo? Why is it critical for creating a thread? - A process is waiting for a signal. In which
TASK_*state is it most likely to be? - What is the difference between an "interruptible" sleep and an "uninterruptible" sleep in Linux?