π Lecture 14 β Overview of todayβs lecture
π Overview: This lecture explores the various paradigms of thread usage in operating systems, drawing from the Hauser et al. paper. It examines how threads are used to exploit CPU and I/O parallelism and for program structuring, while also discussing the pros and cons of different thread usage paradigms. Understanding these paradigms is crucial for designing robust and efficient multi-threaded applications.
ποΈ Topics Covered
The lecture begins with an overview of thread usage paradigms, covering uses of threads for CPU parallelism, I/O parallelism, and program structuring. It then details ten paradigms from the Hauser et al. paper: defer work, general pumps, slack processes, sleepers, one-shots, deadlock avoidance, rejuvenation, serializers, encapsulated fork, and exploiting parallelism. Each paradigm is explained with examples, and the lecture also discusses Bohr-bugs and Heisenbugs in the context of thread-related issues.
π Lecture Summary
Uses of threads
Threads are used for three main purposes in operating systems and applications. First, to exploit CPU parallelism, allowing a program to run two CPUs at once. Second, to exploit I/O parallelism, enabling the system to run I/O while computing or to perform multiple I/O operations simultaneously, such as listening to a window while running code (e.g., allowing commands during an interactive game). Third, threads are used for program structuring, for example, to implement timers.
Paradigms of thread usage (from Hauser et al paper)
The lecture outlines ten distinct paradigms of thread usage derived from the Hauser et al. paper. These paradigms include: defer work, general pumps, slack processes, sleepers, one-shots, deadlock avoidance, rejuvenation, serializers, encapsulated fork, and exploiting parallelism.
Defer work.
This is a very common scenario for thread usage. The client may see an unexpected response because something the client requested is fired off to happen in the background. Examples include forking off a document print operation, updating a window, or sending an email message. The issue arises if the thread hangs for some reason; the client may see confusing behavior on a subsequent request. π‘ Why this matters: Deferring work improves responsiveness but introduces complexity in error handling and state management.
Pumps
Pumps are components of producer-consumer pipelines that take input in, operate on it, then output it downstream. Their value is that they can absorb transient rate mismatches between producers and consumers. A slack process is a type of pump used to explicitly add delay, employed when trying to group small operations into batches.
Sleepers, one-shots
Sleepers and one-shots are threads that wait for some event, then trigger, then wait again. Examples include calling a procedure every 20ms or after some timeout. The lecture notes that you can think of a device interrupt handler as a kind of sleeper thread.
Deadlock avoiders
A deadlock avoider is a thread created to perform some action that might have blocked, launched by a caller who holds a lock and doesnβt want to wait. This prevents the caller from being blocked while holding the lock, which could otherwise lead to deadlock.
Bohr-bugs and Heisenbugs
Bruce Lindsey introduced these terms, referring to models of the atom. A Bohr-bug is like a Bohr nucleus β a nice solid little thing that you can hit reproducibly and hence can fix. A Heisenbug is hard to pin down; if you localize an instance, the bug shifts elsewhere. Heisenbugs result from non-deterministic executions, old corruption in data structures, and similar issues.
Task rejuvenation
This is a nasty style of thread. The application had multiple major subactivities, such as an input handler and renderer. When something awful happened, the approach was to create a new instance and pray. This is a reactive, unreliable method of handling failures. π‘ Why this matters: Rejuvenation is a sign of poor error handling design and should be avoided in favor of robust error recovery mechanisms.
Others
Serializers involve a queue and a thread that removes work from it and processes that work item by item. The code pattern is:
for (;;) { get_next_event(); handle_event(); }
Concurrency exploiters are used for multiple CPUs. Encapsulated forks are hidden threads used in library packages, where the library creates threads internally without the caller's explicit knowledge.
β Key Takeaways
The lecture presents ten paradigms of thread usage, each with specific advantages and drawbacks. The most critical points for an exam are: understanding why defer work can lead to confusing client behavior if threads hang; grasping the role of pumps in absorbing rate mismatches in pipelines; recognizing that deadlock avoiders prevent blocking while holding locks; and being able to distinguish Bohr-bugs (reproducible) from Heisenbugs (non-deterministic and shifting). Additionally, you must understand that serializers process work item by item from a queue, while encapsulated forks hide thread creation in libraries.
π§ Quick Revision Questions
- What are the three main uses of threads mentioned in the lecture?
- What is the primary issue with the "defer work" paradigm when a thread hangs?
- What is a pump, and what value does it provide in producer-consumer pipelines?
- What is the key difference between a Bohr-bug and a Heisenbug?
- In the context of deadlock avoiders, why does a caller launch a new thread rather than performing the action directly?
π Lecture 15 β Threads considered marvelous
π Overview: This lecture examines the dual nature of threadsβtheir benefits for handling blocking operations and their fundamental drawbacks stemming from non-determinism. It then transitions to event-oriented programming as an alternative paradigm and introduces CPU scheduling fundamentals, beginning with the First Come First Served (FCFS) algorithm and its convoy effect problem.
ποΈ Topics Covered
The lecture covers the benefits and harms of threads, classic thread-related issues like address space bloating and synchronization overhead, the event-oriented paradigm as a solution, goals of the perfect scheduler, problem cases in scheduling, and a detailed examination of First Come First Served (FCFS/FIFO) scheduling including the convoy effect with a numerical example.
π Lecture Summary
Threads considered marvelous
Threads are wonderful when some action may block for a while, such as a slow I/O operation or RPC. Your code remains clean and "linear". Moreover, aggregated performance is often far higher than without threading.
Threads considered harmful
They are fundamentally non-deterministic, hence invite Heisenbugs (bugs that disappear when you try to debug them). Reentrant code is really hard to write. Surprising scheduling can be a huge headache. When something "major" changes the state of a system, cleaning up threads running based on the old state is a pain.
Classic issues
Threads that get forked off, then block for some reason cause the address space soon to bloat due to number of threads increasing beyond a threshold, causing the application to crash. Erratic use of synchronization primitives makes the program incredibly hard to debug, with some problems seen only now and then. As threads grow in number, synchronization overhead becomes significant. Semaphore and lock queues become very large due to waiting threads.
π‘ Why this matters: These classic issues explain why multithreaded programs often fail mysteriously in production but work fine in testingβthe non-deterministic nature means bugs appear only under specific timing conditions.
Bottom line?
Concurrency bugs are incredibly common and very hard to track down and fix. Programmers find concurrency unnatural. The question becomes: try to package it better? Or identify software engineering paradigms that can ease the task of gaining high performance?
Event-oriented paradigm
The classic solution is to build an event-driven application and use threads as helpers. Connect each "external event source" to a main hand-crafted "event monitoring routine". This often will use signals or a kernel-supplied event notification mechanism to detect that I/O is available. Then package the event as an event object and put this on a queue. Kick off the event scheduler if it was asleep. The scheduler de-queues events, processes them one by one, and forks lightweight threads as needed (when blocking I/O is required). The Flash web server was built on this paradigm.
Problems with the architecture?
The event-oriented architecture only works if all the events show up and none is missed. It depends on the OS not "losing" eventsβthe event notification mechanism must be efficient and scalable. The scheduler needs a way to block when no work to do and must be sure that event notification can surely wake it up. Common event notification mechanisms in Linux, Windows, etc. are select() and poll(). New much more efficient and scalable mechanisms include epoll (2.6 and above Linux kernel only) and IOCompletionPorts in Windows NT, XP, etc.
Goals of "the perfect scheduler"
The goals are to: minimize latency (metric = response time at user time scales ~50-150 milliseconds, or job completion time); maximize throughput (maximize number of jobs per unit time); maximize utilization (keep CPU and I/O devices busyβa recurring theme with OS scheduling); and ensure fairness (everyone gets to make progress, no one starves).
Problem cases
I/O goes idle because of blindness about job types. Optimization involves favoring jobs of type "A" over "B"βif there are lots of A's, B's starve. An interactive process gets trapped behind others, making response time worsen for no reason. Regarding priorities: if job A depends on job B, but A's priority is greater than B's priority, then B never runs.
First come first served (FCFS or FIFO)
This is the simplest scheduling algorithm: run jobs in the order that they arrive. In uni-programming, run until done (non-preemptive). In multi-programming, put job at back of queue when it blocks on I/O. The advantage is simplicity.
FCFS (2)
The disadvantage is that wait time depends on arrival order. It is unfair to later jobs, with the worst case being a long job arriving first.
π Example: three jobs (times: A=100, B=1, C=2) arrive nearly simultaneously. What's the average completion time?
- Order of execution: A first (100), then B (100+1=101), then C (101+2=103)
- Completion times: A=100, B=101, C=103
- Average completion time: (100+101+103) / 3 = 304/3 β 101.33 time units
FCFS Convoy effect
A CPU-bound job will hold CPU until done, or it causes an I/O burst (rare occurrence, since the thread is CPU-bound). This creates long periods where no I/O requests are issued and the CPU is held. The result is poor I/O device utilization.
π Example: one CPU bound job, many I/O bound jobs:
- CPU bound runs (I/O devices idle)
- CPU bound blocks
- I/O bound job(s) run, quickly block on I/O
- CPU bound runs again
- I/O completes
- CPU bound still runs while I/O devices idle (continues...)
π Definition β Convoy effect: The phenomenon where a single long CPU-bound job holds the CPU, causing many short I/O-bound jobs to wait, leading to poor I/O device utilization and overall system performance degradation.
β Key Takeaways
Threads offer clean linear code and high performance for blocking operations, but introduce critical problems: non-determinism leading to Heisenbugs, address space bloat from blocked threads, and significant synchronization overhead. The event-oriented paradigm addresses this by using an event scheduler with lightweight threads, but depends on reliable OS event notification mechanisms like select(), poll(), epoll, or IOCompletionPorts. The perfect scheduler must balance latency, throughput, utilization, and fairness, but faces real problems like starvation, priority inversions, and the convoy effect. FCFS/FIFO scheduling, while simple, exhibits the convoy effect where a single CPU-bound job causes poor I/O utilization, demonstrated numerically with average completion time of 101.33 for jobs of lengths 100, 1, and 2.
π§ Quick Revision Questions
- What are Heisenbugs and why are they problematic in threaded programs?
- Describe the classic issue where threads that get forked off and then block can cause application crashes.
- How does the event-oriented paradigm solve threading problems, and what are its dependencies on the OS?
- Why does the convoy effect cause poor I/O device utilization in FCFS scheduling?
- Calculate the average completion time for jobs with lengths 5, 3, and 8 arriving nearly simultaneously under FCFS scheduling.
π Lecture 16 β Round robin (RR)
π Overview: This lecture introduces Round Robin scheduling as a solution to job monopolization, then explores its tradeoffs and disadvantages. It progresses to priority scheduling, discusses priority inversion and inheritance, and covers Shortest Time to Completion First (STCF) as an optimal approach, including its practical limitations and approximations like multi-level feedback queues.
ποΈ Topics Covered
The lecture covers Round Robin scheduling with its time slice mechanism and context switching tradeoffs, then moves to priority scheduling with static and dynamic priorities. It addresses thread dependencies through priority inversion and inheritance, explains Shortest Time to Completion First (STCF) and its optimality proof, discusses the challenge of predicting job length, and presents practical approximations including the elevator algorithm for disks and multi-level feedback queues.
π Lecture Summary
Round robin (RR)
Round Robin solves the problem of a job monopolizing the CPU by interrupting it. Each job runs for a defined time slice; when the time is up or the job blocks, it moves to the back of a FIFO queue. Most systems use some variant of this approach. The advantages are fair allocation of CPU across jobs and low average waiting time when job lengths vary.
π Definition β Round Robin (RR): A CPU scheduling algorithm where each ready job runs for a fixed time slice, then is moved to the back of a FIFO queue, ensuring fair CPU distribution.
π Formula: Average completion time = (sum of completion times of all jobs) / (number of jobs)
π Example: If two jobs of time=100 each run under RR with a time slice, average completion time would be higher than FCFS because both jobs finish later; FCFS for same jobs gives completion times of 100 and 200 for an average of 150, while RR interleaving means both finish around 200, giving an average of ~200.
RR Time slice tradeoffs
Performance depends heavily on the length of the time slice. Context switching is not a free operation. If the time slice is set too high, you effectively get FCFS because processes finish or block before their slice is up anyway. If it's set too low, you spend all your time context switching between threads. Typical time slices are set to about 50-100 milliseconds, while context switches cost about 0.5-1 millisecond. The moral is that context switching is usually negligible (less than 1% per time slice) unless you context switch too frequently and lose all productivity.
π‘ Why this matters: Choosing the right time slice is a critical design decision that balances responsiveness against overhead, directly impacting system performance.
Priority scheduling
Not all jobs are equal, so they are ranked. Each process has a priority; the system runs the highest priority ready job and uses round robin among processes of equal priority. Priorities can be static (fixed) or dynamic (changing over time), or both (as in Unix). Most systems use some variant of this approach. Common uses include coupling priority to job characteristics: to fight starvation, increase priority as time since last ran increases; to keep I/O busy, increase priority for jobs that often block on I/O. However, priorities can create deadlock because a high priority always runs over a low priority.
π Definition β Priority scheduling: A scheduling algorithm where each process is assigned a priority, and the highest priority ready job is selected to run, with round robin among equal priorities.
Handling thread dependencies
Priority inversion occurs when a low-priority thread holds a lock needed by a high-priority thread. For example, T1 is high priority, T2 is low priority; T2 acquires lock L. In Scene 1: T1 tries to acquire L, fails, and spins, but T2 never gets to run. In Scene 2: T1 tries to acquire L, fails, and blocks; when T3 enters the system at medium priority, T2 still never gets to run. Scheduling means deciding who should make progress, and obviously a thread's importance should increase with the importance of those that depend on it. This leads to priority inheritance, where a lower-priority thread temporarily inherits the priority of a higher-priority thread waiting for a resource it holds.
π Definition β Priority inheritance: A mechanism where a lower-priority thread temporarily inherits the priority of a higher-priority thread that is waiting for a resource held by the lower-priority thread.
Shortest time to completion first (STCF)
STCF (or shortest-job-first) runs whatever job has the least amount of work to do. It can be pre-emptive or non-pre-emptive. This algorithm is provably optimal: moving a shorter job before a longer job improves the waiting time of the short job more than it harms the waiting time of the long job.
π Formula: Average completion time = (sum of completion times) / (number of jobs)
π Example: Three jobs A (time=1), B (time=2), C (time=100). Under STCF: A finishes at 1, B at 3, C at 103. Average completion = (1+3+103)/3 = ~35. This compares to FCFS where if C runs first, average completion = (100+102+103)/3 = ~101.7.
STCF Optimality Intuition
Consider 4 jobs a, b, c, d run in lexical order. The first (a) finishes at time a, the second (b) finishes at time a+b, the third (c) finishes at time a+b+c, and the fourth (d) finishes at time a+b+c+d. Therefore, average completion = (4a+3b+2c+d)/4. Minimizing this requires that a β€ b β€ c β€ d, meaning jobs should run in order of increasing length.
π Formula: Average completion time for n jobs run in order = (nΓtβ + (n-1)Γtβ + ... + 1Γtβ)/n
STCF β The Catch
The problem with STCF is how to know the job length in advance. It requires predicting how long a job will take, which is difficult in practice.
How to know job length?
One approach is to have the user tell us, and if they lie, kill the job β but this is not useful in practice. Alternatively, use the past to predict the future: a long-running job will probably take a long time more. View each job as a sequence of sequentially alternating CPU and I/O bursts. If previous CPU bursts in the sequence have run quickly, future ones will too (usually). The challenge is what to do when the past does not equal the future.
Approximate STCF
The approximation of STCF predicts the length of the current CPU burst using the length of the previous burst. The system records the length of the previous burst (0 when just created). At a scheduling event (unblock, block, exit, etc.), it picks the job with the smallest past run length off the ready queue.
Practical STCF
For disks, it is possible to predict the length of the next job! A job is a request from the disk, and job length approximates the cost of moving the disk arm to the position of the requested disk block (farther away = more costly). STCF for disks is called shortest-seek-time-first (SSTF): do the read/write request closest to the current position. It can be pre-emptive: if new jobs arrive that can be serviced on the way, do these too. The problem with SSTF is that it can starve faraway requests. The solution is the elevator algorithm: the disk arm has a direction, and it does the closest request in that direction, sweeping from one end to the other.
π Definition β Elevator algorithm (SCAN): A disk scheduling algorithm where the disk arm moves in one direction, servicing the closest request in that direction, then reverses direction at the end, analogous to an elevator.
~STCF vs RR
Two processes P1 and P2: with a 1ms time slice, RR would switch to P1 9 times for no reason (since P1 would still be blocked on I/O). ~STCF offers better I/O utilization because it avoids unnecessary context switches.
Generalizing: priorities + history
~STCF is a good core idea but doesn't have enough state. The usual STCF problem is starvation (which occurs when long jobs never get CPU because shorter jobs keep arriving). The solution is to compute priority as a function of both CPU time the process has consumed and time since the process last ran.
Multi-level feedback queue (or exponential queue) is a priority scheme where priorities are adjusted to penalize CPU-intensive programs and favor I/O-intensive programs. It was pioneered by CTSS at MIT in 1962.
π Definition β Multi-level feedback queue: A scheduling algorithm that uses multiple priority queues and adjusts process priorities dynamically based on observed behavior, penalizing CPU-bound processes and favoring I/O-bound processes.
Visual aid of a multi-level system
Priorities and time-slices change as needed depending on the characteristics of the process. Processes that use their full time slice (CPU-bound) get moved to lower priority queues with larger time slices, while processes that block before their slice ends (I/O-bound) stay at higher priority queues.
π‘ Why this matters: Multi-level feedback queues provide a practical, adaptive scheduling approach that balances responsiveness for interactive tasks with throughput for CPU-intensive tasks without requiring a priori knowledge of job lengths.
β Key Takeaways
Round Robin solves CPU monopolization by using time slices but suffers from context switching overhead and poor performance for equal-sized jobs compared to FCFS. Priority scheduling introduces ranking but can cause priority inversion, which is resolved by priority inheritance. STCF is provably optimal for minimizing average completion time but requires knowing job lengths in advance, which is impractical. Approximations like predicting from past behavior and multi-level feedback queues offer practical solutions that dynamically adjust priorities based on observed CPU and I/O patterns, with the elevator algorithm solving starvation in disk scheduling.
π§ Quick Revision Questions
- What is the main disadvantage of Round Robin scheduling when all jobs have equal size?
- How does priority inheritance solve the priority inversion problem?
- Why is STCF considered provably optimal, and what is its main practical limitation?
- How does the multi-level feedback queue address the starvation problem in priority scheduling?
- What is the elevator algorithm, and how does it solve the starvation problem in disk scheduling?
π Lecture 17 β Some Problems with Multilevel Queue Concept & Linux Scheduling & Lottery Scheduling & Multiprocessor Systems
π Overview: This lecture addresses fundamental flaws in traditional multilevel queue scheduling, introduces Linux's real-time scheduling classes as an improvement, proposes Lottery Scheduling as a probabilistic alternative to ad hoc priority systems, and classifies multiprocessor systems by coupling and granularity of parallelism. These concepts are critical for understanding how modern OSes balance fairness, responsiveness, and scalability across diverse workloads.
ποΈ Topics Covered
The lecture begins by diagnosing problems with multilevel queues (starvation and predictive inaccuracy). It then summarizes classic scheduling algorithms (FIFO, RR, STCF, Multilevel Feedback). Next, it covers Linux scheduling with its three classes (SCHED_FIFO, SCHED_RR, SCHED_OTHER) and specific rules. Lottery scheduling is introduced as a simpler, probabilistic method using tickets. Finally, the lecture classifies multiprocessor systems by coupling type and granularity of parallelism (coarse, medium, fine-grained).
π Lecture Summary
Some problems with multilevel queue concept
- Canβt low priority threads starve? The ad hoc fix is: when a thread is skipped over, increase its priority to prevent indefinite starvation.
- What about when past doesnβt predict future? Example: a CPU-bound process switches to I/O-bound. Past behavior is no longer a good predictor. The solution is to let past predictions age, so older data counts less toward the current view of the world.
π Definition β Aging: A technique where the weight of a process's past behavior decreases over time, so recent behavior influences scheduling decisions more than older history.
Summary
- FIFO: + simple; - short jobs can get stuck behind long ones; poor I/O performance.
- RR: + better for short jobs; - poor when jobs are the same length.
- STCF: + optimal for average response time and average time-to-completion; - hard to predict the future; - unfair to long jobs.
- Multi-level feedback: + approximates STCF; - unfair to long-running jobs.
π‘ Why this matters: No single algorithm is perfect; each trades off simplicity, fairness, responsiveness, and predictability.
Some Unix scheduling problems
- How does the priority scheme scale with the number of processes?
- How to give a process a given percentage of CPU?
- OS implementation problem: The OS takes precedence over user processes, but a user process can create lots of kernel work (e.g., many network packets arrive), forcing the OS to process them during read/write system calls.
Linux Scheduling
- Builds on traditional UNIX multi-level feedback queue scheduler by adding two new scheduling classes.
- Linux scheduling classes:
- SCHED_FIFO: FCFS real-time threads.
- SCHED_RR: round-robin real-time threads.
- SCHED_OTHER: other non-real-time threads.
- Multiple priorities may be used within a class. Priorities in real-time classes are higher than non-real-time classes.
- Rules for SCHED_FIFO:
- The system will not interrupt a SCHED_FIFO thread except in the following cases: a. Another FIFO thread of higher priority becomes ready. b. The executing FIFO thread blocks on I/O, etc. c. The executing FIFO thread voluntarily gives up the CPU (e.g., terminates or yields).
- When an executing FIFO thread is interrupted, it is placed in the queue associated with its priority.
- SCHED_RR is similar to SCHED_FIFO except that there is a time-slice. On expiry of the time slice, if the thread is still executing, it is preempted and another thread from either SCHED_FIFO or SCHED_RR is selected.
- SCHED_OTHER is managed by the traditional UNIX scheduling algorithms (multi-level feedback queue).
Lottery scheduling: random simplicity
- Problem: The whole priority thing is really ad hoc. How to ensure processes will be equally penalized under load?
- Lottery scheduling: Very simple idea:
- Give each process some number of lottery tickets.
- On each scheduling event, randomly pick a ticket.
- Run the winning process.
- To give process P n% of CPU, give it (total tickets) * n%.
- How to use?
- Approximate priority: low-priority β few tickets; high-priority β many tickets.
- Approximate STCF: give short jobs more tickets, long jobs fewer.
- Key: If a job has at least 1 ticket, it will not starve.
- Grace under load change:
- Add or delete jobs (and their tickets) affects all proportionally.
- Example: Give all jobs 1/n of CPU? 4 jobs, 1 ticket each β each gets (on average) 25% of CPU. Delete one job β automatically adjusts to 33% of CPU!
- Easy priority donation: Donate tickets to a process youβre waiting on. Its CPU% scales with tickets of all waiters.
π Definition β Lottery Scheduling: A probabilistic CPU scheduling algorithm where each process receives a number of tickets, and a random ticket draw decides which process runs next; CPU share is proportional to ticket count.
π Formula: Process P gets n% of CPU = (tickets of P / total tickets) Γ 100% π Example: With 4 jobs, each gets 1 ticket: total = 4 tickets. Each job gets (1/4) Γ 100% = 25% of CPU. If one job is deleted, total = 3 tickets, each remaining job gets (1/3) Γ 100% β 33% of CPU.
Classifications of Multiprocessor Systems
- Loosely coupled or distributed multiprocessor, or cluster: Each processor has its own memory and I/O channels.
- Functionally specialized processors: Such as I/O processor, controlled by a master processor.
- Tightly coupled multiprocessing: Processors share main memory, controlled by operating system.
π Definition β Tightly Coupled Multiprocessing: A multiprocessor architecture where all processors share a single main memory and are coordinated by the OS, enabling coherent data sharing.
Granularity of parallelism
- Coarse and Very Coarse-Grained Parallelism: Synchronization among processes at a very gross level (after > 2000 instructions on average). Good for concurrent processes on a multiprogrammed uniprocessor. Can be supported on a multiprocessor with little change.
- Medium grained parallelism: Single application is a collection of threads. Threads usually interact frequently.
- Fine-Grained Parallelism: Highly parallel applications. Specialized and fragmented area.
π Definition β Granularity of Parallelism: The average amount of computation (in instructions) between synchronization points in a parallel program; coarse grain has large intervals, fine grain has small intervals.
β Key Takeaways
For the exam, remember that multilevel queues suffer from starvation and predictive inaccuracy, solved by priority boosting and aging. Linux scheduling extends the traditional feedback queue with SCHED_FIFO (non-preemptible except by higher priority or blocking) and SCHED_RR (preemptible via time-slice). Lottery scheduling solves the ad hoc priority problem by allocating CPU proportionally via random ticket draws, automatically adjusting under load changes and enabling simple priority donation. Multiprocessor systems are classified by coupling (loosely coupled clusters vs. tightly coupled shared memory) and parallelism granularity (coarse > 2000 instructions, medium threads, fine-grained). STCF is optimal in theory but impractical because it requires predicting the future.
π§ Quick Revision Questions
- What two main problems does the multilevel queue concept face, and what are the ad hoc solutions?
- List the three Linux scheduling classes and explain the key rule difference between SCHED_FIFO and SCHED_RR.
- In lottery scheduling, if you have 5 jobs each with 10 tickets, what percentage of CPU does each get? What happens if you add two more jobs each with 10 tickets?
- How does lottery scheduling automatically handle graceful adjustment when jobs are added or removed?
- What is the difference between coarse-grained and fine-grained parallelism in terms of the number of instructions between synchronizations?
π Lecture 18 β Assignment of Processes to Processors
π Overview: This lecture covers how processes and threads are assigned to processors in multiprocessor systems, comparing different scheduling architectures. It also introduces real-time systems, distinguishing between hard and soft real-time requirements and outlining the characteristics of real-time operating systems.
ποΈ Topics Covered
The lecture covers assignment of processes to processors using group scheduling, global queues, master/slave architecture, and peer architecture. It then discusses process scheduling with single and multiple queues, thread scheduling and multiprocessor thread scheduling using dedicated processor assignment and dynamic scheduling. Load sharing, its disadvantages, gang scheduling, and dedicated processor assignment are explained in detail. Finally, the lecture introduces real-time systems, hard vs. soft real-time systems, and the characteristics of real-time operating systems including determinism, responsiveness, and user control.
π Lecture Summary
Assignment of Processes to Processors
Treat processors as a pooled resource and assign process to processors on demand. One approach is to permanently assign a process to a processor, known as group or gang scheduling. This dedicates a short-term queue for each processor, resulting in less overhead, but a processor could be idle while another processor has a backlog. An alternative is a global queue, where processes are scheduled to any available processor.
The master/slave architecture has key kernel functions always run on a particular processor. The master is responsible for scheduling, and the slave sends service requests to the master. Disadvantages are that failure of the master brings down the whole system and the master can become a performance bottleneck. The peer architecture allows the operating system to execute on any processor, with each processor doing self-scheduling, but this complicates the operating system.
Process Scheduling
Process scheduling uses a single queue for all processes, but multiple queues are used for priorities. All queues feed to the common pool of processors.
Thread Scheduling
Thread scheduling executes separate from the rest of the process. An application can be a set of threads that cooperate and execute concurrently in the same address space. Threads running on separate processors yield a dramatic gain in performance.
Multiprocessor Thread Scheduling
Dedicated processor assignment threads are assigned to a specific processor. Dynamic scheduling allows the number of threads to be altered during the course of execution.
Load Sharing
Load is distributed evenly across the processors. No centralized scheduler is required. Load sharing uses global queues and is the most commonly used paradigm, used by Linux, Windows, and several UNIX flavors on multi-processor machines.
Disadvantages of Load Sharing
The central queue needs mutual exclusion, which may be a bottleneck when more than one processor looks for work at the same time. Preemptive threads are unlikely to resume execution on the same processor, so cache use is less efficient. If all threads are in the global queue, all threads of a program will not gain access to the processors at the same time.
Gang Scheduling
Gang scheduling is the simultaneous scheduling of threads that make up a single process. It is useful for applications where performance severely degrades when any part of the application is not running, and threads often need to synchronize with each other.
Dedicated Processor Assignment
When an application is scheduled, its threads are assigned to processors using a graph theoretic optimization algorithm on the control flow graph (CFG) of the application where each node in the CFG is a thread. Disadvantages are that some processors may be idle and there is no multiprogramming of processors.
Real-Time Systems
The correctness of the system depends not only on the logical result of the computation but also on the time at which the results are produced. Tasks or processes attempt to control or react to events that take place in the outside world. These events occur in "real time" and tasks must be able to keep up with them.
Hard real-time systems
If deadlines are not met, results are catastrophic. Examples include air traffic control and aircraft auto-pilot.
Soft real-time systems
The system tries to meet all deadlines in a best effort manner, and succeeds in meeting most of them. The overall percentage of met deadlines is very high on the average. Missing a few occasionally does not result in catastrophic conditions.
Characteristics of Real-Time Operating Systems
Deterministic: Operations are performed at fixed, predetermined times or within predetermined time intervals. It is concerned with how long the operating system delays before acknowledging an interrupt and there is sufficient capacity to handle all the requests within the required time.
Responsiveness: How long, after acknowledgment, it takes the operating system to service the interrupt. This includes the amount of time to begin execution of the interrupt, the amount of time to perform the interrupt, and the effect of interrupt nesting.
User control: The user specifies priority, specifies paging, what processes must always reside in main memory, disks algorithms to use, and the rights of processes.
β Key Takeaways
The key to multiprocessor scheduling is choosing between architectures like master/slave (single point of failure and bottleneck) and peer (complex but robust), and between approaches like gang scheduling (synchronized threads) and load sharing (efficient but has cache and bottleneck issues). Real-time systems are defined by their timing correctness, with hard real-time systems requiring absolute deadline adherence to avoid catastrophic failure, while soft real-time systems can tolerate occasional missed deadlines. The three critical characteristics of a real-time OS are determinism (predictable interrupt acknowledgment), responsiveness (fast interrupt handling), and user control (priority and resource allocation authority).
π§ Quick Revision Questions
- What are the two main approaches to assigning processes to processors, and what is the key difference between them?
- Describe the major disadvantage of the master/slave architecture in multiprocessor systems.
- Why does load sharing lead to inefficient cache use?
- What is the fundamental difference between hard real-time and soft real-time systems regarding the consequences of missing a deadline?
- List the three key characteristics of a real-time operating system and briefly explain what each one means.
π Lecture 19 β Characteristics of Real-Time Operating Systems
π Overview: This lecture examines the defining characteristics and features of Real-Time Operating Systems (RTOS), focusing on reliability, fail-soft operation, and specialized scheduling techniques. It also introduces fundamental concepts of dynamic memory allocation, comparing explicit and implicit allocators, and explains the process memory image structure.
ποΈ Topics Covered
The lecture covers the characteristics of RTOS including reliability and fail-soft operation, followed by detailed features such as fast process switching and priority-based preemptive scheduling. It then explores four categories of real-time scheduling: static table-driven, static priority-driven preemptive, dynamic planning-based, and dynamic best effort. Deadline scheduling information parameters are listed, followed by Rate Monotonic Scheduling (RMS). The lecture concludes with dynamic memory allocation concepts, comparing explicit vs. implicit allocators, and describes the process memory image with assumptions and constraints.
π Lecture Summary
Characteristics of Real-Time Operating Systems
Real-Time Operating Systems prioritize reliability because degradation of performance may have catastrophic consequences. The system must support fail-soft operation, which is the ability of a system to fail in such a way as to preserve as much capability and data as possible. This property contributes to system stability.
Features of Real-Time Operating Systems
RTOS systems are designed with specific features to meet timing constraints. These include fast process or thread switch, small size, and the ability to respond to external interrupts quickly. They support multitasking with inter-process communication tools such as semaphores, signals, and events. RTOS uses special sequential files that can accumulate data at a fast rate and implements preemptive scheduling based on priority. Other features include minimization of intervals during which interrupts are disabled, the ability to delay tasks for a fixed amount of time, and support for special alarms and timeouts.
Real-Time Scheduling
Real-time scheduling is classified into four approaches. Static table-driven scheduling determines at load or even compile time when a task begins execution. Static priority-driven preemptive scheduling uses a traditional priority-driven scheduler. Dynamic planning-based scheduling determines feasibility at run time. Dynamic best effort scheduling performs no feasibility analysis.
Deadline Scheduling
Deadline scheduling uses several pieces of information to make decisions: ready time, starting deadline, completion deadline, processing time, resource requirements, priority, and subtask scheduler.
π‘ Why this matters: These parameters allow the scheduler to determine whether a set of tasks can meet their deadlines, which is critical for systems where timing failures can be catastrophic.
Rate Monotonic Scheduling
Rate Monotonic Scheduling (RMS) assigns priorities to tasks on the basis of their periods. The highest-priority task is the one with the shortest period. This is a static priority assignment that is optimal among fixed-priority scheduling algorithms.
Dynamic Memory Allocation
Dynamic memory allocation involves two types of allocators. An explicit memory allocator requires the application to allocate and free space (e.g., malloc and free in C). An implicit memory allocator requires the application to allocate space but does not free it manually (e.g., garbage collection in Java, ML, or Lisp). In both cases, the memory allocator provides an abstraction of memory as a set of blocks and gives out free memory blocks to the application.
π Definition β explicit memory allocator: An allocator where the application both allocates and frees space (e.g., malloc/free in C)
π Definition β implicit memory allocator: An allocator where the application allocates but does not free space (e.g., garbage collection in Java, ML, or Lisp)
Process Memory Image
The process memory image is assumed to be word addressed (each word can hold a pointer). The lecture notes the following constraints for applications and allocators. Applications: can issue arbitrary sequence of allocation and free requests; free requests must correspond to an already allocated block. Allocators: cannot control the number or size of allocated blocks; must respond immediately to all allocation requests (cannot reorder or buffer requests); must allocate blocks from free memory (can only place un-allocated blocks in free memory); must align blocks to satisfy all alignment requirements (e.g., 8 byte alignment for GNU malloc (libc malloc) on Linux machines).
β Key Takeaways
Real-Time Operating Systems must be highly reliable because any performance degradation can lead to catastrophic outcomes, and they must support fail-soft operation to preserve data and capability during failures. Their features include fast process switching, quick interrupt response, priority-based preemptive scheduling, and inter-process communication tools. There are four types of real-time scheduling: static table-driven, static priority-driven preemptive, dynamic planning-based, and dynamic best effort. Rate Monotonic Scheduling assigns highest priority to the task with the shortest period. In dynamic memory allocation, explicit allocators require both allocation and freeing by the application, while implicit allocators handle freeing automatically through garbage collection, and all allocators must meet strict constraints regarding immediate response, alignment, and block management.
π§ Quick Revision Questions
- What are the two key characteristics of Real-Time Operating Systems related to failure and performance?
- Name four specific features of an RTOS that help it meet timing requirements.
- How does static priority-driven preemptive scheduling differ from dynamic planning-based scheduling?
- In Rate Monotonic Scheduling, which task receives the highest priority?
- What is the fundamental difference between an explicit memory allocator and an implicit memory allocator?