π Lecture 35 β Exception Handling
π Overview: This lecture introduces exception handling as a technique that separates error-handling code from normal code and provides a consistent mechanism for handling errors. It explores how exceptions increase code complexity by introducing invisible execution paths, and discusses the challenges of writing exception-safe code that guarantees proper behavior when exceptions occur.
ποΈ Topics Covered
The lecture covers exception handling mechanisms (try/catch/throw), how exceptions can cross function boundaries, the impact of exceptions on code complexity (analyzing 23 execution paths in simple code), exception safety guarantees (basic, strong, no-throw), and practical techniques for achieving exception safety including the use of auto_ptr and handling multiple side-effects.
π Lecture Summary
Exception handling
Exception handling is a powerful technique that separates error-handling code from normal code. It also provides a consistent error handling mechanism. The greatest advantage of exception handling is its ability to handle asynchronous errors.
The idea is to raise some error flag every time something goes wrong. There is a system that is always on the lookout for this error flag. The raising of the imaginary error flag is simply called raising or throwing an error. When an error is thrown the overall system responds by catching the error. Surrounding a block of error-sensitive code with exception handling is called trying to execute a block.
One of the most powerful features of exception handling is that an error can be thrown over function boundaries. This allows programmers to put the error handling code in one place, such as the main-function of your program.
π Definition β Exception handling: A technique that separates error-handling code from normal code and provides a consistent mechanism for handling asynchronous errors.
π Concept: try/catch/throw β try block contains error-sensitive code, throw raises an error, catch handles the thrown error
π Example:
try {
// error-sensitive code
throw Exception()
} catch( Exception e )
{
// error handling code
}
Exceptions and code complexity
A number of invisible execution paths can exist in simple code in a language that allows exceptions. The complexity of a program may increase significantly if there are exceptional paths in it.
Consider the following code:
String EvaluateSalaryAnadReturnName( Employee e)
{
if (e.Title() == "CEO" || e.Salary() > 10000)
{
cout << e.First() << " " << e.Last() << " is overpaid" << endl;
}
return e.First() + " " + e.Last();
}
Assumptions:
- Different order of evaluating function parameters are ignored
- Failed destructors are ignored
- Called functions are considered atomic
- To count as different execution paths, an execution path must be made-up of a unique sequence of function calls performed and exited in the same way
Question: How many more execution paths are there? Ans: 23 β There are 3 non-exceptional paths and 20 exceptional paths.
Non-exceptional paths:
- If
e.Title() == "CEO"is true, second part not evaluated, cout performed - If
e.Title() != "CEO"ande.Salary() > 10000, both parts evaluated, cout performed - If
e.Title() != "CEO"ande.Salary() <= 10000, both parts evaluated, cout not performed
Exceptional Code Paths (20 paths):
- Copy constructor for passing
eby value might throw 2-3.e.Title()might throw or its return copy might throw 4-5. String literal conversion to temporary object for==might throw 6-8.operator==()might throw 9-13. Any of the five<<operator calls might throw 14-15. Similar to 2 and 3 fore.First()ande.Last()16-19. Similar to 14-15 and 6-8 for return value construction - Similar to 4 for return value
π Definition β Invisible execution paths: Additional execution paths created by exceptions that are not visible in the normal flow of code
π‘ Why this matters: Simple looking code can have many hidden exceptional paths (23 vs 3 normal paths) β always be exception-aware.
Exception-Safety and Exception Neutrality
Exception-Safety: A function is exception safe if it might throw but does not have any side effects if it does throw, and any objects being used, including temporaries, are exception safe and clean-up their resources when destroyed.
Exception Neutral: A function is said to be exception neutral if it propagates all exceptions to the caller.
Levels of Exception Safety
- Basic Guarantee: Ensures that temporaries are destroyed properly and there are no memory leaks
- Strong Guarantee: Ensures basic guarantee as well as there is full-commit or roll-back
- No-throw Guarantee: Ensures that a function will not throw
Analysis of the original function:
- Basic Guarantee? Yes β the function does not create any objects, so no resource leaks
- Strong Guarantee? No β two distinct side-effects exist: emitting message to cout and returning a name string. If exception occurs after partial message emission or after full message but before return, state is inconsistent
- No-throw Guarantee? No β many operations might throw
Strong Guarantee Attempts
First attempt:
String EvaluateSalaryAnadReturnName( Employee e)
{
String result = e.First() + " " + e.Last();
if (e.Title() == "CEO" || e.Salary() > 10000)
{
String message = result + " is overpaid\n";
cout << message;
}
return result;
}
Problem: When caller assigns theName = evaluateSalarayAndReturnName(someEmployee), if copy constructor or copy assignment fails, the side-effects are completed but the result is irretrievably lost.
Second attempt:
String EvaluateSalaryAnadReturnName( Employee e, String &r)
{
String result = e.First() + " " + e.Last();
if (e.Title() == "CEO" || e.Salary() > 10000)
{
String message = result + " is overpaid\n";
cout << message;
}
r = result;
}
Problem: Assignment to r might still fail, leaving one side-effect completed and other incomplete.
Third attempt (using auto_ptr):
auto_ptr<String> EvaluateSalaryAnadReturnName( Employee e)
{
auto_ptr<String> result = new String(e.First() + " " + e.Last());
if (e.Title() == "CEO" || e.Salary() > 10000)
{
String message = (*result) + " is overpaid\n";
cout << message;
}
return result; // rely on transfer of ownership (can't throw)
}
Solution: Hides all work to construct the second side-effect (the return value) and ensures it can be safely returned using only non-throwing operation after the first side-effect completes. The auto_ptr semantics guarantee that if the caller accepts the returned value, they take ownership; if they ignore it, the allocated string is automatically destroyed with proper clean-up.
Exception Safety and Multiple Side-effects
It is difficult and sometimes impossible to provide strong exception safety when there are two or more side-effects in one function and these side-effects are not related with each other. For example, two output messages (one to cout, one to cerr) cannot be combined.
When such a situation occurs with two or more unrelated side-effects which cannot be combined, the best way to handle it is to break it into two separate functions. That way, the caller would know that these are two separate atomic steps.
β Key Takeaways
Exception handling allows errors to cross function boundaries but significantly increases code complexity β a simple 3-line function can have 23 execution paths (3 normal + 20 exceptional). There are three levels of exception safety: basic (no leaks), strong (commit/roll-back), and no-throw (never throws). Achieving strong exception safety often requires performance trade-offs, like using auto_ptr to manage return value ownership. When a function has multiple unrelated side-effects that cannot be combined, it may be impossible to provide strong exception safety, and splitting into separate functions is the best approach.
π§ Quick Revision Questions
- What is the difference between exception safety and exception neutrality?
- How many total execution paths exist in the
EvaluateSalaryAnadReturnNamefunction and why? - What are the three levels of exception safety guarantees?
- Why does the original
EvaluateSalaryAnadReturnNamefunction fail to meet the strong guarantee? - How does using
auto_ptrhelp achieve strong exception safety in the third attempt?
π Lecture 36 β Software Verification and Validation
π Overview: This lecture introduces the foundational concepts of software testing, including verification and validation, defect analysis, and the limitations of testing. It explains why testing cannot prove the absence of defects and establishes the importance of systematic test case design, setting the stage for black-box and white-box testing techniques covered in subsequent lectures.
ποΈ Topics Covered
The lecture covers the core definitions of verification versus validation, the nature of software defects, and the fundamental objectives of software testing. It then explores the limitations of testing through a detailed code example, the structure of test cases and test data, the parallel lifecycle of testing and development, and the distinct roles of developers and testers. Finally, it introduces black-box and structural (white-box) testing as two fundamental approaches.
π Lecture Summary
11.1 Software Testing
To understand software testing correctly, we must first understand related concepts. Software verification and validation are processes that check a product against its specifications and user expectations. According to Barry Boehm, Verification asks: "Does the product meet system specifications?" and "Have you built the product right?" In contrast, Validation asks: "Does the product meet user expectations?" and "Have you built the right product?" It is possible for software to fulfill its specifications but deviate from user expectations, meaning it is verified but not validated. This can happen when user needs were not captured precisely during requirements engineering.
11.2 Defect
A defect is a variance from a desired product attribute, involving system specifications and user expectations. Anything that causes customer dissatisfaction is a defect. A software defect is a phenomenon where software deviates from its expected behavior, representing non-compliance with written specifications or stakeholder needs. As Kernighan stated: "Death, taxes, and bugs are the only certainties in the life of a programmer." Software and defects cannot be separated, but discovering defects at an appropriate stage improves software quality.
π Definition β Software Testing: The process of examining a software product against its requirements, involving verification of the product against written requirements and conformance of requirements with user needs. It is also the process of executing software on test data and examining its output vis-Γ -vis documented behavior.
Software testing objective: According to Popper (1965), the correct approach is not to verify a theory but to seek to refute it. The goal of testing is to expose latent defects before the software is put to use. A tester tries to break the system to show the presence of a defect, not its absence. Testing cannot show the absence of a defect; it only increases confidence because exhaustive testing is impossible β it requires virtually infinite resources.
π Definition β Successful Test: As Myers (1979) stated, if your task is to find problems, you will look harder than if you think your task is to verify the program has none. A test is successful if it discovers an error, analogous to a doctor's diagnosis. Success depends on the ability to discover a bug, not prove the absence of one.
π Example (adapted from Backhouse): A function compares two strings for equality. A tester devised test cases including "cat"/"dog" (False), ""/"" (True), "hen"/"hen" (True), "hen"/"heN" (False), " "/"" (False), ""/"ball" (False), "cat"/"" (False), "HEN"/"hen" (False), "rat"/"door" (False), and " "/" " (True). The function passed all tests but still had a defect. The code bool isStringsEqual(char a[], char b[]) used strlen comparison and a loop checking a[i] == b[i] but set result = true on each matching character without resetting for mismatches. This caused "cut" and "rat" to return true, which is incorrect.
Testing limitations: To prove a formula is incorrect requires only one counterexample, but millions of examples cannot prove it correct β they only enhance comfort level. You cannot test a program completely because the domain of possible inputs is too large and there are too many possible paths through the program.
11.3 Test Cases and Test Data
Test cases correspond to application functionality and include input/output specifications, a statement of the function under test, steps to perform the function, and expected results. Test data includes inputs devised to test the system.
π Definition β Test Case: A set of steps that should be followed to achieve certain functionality, including input/output specification plus steps and expected results.
11.3 Testing vs. Development
Testing is an intellectually demanding activity with a lifecycle parallel to software development. It demands grip over domain and application functionality, scenario-building capabilities, and destructive instincts to break the system. The functional specification document (FS) is the starting point for both testing and development. The development team performs analysis, design, and coding while the testing team performs analysis for test planning, test cases, and test data generation. System comes into testing after development. Test cases are executed, actual results are compared with expected results, and upon discovering defects, the tester generates a bug report. The development team reproduces the defect, identifies root cause, fixes it, and sends a patch. The testing team verifies the fix.
11.5 The Developer and Tester
Development is a creative activity with the objective to show the program works. Testing is a destructive activity with the objective to show the program does not work. Scenarios missed during development analysis would never be tested correctly because corresponding test cases would be missing or incorrect. If the same person who developed a system tests it, chances of carrying the same misunderstanding into testing are very high. Therefore, an independent testing team is essential.
11.6 Usefulness of Testing
The objective of testing is to discover and fix as many errors as possible before software is shipped to the client. Without testing, clients may demand free fixes or sue for damages. Testers are essential β a good tester has a knack of smelling errors, like auditors.
11.7 Testing and Software Phases
Testing phases include: Unit testing (individual components), Module testing (collection of dependent components), Subsystem testing (collection of modules to find interfacing problems), System testing (system as a whole), Acceptance test (validation against user expectations, usually at client premises), Alpha testing (acceptance testing for customized projects, in-house for products), and Beta testing (field testing with potential customers before general release).
11.8 Black Box Testing
In black box testing, a component or system is treated as a black box and tested for required behavior without concern for how inputs are transformed into outputs. The tester gives inputs through the system's interface and tests the output against expected results.
11.9 Structural Testing (White Box)
In structural or white box testing, we look inside the system and evaluate what it consists of and how it is implemented. We analyze internal structures like design, code structure, and documentation to devise test cases.
Effective testing: The objective is to discover the maximum number of defects with minimum resources before delivery. A good tester carries out thorough analysis to develop a representative set of test cases from a huge set of possibilities.
π Definition β Equivalence Classes: Two tests are considered equivalent if it is believed that if one discovers a defect, the other probably will too, and if one does not discover a defect, the other probably won't either.
Equivalence partitioning guidelines: Organize classes in some order, determine boundary conditions, and do not forget invalid inputs.
π Example: For string matching, equivalence partitions include: Equal strings: Two equal strings of arbitrary length (lower case "cat"/"cat", upper case "CAT"/"CAT", mixed case "Cat"/"Cat", numeric "123"/"123", strings with blanks only " "/" ", numeric and character mixed "Cat1"/"Cat1", strings with special characters "Cat#1"/"Cat#1"), and two NULL strings ""/"". Unequal strings: Two different equal strings of arbitrary length (different length "cat"/"mouse", same length "cat"/"dog"), case sensitivity check ("Cat"/"caT"), and one string empty (first NULL ""/"cat", second NULL "cat"/"").
11.10 Basis Code Structures
There are four basic coding structures: sequence, if statement, case statement, and while loop. Flow graph notation describes flow of data or control. Sequence lumps several sequential instructions in one node. If has a decision node with two branches (true and false). Case has a switch node with multiple branches. While has a loop guard that controls iteration.
π Example: Flow graph for bubble sort has six nodes. Node 1 is the while loop, node 2 is the for loop, node 3 is swapping, nodes 4-6 are ending instructions. Possible paths include: Path1: 1-6, Path2: 1-2-3-4-5-1-6, Path3: 1-2-4-5-1-6, Path4: 1-2-4-2-3-4-5-6-1.
White Box Testing
Three coverage schemes are used: Statement Coverage tests all statements on a path; Branch Coverage tests all possible branches of decision structures; Path Coverage tests all possible paths from input to output.
π Example for sorted = false; while (!sorted) { sorted = true; for (i=0; i < N-1; i++) { if a[i] > a[i+1] { swap(a[i], a[i+1]); sorted = false; } } }:
- Statement coverage: execute with any valid input
- Branch coverage: if condition equals branches 1-2 and 1-3
- Path coverage: same as branch testing for simple if statements
π Example: For a loop for (i = 0; i < N; i++) { if (condition1) // do something else // do something else }:
- N=0: 1 path (1-5)
- N=1: 2 paths (1-2-4-1-5, 1-3-4-1-5)
- N=2: 4 paths
- Generalizing: 2^N paths possible. For N=20, more than 1 million paths.
Cyclomatic complexity is a quantitative measure of logical complexity that defines the number of independent paths in a program's basis set. It provides an upper bound for the number of tests that must be conducted to ensure all statements and branches are executed at least once.
π Formula: V(G) = E - N + 2, where E is the number of edges and N is the number of nodes in the flow graph G.
π Example: For the bubble sort flow graph with 8 edges and 6 nodes: C(G) = 8 - 6 + 2 = 4. Paths to test: Path1: 1-6, Path2: 1-2-3-4-5-1-6, Path3: 1-2-4-5-1-6, Path4: 1-2-4-2-3-4-5-6-1.
π Definition β Infeasible Path: A path through a program that is never traversed for any input data. A good programming practice minimizes infeasible paths to zero.
π Example: Code with if (a == b) c = c-1; if (a != b) c = c+1; has infeasible paths 1-2-3-4-5 and 1-3-5. Modified code with if (a == b) c = c-1; else c = c+1; has no infeasible paths.
β Key Takeaways
The most critical concept is that verification ("built the product right") and validation ("built the right product") are distinct but equally important processes β software can meet specifications yet fail user expectations. Testing is a destructive activity aimed at discovering defects, not proving their absence, and a successful test is one that finds a bug. Exhaustive testing is impossible due to infinite input domains and program paths, so equivalence partitioning helps select representative test cases by dividing the input domain into classes where tests are considered equivalent. Cyclomatic complexity (V(G) = E - N + 2) provides a practical upper bound for the number of test cases needed to achieve statement and branch coverage, while minimizing infeasible paths through proper coding practices improves testability.
π§ Quick Revision Questions
- What is the fundamental difference between verification and validation according to Barry Boehm?
- Why can't testing prove the absence of defects, and what does a "successful test" mean?
- How did the string equality example demonstrate a limitation of testing despite passing all test cases?
- What are equivalence classes, and why is it important to include invalid inputs and boundary conditions when partitioning?
- What does cyclomatic complexity measure, and how does it help determine the number of test cases needed for white-box testing?
π Lecture 40 β Unit Testing
π Overview: This lecture introduces unit testing as a fundamental software testing technique where individual program units are tested in isolation. It covers the principles, benefits, and best practices of unit testing, then extends into defect removal efficiency, defect origination, and the comparison between inspection and testing techniques, emphasizing that testing alone is insufficient for high-quality software.
ποΈ Topics Covered
Unit testing principles and benefits, testing against the contract with a square root example, unit testing tips for project organization, defect removal efficiency showing how combining inspections with testing dramatically improves defect detection rates (from 53% to 99.9%), defect origination points throughout the development lifecycle, the complementary nature of inspection versus testing, inspection pre-conditions and checklists covering various fault classes, and static analyzers as supplementary tools for code verification.
π Lecture Summary
11.11 Unit testing
A software program is made up of units that include procedures, functions, classes, etc. The unit testing process involves the developer in testing of these units in isolation to verify their behavior. Typically, the unit test establishes an artificial environment, invokes routines in the module being tested, then checks results against known values or previous test runs (regression testing). When modules are assembled, the same tests can test the system as a whole.
Software should be tested more like hardware with built-in self testing (each unit tested independently), internal diagnostics (diagnostics defined for program units), and a test harness. The emphasis is on built-in testability from the very beginning.
Unit Testing Principles:
- Developers test their own code units during implementation.
- Normal and boundary inputs against expected results are tested.
- Unit testing is a great way to test an API.
Quantitative Benefits:
- Repeatable: Unit test cases can be repeated to verify no unintended side effects from code modifications.
- Bounded: Narrow focus simplifies finding and fixing defects.
- Cheaper: Find and fix defects early.
Qualitative Benefits:
- Assessment-oriented: Writing unit tests forces dealing with design issues like cohesion and coupling.
- Confidence-building: Know what works at an early stage; easier to change when retesting is easy.
Testing against the contract (Example) When writing unit tests, we write test cases that ensure a given unit honors its contract. This reveals whether the code meets the contract and whether the contract means what we think.
Contract for square root routine:
result = squareRoot(argument);
assert (abs (result * result β argument) < epsilon);
π Definition β Unit Testing: Testing individual program units (procedures, functions, classes) in isolation to verify their behavior against expected results.
π Example β Square Root Contract Testing: The contract tells us what to test:
- Pass in a negative argument and ensure it is rejected.
- Pass in an argument of zero to ensure it is accepted (boundary value).
- Pass in values between zero and the maximum expressible argument and verify that the difference between the square of the result and the original argument is less than some value epsilon.
When designing a module, you should design both its contract and the code to test that contract. By building tests before implementing code, you try out the interface before committing to it.
Unit Testing Tips:
- For small projects, embed unit tests in the module itself.
- For larger projects, keep tests in the package directory or a /test subdirectory.
- Making code accessible provides developers with examples of how to use all functionality and a means to build regression tests.
- Use the main routine with conditional compilation to run unit tests.
11.12 Defect removal efficiency
Defect removal efficiency is the ability to remove defects from an application. Data published after analyzing 1500 projects, where four quality assurance mechanisms were employed, shows:
- Testing alone removes only 53% of defects (worst) to 60% (best), median 53%.
- Testing + Quality Assurance yields up to 65% efficiency.
- Code Inspection + Testing yields up to 75%.
- Design Inspections + Testing yields up to 80%.
- Design Inspections + Quality Assurance + Testing yields up to 95%.
- All four techniques combined yields up to 99.9%.
| Technique Combination | Worst | Median | Best |
|---|---|---|---|
| Testing alone | 30% | 53% | 60% |
| Testing + QA | 50% | 65% | 75% |
| Code Inspection + Testing | 55% | 70% | 80% |
| Design Inspection + Testing | 65% | 80% | 87% |
| Design Insp. + QA + Testing | 75% | 87% | 93% |
| All four techniques | 77% | 97% | 99.9% |
π Definition β Defect Removal Efficiency: The measure of how effectively defects are removed from an application using various quality assurance mechanisms.
Inspection and chaotic zone A chaotic zone forms when defects are not discovered and fixed at the appropriate stage. These defects pile up at testing and maintenance phases, destabilizing the application. Fixing requirement or design defects during testing or maintenance becomes extremely expensive as underlying code must be changed.
π‘ Why this matters: Testing alone does not suffice. Inspection techniques must be combined with testing to increase defect removal effectiveness.
11.13 Defect origination
In inspections, emphasis is on early detection and fixing of defects. Points in the development lifecycle where defects enter include:
- Requirements
- Design
- Coding
- User documentation
- Testing itself (can cause defects due to bad fixes)
- Change requests at maintenance or initial usage time
It is important to identify defects and fix them as near to their point of origination as possible.
π Definition β Defect Origination: The stage in the development lifecycle where a defect is introduced into the program.
Lecture No. 41 β 11.14 Inspection versus Testing
Inspections and testing are complementary, not opposing, verification techniques. Both should be used in the verification and validation process.
Key Differences:
- Inspections can check conformance with a specification but not conformance with the customer's real requirements.
- Inspections cannot check non-functional characteristics like performance, usability.
- Inspections do not require program execution and may be used before implementation.
- Many different defects may be discovered in a single inspection.
- In testing, one defect may mask another, so several executions are required.
- For inspections, checklists are prepared containing defect information. Reuse domain and programming knowledge helps prepare these checklists.
- Inspections involve people examining source representation to discover anomalies and defects.
- Inspections may be applied to any representation of the system (requirements, design, test data, etc.).
Inspection pre-conditions:
- A precise specification must be available.
- Team members must be familiar with organization standards.
- Syntactically correct code must be available.
- Inspectors should prepare a checklist to help during inspection.
Inspection checklists: Checklists of common errors should be developed and used to drive the inspection process. These are programming language dependent. For example, in a language with weak type checking, the checklist can be larger. Examples of language-dependent defects include variable initialization, constant naming, loop termination, and array bounds.
Inspection Checklist Example:
| Fault Class | Inspection Check |
|---|---|
| Exception management faults | Have all possible error conditions been taken into account? |
| Data faults | Are all program variables initialized before their values are used? Have all constants been named? Should the lower bound of arrays be 0, 1, or something else? Should the upper bound of arrays be size or size-1? If character strings are used, is a delimiter explicitly assigned? |
| Control faults | For each conditional statement, is the condition correct? Is each loop certain to terminate? Are compound statements correctly bracketed? In case statements, are all possible cases accounted for? |
| Input/Output faults | Are all input variables used? Are all output variables assigned a value before they are output? |
| Interface faults | Do all function and procedure calls have correct number of parameters? Do formal and actual parameter types match? Are the parameters in right order? If components access shared memory, do they have the same model of shared memory structure? |
| Storage management faults | If a linked structure is modified, have all links been correctly assigned? If dynamic storage is used, has space been allocated correctly? Is space explicitly de-allocated after it is no longer required? |
These inspection checks are outcomes of experience gained from developing or testing similar programs.
11.15 Static analyzers
Static analyzers are software tools for source text processing. They parse the program text and try to discover potentially erroneous conditions, bringing them to the attention of the verification and validation team. These tools are very effective as an aid to inspections but are a supplement to, not a replacement for, inspections.
Checklist for static analysis:
| Fault Class | Static Analysis Checks |
|---|---|
| Data faults | Variable used before initialization; variable declared but never used; variables assigned twice but never used between assignments; possible array bound violations; undeclared variables |
| Control faults | Unreachable code; unconditional branches into loops |
| Input/Output faults | Variable output twice with no intervening assignment |
| Storage Management fault | Unassigned pointers; pointer arithmetic |
π Definition β Static Analyzers: Software tools that parse program text to discover potentially erroneous conditions without executing the program.
β Key Takeaways
Unit testing is essential for verifying individual program units in isolation, and testing against a contract helps ensure code meets its specifications while revealing design issues early. Defect removal efficiency data proves that testing alone (53% median) is insufficient; combining design inspections, code inspections, and quality assurance with testing achieves up to 99.9% defect removal. Inspections and testing are complementary techniques β inspections catch specification and design issues early without program execution, while testing verifies actual runtime behavior including non-functional characteristics. Checklists for inspections and static analyzers provide systematic approaches to finding defects across multiple fault classes including data, control, interface, and storage management issues. The critical lesson is that defects must be detected and fixed as close to their point of origination as possible to avoid entering the chaotic zone where accumulated defects become extremely expensive to resolve.
π§ Quick Revision Questions
- What are the three quantitative benefits of unit testing, and what does each mean?
- Using the square root contract example, what three test cases must be created to verify the contract?
- According to the defect removal efficiency data, what is the median defect removal rate for testing alone versus combining all four techniques?
- What is the chaotic zone in defect management, and why is it problematic?
- List at least four fault classes from the inspection checklist and provide one example inspection check for each.
π Lecture 42 β Debugging
π Overview: This lecture defines software bugs, traces the history of the term "debugging" from Admiral Grace Hopper's moth incident, and explains the critical importance of debugging in software maintenance. It then details the infamous 1990 AT&T outage caused by a simple code error, discusses debugging as both an art and a science, and introduces several major classes of bugs with examples, symptoms, and code snippets.
ποΈ Topics Covered
The lecture begins by defining what a bug is and provides a brief history of debugging from Grace Hopper's moth to modern debuggers. It explains the huge cost of debugging in the maintenance phase and presents a real-world case study of the 1990 AT&T telephone outage caused by a misplaced break statement. The lecture then distinguishes debugging as a scientific process rather than an art, using the "you miss the obvious" phenomenon to explain the need for another pair of eyes. Finally, it introduces major bug classes including memory/resource leaks, logical errors, coding errors, memory over-runs, loop errors, pointer errors, and Boolean bugs.
π Lecture Summary
12.1 Debugging
Nothing in software development is certain except death, taxes, and software bugs. A bug is defined as anything the software does that it is not supposed to do, or alternatively, something the software doesn't do that it is supposed to. Bugs range from program crashes to returning incorrect information to having garbled displays.
π Definition β Bug: A software defect where the software does something it is not supposed to do, or does not do something it is supposed to.
12.2 A Brief History of Debugging
The term "bug" was coined by Admiral Grace Hopper while working on the Mark II computer at Harvard University. The first actual "bug" was a moth that flew into a relay, shorting out two contacts and shutting down the system. The moth was removed and pasted into the project logbook. From that point on, her team called their troubleshooting efforts "debugging."
Early debugging efforts centered around data dumps or output devices like printers and display lights. The next major evolution came with command-line debuggers, which represented the first real attempt to turn debugging from a hit-or-miss proposition into a reproducible process.
π‘ Why this matters: Understanding the history shows that debugging has evolved from a haphazard activity into a structured process that is essential for modern software development.
Importance of Debugging
During the maintenance phase, 20% of the lifecycle cost is attributed to defects found after installation. Since maintenance accounts for 2/3rd of the overall software cost, this 20% represents a huge financial burden. System downtime puts tremendous pressure on developers, and every second of outage costs huge losses to the organization.
12.4 Problem at AT&T
On January 15, 1990, AT&T had a US-wide telephone system outage lasting nine hours. The cause was a program error in software meant to make the system more efficient. The code snippet that caused the outage is illustrated below:
do {
switch (expression) {
case 0: {
if (some_condition) {
// ...
break; // Line 7 - the culprit
} else {
// ...
}
// ...
break;
}
}
} while (some_other_condition);
π Definition β The Culprit: The break statement at line 7 caused the program to exit the entire switch block, when the programmer intended it to only break the if-then clause and continue execution at line 11.
π Lesson: Code inspection might have caught this bug only if another engineer saw this line of code and questioned the original programmer's intention.
12.5 Art and Science of Debugging
Debugging is taken as an art but in fact it is a scientific process. As people learn about different defect types, they develop heuristics β shortcuts that help solve similar problems faster next time.
The lecture introduces the phenomenon of "you miss the obvious." When a person writes code, they develop a personal bias toward their creation. When checking this code, they can potentially miss obvious mistakes due to this bias. Therefore, it is strongly recommended to use "another pair of eyes" β ask a companion to help discover the defect.
Example β Bulletin Board Code
while (i = 0; i < 10; i++) {
cout << i << newl;
}
The problem is obvious once pointed out: the loop uses while syntax but the syntax inside is actually a for loop.
Lecture No. 43 β Bug Classes
12.6 Bug Classes
Memory and Resource Leak
A memory leak occurs when memory is allocated from either the operating system or an internal memory pool, but never deallocated when the memory is finished being used.
Symptoms: System slowdowns, crashes that occur "randomly" over a long period of time.
Example 1:
char *buffer = new char[kMaxBufferSize+1];
memset(buffer, 0, kMaxBufferSize+1);
if (IsError(nCondition)) {
return FailureCode; // Memory not freed here!
}
delete buffer;
return okCode;
If no error occurs, memory is freed properly. But if an error occurs, the memory is not freed β a leak occurs.
Example 2:
class Foo {
char *sString;
public:
Foo() { sString = new char[21]; }
~Foo() { delete sString; }
void SetString(const char *inString) {
sString = new char[strlen(inString+1)]; // Old memory lost
if(inString == NULL) return;
strncpy(sString, inString, strlen(inString));
}
};
π Issue: The SetString method overwrites the previously allocated string without freeing it, causing an instant memory leak.
Logical Errors
A logical error occurs when the code is syntactically correct but does not do what you expect it to do.
Symptoms: Code misbehaves in an unexplainable way, program flow takes odd branches, results are opposite of expected, output looks strange.
Example:
if((input >= 1 && input <= 10) && (input >= 15 && input <= 20)) {
// Valid case
} else {
// Invalid case
}
π Problem: A number cannot simultaneously be between 1-10 AND between 15-20. This is a typical logical error using && when || was intended.
π Formula for logical errors: When validating ranges that have gaps, always use logical OR (||) for disjoint ranges, not AND (&&).
Coding Errors
A coding error is a simple problem in writing the code β failure to check error returns, failure to check for valid conditions, or failure to account for other parts of the system.
Symptoms: Unexpected errors in black box testing, compiler warnings, lack of attention to details.
Example:
void convertToString(int InInteger, char* OutString, int* OutLength) {
switch(InInteger) {
case 1: OutString = "One"; OutLength = 3; break;
case 2: OutString = "Two"; OutLength = 3; break;
// ... cases 3-9
}
// No default case β zero value crashes
}
π Issues: Does not handle all cases (e.g., zero value), does not initialize output variables.
Memory Over-runs
A memory overrun occurs when you use memory that does not belong to you β overstepping an array boundary or copying a string too big for its allocated block.
Symptoms: Program crashes regularly after a given routine, or a prior routine has already trashed variables.
Example:
const kMaxEntries = 50;
int gArray[kMaxEntries];
int ZeroArray(int *pArray) {
for(int i=0; i<100; ++i) // Writes past 50-element array
pArray[i] = 0;
}
π Problem: The loop goes past the array's 50-element boundary, corrupting memory beyond its control.
Loop Errors
Loop errors break down into several subtypes: infinite loops, off-by-one loops, and improperly exited loops.
Symptoms: Program locks up (infinite loop), incorrect calculations by the last data point (off-by-one), process terminates unexpectedly.
Example 1 β Infinite Loop:
bool doneFlag = false;
while(!doneFlag) {
if(impossibleCondition)
doneFlag = true; // Never becomes true
}
Example 2 β Off-by-One:
int anArray[50];
int i = 50;
while(i >= 0) {
anArray[i] = 0; // Accesses index 50 β out of bounds
i = i - 1;
}
The loop performs 51 times but the array only has 50 elements.
Example 3 β Improper Exit Condition:
for(int i=0; i<kMaxIterations; ++i) {
while(nIndex < 20) {
ComputeSomething(i*20 + nIndex);
nIndex ++;
}
}
First iteration works correctly, but nIndex becomes 21 and the inner loop never executes for subsequent outer loop iterations.
Pointer Errors
A pointer error occurs when something is being used as an indirect pointer to another item β uninitialized pointers, deleted pointers still being used, or invalid pointers.
Symptoms: Program crashes or behaves unpredictably, stack corruptions, memory allocation failures, odd changing of variable values.
Example:
void cleanup_function(char *ptr) {
SaveToDisk(ptr);
delete ptr; // First delete
}
int func() {
char *s = new char[80];
cleanup_function(s);
delete s; // Second delete β double deletion!
}
π Problem: The pointer is deleted twice β once in cleanup_function and once in func(). At best, the delete recognizes this; at worst, memory is corrupted.
Boolean Bugs
Boolean bugs occur because the mathematical precision of Boolean algebra has virtually nothing to do with equivalent English words. When we say "and," we really mean the boolean "or" and vice versa.
Symptoms: Program does exactly the opposite of what you expect.
Example:
int DoSomeAction(int InputNum) {
if(InputNum < 1 || InputNum > 10) return 0; // Failure
else {
PerformTheAction(InputNum);
return NumberOfAction + 1; // Returns nonzero for success
}
}
Ret = DoSomeAction(11); // Error case
if(Ret) // Ret is 0 (failure), but condition checks for nonzero
cout << "Error in DoSomeAction";
π Problem: The function returns 0 for failure and nonzero for success. This is counter-intuitive β the error message is never triggered because the condition if(Ret) expects nonzero for failure.
Lecture No. 44 β Holistic Approach & Debugging Process
12.7 Holistic Approach
Holistic means emphasizing the importance of the whole and the interdependence of its parts β concerned with wholes rather than analysis into parts.
In debugging, a holistic approach focuses on the entire system rather than whatever piece appears broken. You cannot treat symptoms; you must focus on the application system as a whole.
Example: A program crashes on mX = X; in a simple setter function void SetX(int X) { mX = X; }. Inserting a message statement makes the problem "go away" β but the problem is not fixed. This is treating symptoms rather than finding the root cause.
12.8 The Debugging Process
In normal circumstances, you will have a user description of the problem. This data must be considered suspiciously until you can get a first-hand description.
Three user accounts of the same bug:
- "I selected Favorites from the menu, scrolled to third entry, pressed Enter, then clicked fourth entry on submenu, entered my name, clicked OK... crashed."
- "I selected New menu option, clicked Create Object, entered HouseObject, clicked OK... crashed."
- "I selected New Project menu option, clicked Gear icon, entered Rudolph, clicked OK... crashed."
Analysis process:
- Each user clicked the OK button to finalize the process β likely OK handler contains a fatal flaw
- Each user worked with a menu β examine menu handler for mouse vs keyboard issues
- Each user used both keyboard and mouse β look for combination handling bugs
Good Clues, Easy Bugs β Get a Stack Trace:
- Source line numbers in stack trace are the single most useful piece of debugging information
- Check argument values for improbability (zero, very large, negative, non-alphabetic character strings)
- Use debuggers to display local or global variables
Non-reproducible Bugs:
- Bugs that won't "stand still" (almost random) are the most difficult
- Randomness itself is information
- Check if all variables are initialized
- If bug disappears when debugging code is inserted, suspect memory allocation (
malloc) problems - Check for dangling pointers
Example β Dangling Pointer:
char *msg(int n, char *s) {
char buf[100];
sprintf(buf, "error %d: %s\n", n, s);
return buf; // Returns pointer to local array
}
p = msg(20, "Output values");
q = msg(30, "Input values");
printf("%s\n", p); // p points to invalid memory (stack frame destroyed)
π Problem: Returns a pointer to a local array buf which is destroyed when the function returns.
β Key Takeaways
Debugging is not merely an art but a scientific process requiring methodical investigation of code. The most critical concept is understanding that you will miss the obvious in your own code due to personal bias β always use "another pair of eyes." The AT&T 1990 outage teaches us that even a single misplaced break statement in a switch block can cause millions in losses. For the exam, memorize the seven bug classes (memory/resource leaks, logical errors, coding errors, memory over-runs, loop errors, pointer errors, and Boolean bugs) and their specific symptoms and examples. Finally, adopt a holistic approach β never treat symptoms alone; always investigate the entire system to find the root cause.
π§ Quick Revision Questions
- What was the first actual "bug" in computing history, and who coined the term "debugging"?
- Describe the code error that caused the 1990 AT&T nine-hour outage. What was the programmer's intention versus what actually happened?
- List three symptoms of a memory leak bug and provide a code example where memory is not deallocated in all paths.
- What is the difference between a logical error and a coding error? Give one code example of each.
- What makes a "non-reproducible bug" particularly difficult to debug, and what two things should you check first when you encounter one?