π Lecture 23 β Fraction in Binary
π Overview: This lecture introduces fractional binary numbers, explaining how the radix point separates whole and fractional parts, and demonstrates addition operations with binary fractions. Understanding fractional binary is essential for precise numerical computing and data representation in digital systems.
ποΈ Topics Covered
Introduction to the radix point concept in binary systems, examples showing how to read fractional binary numbers, and step-by-step demonstration of addition operations with binary fractions, including worked examples with alignment of radix points.
π Lecture Summary
24. Fraction in Binary
24.1. Radix Point
Just as decimal numbers use a decimal point to separate whole and fractional parts, binary numbers use a radix point. Digits on the left side of the radix point represent the whole number portion, while digits on the right side represent the fractional part.
π Definition β Radix Point: The point that separates the integer part from the fractional part in a binary number, analogous to the decimal point in decimal numbers.
24.2. Example of Radix Point
The example shown in Figure 30 demonstrates how a binary number with a radix point is structured. To correctly read binary fractions, consider Figure 31, which explains that each position to the right of the radix point represents negative powers of 2 (2β»ΒΉ, 2β»Β², 2β»Β³, etc.).
π Definition β Binary Fraction: A binary number that contains a radix point, where digits to the right represent fractional values as negative powers of 2.
π Formula: Position value = 2^(-n) β where n is the position number to the right of the radix point, starting at 1 for the first position
π Example: In the binary number 10.011:
- Left of radix point: "10" = (1 Γ 2ΒΉ) + (0 Γ 2β°) = 2 + 0 = 2 (whole number part)
- Right of radix point: "011" = (0 Γ 2β»ΒΉ) + (1 Γ 2β»Β²) + (1 Γ 2β»Β³) = 0 + 0.25 + 0.125 = 0.375 (fractional part)
- Complete value: 2.375 in decimal
π‘ Why this matters: Understanding binary fractions is critical for accurate representation of non-integer values in computing, from simple measurements to complex scientific calculations.
24.3. Addition in fraction
Binary fraction addition follows the same rules as decimal fraction addition. The key requirement is to align the radix points of both numbers before performing the addition, using the same binary addition rules learned previously for whole numbers.
π Rule β Radix Point Alignment: When adding binary fractions, you must first align the radix points vertically, just as you would align decimal points in decimal addition.
π Formula: Binary addition of fractions = align radix points + apply binary addition rules (0+0=0, 0+1=1, 1+0=1, 1+1=0 carry 1)
π Example: Add 10.011 and 100.11
- Step 1: Align radix points:
010.011 + 100.110 ----------- - Step 2: Add column by column from right to left:
- Rightmost column: 1 + 0 = 1
- Next column: 1 + 1 = 0, carry 1
- Next column: 0 + 1 + carry(1) = 0, carry 1
- Next column: 0 + 0 + carry(1) = 1
- Next column: 1 + 0 = 1
- Leftmost column: 0 + 1 = 1
- Step 3: Place radix point in the same position: Result: 111.001
π‘ Why this matters: Mastering binary fraction addition enables you to perform precise arithmetic on fractional values in digital systems, from financial calculations to engineering computations.
β Key Takeaways
The radix point in binary serves the same purpose as the decimal point in decimal, separating whole numbers from fractional parts. Binary fractional digits represent negative powers of 2, with each position to the right becoming increasingly smaller. When adding binary fractions, the essential step is aligning the radix points before applying standard binary addition rules. The example 10.011 + 100.11 = 111.001 demonstrates this process, where the result includes both whole and fractional components. Understanding binary fractions is foundational for working with real numbers in digital computing and embedded systems.
π§ Quick Revision Questions
- What is the radix point, and how does it function in binary numbers?
- In the binary fraction 10.011, what are the whole number part and fractional part, and what is their decimal equivalent?
- What is the first and most critical step when adding two binary fractions?
- What is the result of adding the binary fractions 10.011 and 100.11?
- What power of 2 does the first position to the right of the radix point represent?
π Lecture 24 β Module 25: 2βs Complement Notation to Store Numbers
π Overview: This lecture introduces twoβs complement notation, the most popular system for representing integers in modern computers. It explains how positive and negative numbers are stored, how to convert between them, how addition works, and the problem of overflow when using a fixed number of bits.
ποΈ Topics Covered
The lecture covers integer representation in 2βs complement using fixed bit patterns, including the role of the sign bit and patterns for three-bit and four-bit examples. It then explains the trick for converting between positive and negative representations, demonstrates addition in 2βs complement notation, and discusses the problem of overflow when results exceed the representable range.
π Lecture Summary
25.1. Integer Representation in 2βs Complement
Twoβs complement notation uses a fixed number of bits to represent each integer value. Normally, 32 bits are used in modern computers, but smaller examples are discussed for demonstration. For positive integers, the representation starts from zero (all zeros) and goes upward until a single zero is followed by all 1βs. For negative integers, the representation starts from all 1βs and goes downward until a single 1 is followed by all 0βs. The leftmost bit functions as the sign bit (0 for positive, 1 for negative). Patterns for three-bit lengths and four-bit lengths are shown in Figures 33 and 34 respectively.
25.2. Conversion between positive and negative representations
To convert between positive and negative numbers in 2βs complement, use this trick:
- Start from the rightmost bit.
- Copy all bits until you encounter the first 1.
- After the first 1 is found, complement all remaining bits (change 0 to 1 and 1 to 0).
π Definition β Twoβs complement negation: To find the negative of a number, copy bits from the right until the first 1, then flip all remaining bits to the left.
π Example: The binary representation of +7 in 2βs complement (4 bits) is 0111. To find -7, start from the right: copy the first 1 (rightmost bit is 1, copy it), then complement the remaining bits: 011 becomes 100. Result: 1001. Thus, -7 in 2βs complement is 1001.
π‘ Why this matters: This trick allows the same hardware to handle both addition and subtraction without special circuits.
25.3. Addition in 2βs complement notation
The advantage of 2βs complement notation is that it works perfectly with the same method of addition used for binary numbers. Standard binary addition can be applied directly, and the result is automatically correct in twoβs complement form. Figure 35 in the lecture demonstrates this with examples.
25.4. Problem of Overflow
Overflow occurs when the result of an addition exceeds the range that can be represented with the given number of bits. Using 4 bits, the maximum positive number is 7 (0111) and the minimum negative number is -8 (1000). For example, 5 + 4 = 9, which cannot be stored in four bits; the result would incorrectly appear as -7 (since 9 exceeds the range). Overflow can be detected by checking the sign bit of the result against the expected sign. Todayβs computers use longer bit patterns (normally 32 bits), giving a maximum positive value of 2,147,483,647. If overflow still occurs, we can use even more bits or change the units (e.g., calculating the answer in kilometers instead of meters).
π Definition β Overflow: When the result of an arithmetic operation falls outside the representable range of a given number of bits. π Example: Using 4-bit twoβs complement: 5 (0101) + 4 (0100) = 9 (1001). The sign bit is 1, indicating a negative result (-7), which is incorrect. The correct sum (9) requires more than 4 bits.
β Key Takeaways
Twoβs complement notation is the standard for representing signed integers in computers because it allows addition to be performed using the same hardware for both positive and negative numbers. The leftmost bit always indicates the sign, and the system has a single representation for zero. Converting between positive and negative values uses the copy-and-complement trick from the rightmost bit. Overflow must be detected when the result exceeds the bit-length limit; modern systems use 32 bits, providing a range from -2,147,483,648 to 2,147,483,647. For exam success, memorize the range for a given bit length and practice the conversion trick and addition with overflow detection.
π§ Quick Revision Questions
- What is the range of integers that can be represented in 4-bit twoβs complement notation?
- Convert +5 (binary 0101) to its twoβs complement negative form (-5) using the copy-and-complement trick.
- What result does 5 + 4 produce in 4-bit twoβs complement, and why is it considered an overflow?
- In twoβs complement notation, what does the leftmost bit represent?
- If 32-bit twoβs complement can store up to 2,147,483,647, what is the smallest negative number it can store?
π Lecture 26 β Excess Notation
π Overview: This lecture introduces Excess Notation as an alternative method for representing integer values in binary. It explains how this notation works, focusing on Excess 8 and Excess 4 systems, and compares them with binary and 2's complement representations.
ποΈ Topics Covered
The lecture covers the fundamental concept of Excess Notation for integer representation, detailing how fixed bit patterns are used to represent zero and both positive and negative numbers. It specifically examines Excess 8 Notation and Excess 4 Notation through figures, and concludes with a comparison of bit patterns across Binary, Excess 4, and 2's complement systems.
π Lecture Summary
26. Excess Notation
This is another method of representing integer values. Unlike standard binary, Excess notation uses a bias or excess value to shift the range of representable numbers. The system uses a fixed number of bits to represent each value, writing down all possible bit patterns of the same length. The first bit pattern with a 1 in the most significant bit is used to represent zero. Following values represent positive numbers, and preceding values represent negative numbers. Each value in Excess notation is the excess of its original value in Binary. For example, 1011 represents 11 in Binary but here it represents 3 (excess of 8). The notation is named after the excess value used, such as Excess 16 to represent 10000 as Zero, or Excess 4 to represent 100 as Zero. Excess 8 notation is shown in Figure 36. π‘ Why this matters: Excess notation allows representing negative numbers without a dedicated sign bit, which is useful in certain computing systems.
π Definition β Excess Notation: A method of representing integer values where a fixed pattern of bits is assigned a value that is offset by a predetermined excess (bias). The bit pattern with a 1 in the most significant bit is assigned the value zero, positive values are patterns following this, and negative values are patterns preceding this.
π Formula: Represented Value = Binary Value of Bit Pattern - Excess Value β The actual integer is obtained by subtracting the excess from the binary value of the bit pattern.
π Example: In Excess 8, the bit pattern 1011 has a binary value of 11. The excess is 8. Therefore, the represented value = 11 - 8 = 3. This shows how the same bit pattern can represent a different integer in Excess notation.
26.1. Integer Representations in Excess Notation
Fixed number of bits to represent each value. Write down all bit patterns of the same length. The first bit pattern with a 1 in the most significant bit is used to represent Zero. Following values are used to represent positive numbers and preceding values to represent negative numbers. Each value in Excess notation is the excess of its original value in Binary. For instance, 1011 represents 11 in Binary but here it represents 3 (excess of 8). Various excess values can be used, such as Excess 16 to represent 10000 as Zero, or Excess 4 to represent 100 as Zero. Excess 8 notation is shown in Figure 36.
Figure 36: Excess eight notation
26.2. Excess Four Notation
Excess 4 is used to represent 100 as Zero. Values are the excess of 4, as shown in Figure 37.
Figure 37: Excess Four notation
π Definition β Excess 4 Notation: A specific case of Excess notation where the excess (bias) value is 4. The bit pattern 100 (binary for 4) represents the value zero.
π Example: With a 3-bit Excess 4 system, the bit pattern 100 represents 0. The pattern 101 represents +1 (5 - 4 = 1), 110 represents +2 (6 - 4 = 2), and 111 represents +3 (7 - 4 = 3). For negative numbers, 011 represents -1 (3 - 4 = -1), 010 represents -2 (2 - 4 = -2), 001 represents -3 (1 - 4 = -3), and 000 represents -4 (0 - 4 = -4).
26.3. Excess Four Notation Comparisons
The lecture compares the differences in bit patterns represented in Binary, Excess 4, and 2's complement. Such a comparison is shown in Figure 38.
Figure 38: Comparison of Binary, Excess 4, and 2's complement
π Comparison point: In standard Binary, the bit pattern 100 represents the value 4. In Excess 4, 100 represents 0. In 2's complement, 100 represents -4 (for a 3-bit system). This highlights how the same sequence of bits can represent completely different integer values depending on the representation system.
β Key Takeaways
Excess Notation is a clever method for representing both positive and negative integers by using a bias or excess value. The critical rule is that the first bit pattern with a 1 in the most significant bit represents zero, with larger patterns being positive and smaller patterns negative. The actual integer value is calculated by subtracting the excess from the binary value of the bit pattern. Different excess values, such as 8 or 4, create different number ranges. This system is meaningfully different from both standard binary and 2's complement, and the same bit pattern can represent vastly different numbers in these systems.
π§ Quick Revision Questions
- How does Excess Notation represent zero?
- If a 4-bit system uses Excess 8, what is the bit pattern for the value 5?
- What is the integer value represented by the 3-bit pattern 100 in Excess 4 notation?
- Compare the representation of the bit pattern 111 in Binary, Excess 4, and 2's complement (all 3-bit systems).
- What is the main advantage of using Excess Notation over signed magnitude representation?
π Lecture 26 β Floating Point Notation
π Overview: This lecture explains how computers store numbers with fractional parts using floating-point notation. It covers the storage of radix points, the division of bits into exponent and mantissa fields, and the process of encoding and decoding values in an 8-bit storage system using excess notation for exponents.
ποΈ Topics Covered
The lecture covers the concept of floating-point notation for storing fractional numbers in binary, including the importance of storing the radix position, the division of bits into sign, exponent, and mantissa fields, the use of excess notation for exponents, and the step-by-step process of decoding and encoding values using an 8-bit example system.
π Lecture Summary
27.1. Storing Radix
Numbers that include a fractional part require storing not only the binary pattern of 0s and 1s but also the position of the radix point (the binary equivalent of a decimal point). A popular method for this is floating-point notation, which is based on scientific notation. For demonstration purposes, the lecture uses an 8-bit storage system.
27.2. Storing Fractions
The 8-bit byte is divided into three fields: the sign bit (high-order bit, bit 7), the exponent field (next 3 bits), and the mantissa field (remaining 4 bits). A sign bit of 0 means the value is nonnegative, and 1 means negative.
Decoding Example 1: The byte 01101011 is analyzed.
- Sign bit = 0 (nonnegative)
- Exponent =
110 - Mantissa =
1011 - First, extract the mantissa and place a radix point on its left side: .1011
- Next, interpret the exponent
110using the 3-bit excess method (excess four notation). The pattern110represents +2 (since 110β = 6, and 6 - 4 = 2). This tells us to move the radix to the right by 2 bits. - Result: 10.11 (binary) = 2 + 0.5 + 0.25 = 2Β³ββ.
- Since the sign bit is 0, the value is +2Β³ββ.
Decoding Example 2: The byte 00111100.
- Sign bit = 0
- Exponent =
011 - Mantissa =
1100 - Place radix on left of mantissa: .1100
- Exponent
011in excess four = 3 - 4 = -1. Move radix left by 1 bit. - Result: .01100 (binary) = 0/2 + 1/4 + 1/8 + 0/16 + 0/32 = 3/8.
- Sign bit 0 means value = +3/8.
Encoding Example: To store the value 1ΒΉββ:
- Convert to binary: 1.001
- Copy the bit pattern into the mantissa field from left to right, starting with the leftmost 1. The mantissa becomes:
- - - - 1 0 0 1 - To fill the exponent, imagine the mantissa with a radix at its left: .1001. To obtain the original binary number
1.001, the radix must move 1 bit to the right. Therefore, the exponent should be +1. - In excess four notation for 3 bits, +1 is represented as
101(since 4 + 1 = 5 = 101β). - Sign bit = 0 (nonnegative).
- Final byte: 0 1 0 1 1 0 0 1.
π Definition β Floating-point notation: A method for storing numbers with fractional parts by representing them as a mantissa and an exponent, similar to scientific notation, with the radix point position determined by the exponent.
π Definition β Excess notation (excess four): A method for representing signed integers where the stored value is the actual integer plus a bias (here, 4). For example, to store +2, you store 2+4 = 6 = 110; to store -1, you store -1+4 = 3 = 011.
π Definition β Mantissa: The field (4 bits in this 8-bit example) that contains the significant digits of the binary number, with an assumed radix point on its left side after extraction.
π Definition β Radix point: The point that separates the integer part from the fractional part in a number (equivalent to a decimal point in base 10, but used in any base; here, binary).
π Formula: Decoding value = (-1)^(sign) * (1.mantissa) * 2^(exponent - bias), but for this simplified 8-bit system, the leading 1 is not stored (it is implied). The actual process: place radix left of mantissa β move radix right (positive exponent) or left (negative exponent) by |exponent| bits β interpret resulting binary number.
π Formula: Storing exponent: exponent_stored = actual_exponent + 4 (the bias)
π Example: Encode -3/8.
- Binary of 3/8 is 0.011
- Starting with leftmost 1, mantissa becomes:
1 1 0 0(the leading 0s are dropped; the bits are1 1 0 0) - For .1100 to become 0.011, the radix must move 1 bit to the left (exponent = -1)
- Excess four: -1 + 4 = 3 =
011 - Sign bit = 1 (negative)
- Final byte: 1 0 1 1 1 1 0 0 (sign=1, exponent=011, mantissa=1100)
π‘ Why this matters: Floating-point notation is the foundation for how all modern computers represent non-integer numbers (IEEE 754 standard). Understanding this simplified 8-bit version builds intuition for how real 32-bit and 64-bit floating-point numbers work.
[27.2 continued β Encoding Subtlety]
There is a subtle point: when filling in the mantissa field, you copy the bit pattern from the binary representation starting with the leftmost 1. For example, to store 3/8 = .011 in binary, the mantissa becomes 1 1 0 0 (not 0 1 1 0). The leading 1 is always the first bit placed into the mantissa field, regardless of whether it is to the left or right of the radix point in the original number.
β Key Takeaways
Floating-point notation stores fractional numbers by dividing bits into sign, exponent, and mantissa fields, with the radix point's position encoded in the exponent using excess notation. The mantissa is extracted by placing a radix on its left, and the exponent (interpreted via the bias) determines how many bits and in which direction to shift that radix. When encoding, the mantissa is filled starting with the leftmost 1 of the binary representation. The sign bit determines whether the final value is positive or negative. This system allows a fixed number of bits to represent a wide range of values by trading precision for range.
π§ Quick Revision Questions
- In the 8-bit floating-point system described, what are the three fields and how many bits does each have?
- Decode the byte
01011011: what value does it represent (show all steps)? - Encode the value -1.5 (binary 1.1) using the 8-bit floating-point system. What is the final bit pattern?
- Why is excess notation used for the exponent field instead of two's complement?
- When encoding 0.011 (binary), why does the mantissa become
1100and not0110?
π Lecture 27 β Truncation Errors in Floating Point Notation
π Overview: This lecture explores the problems that arise when storing real numbers in floating-point notation, specifically truncation (roundoff) errors. It explains why these errors occur, how normalized form reduces ambiguity, and introduces both primitive and intelligent methods for handling such errors in computation.
ποΈ Topics Covered
The lecture begins by reviewing normalized form in floating-point representation, where the mantissa always starts with a leading 1, and explains how this eliminates multiple representations for the same value. It then introduces truncation errors using the example of storing 2β , showing how bits are lost when the mantissa field is too small. The lecture covers primitive methods like using longer mantissa fields and the problem of nonterminating binary expansions. Finally, it presents an intelligent processing strategy where the order of addition can reduce truncation errors.
π Lecture Summary
Normalized Form in Floating-Point Representation
When encoding a value in floating-point notation, we fill the mantissa field starting with the leftmost 1 in the binary representation. Representations that follow this rule are said to be in normalized form. Using normalized form eliminates the possibility of multiple representations for the same value. For example, both bit patterns 00111100 and 01000110 would decode to the value 3β8, but only the first pattern is in normalized form. Complying with normalized form also means that the representation for all nonzero values will have a mantissa that starts with 1. The value zero, however, is a special case; its floating-point representation is a bit pattern of all 0s.
π Definition β Normalized Form: A floating-point representation where the mantissa field is filled starting with the leftmost 1 in the binary representation, ensuring a unique representation for each value. π Rule: For all nonzero values, the mantissa must start with 1. Zero is represented as all 0s. π Example: The value 3β8 in binary is 0.011. Under normalized form, the mantissa starts with the leftmost 1, giving 0.11 Γ 2β»ΒΉ. This produces the bit pattern 00111100. The alternative pattern 01000110 also represents 3β8 but is not in normalized form.
π‘ Why this matters: Without normalized form, the same numeric value could have multiple binary encodings, leading to ambiguity and inconsistency in computational results.
Truncation Errors in Floating Point Notation
Consider the problem of storing the value 2β with a one-byte floating-point system. First, 2β is written in binary as 10.101. However, when copying this into a 4-bit mantissa field, we run out of room, and the rightmost 1 (which represents the last 1β8) is lost. If we ignore this problem and fill in the exponent field and sign bit, we end up with the bit pattern 01101010, which represents 2Β½ instead of 2β . This is called a truncation error, or roundoff errorβmeaning that part of the value being stored is lost because the mantissa field is not large enough.
π Definition β Truncation Error (Roundoff Error): An error that occurs when part of the value being stored is lost because the mantissa field is not large enough to hold the complete binary representation. π Explanation: When converting 2β to binary: 2 = 10, β = 0.101, total = 10.101. The 4-bit mantissa can only hold "1010" (the first 4 bits after the leading 1), dropping the last 1 bit representing β . π Example: Storing 2β produces bit pattern 01101010, which decodes to 2Β½ instead of 2β . The loss of β is the truncation error.
Primitive Methods for Handling Truncation Errors
The significance of such errors can be reduced by using a longer mantissa field. Most computers today use at least 32 bits for storing values in floating-point notation instead of the 8 bits used in this lecture, allowing for a longer exponent field at the same time. Even with these longer formats, however, there are still times when more accuracy is required. Another source of truncation errors is the problem of nonterminating expansions. Some values cannot be accurately expressed regardless of how many digits we use. The difference between base 10 and binary notation is that more values have nonterminating representations in binary than in decimal notation. For example, the value 1/10 is nonterminating when expressed in binary. Imagine the problems this might cause when using floating-point notation to store and manipulate dollars and cents. If the dollar is used as the unit of measure, the value of a dime could not be stored accurately. A solution is to manipulate the data in units of pennies so that all values are integers that can be accurately stored using a method such as two's complement.
π Definition β Nonterminating Expansion: A numeric value that cannot be expressed with a finite number of digits in a given base, leading to unavoidable truncation errors regardless of field size. π Example: The value 1/10 in binary is a repeating fraction (0.0001100110011...), making it impossible to store accurately in floating-point notation. The solution is to use integer representation (pennies) instead of dollar units.
Intelligent Processing for Handling Truncation Errors
Suppose we want to store the result of adding 2Β½ + β + β . If we add 2Β½ to β first, we end up with 2β (10.101 in binary), which cannot be stored in a 4-bit mantissa. The result 10.10 would be 2Β½, meaning the β is truncated. However, a better approach is possible. Let's add the two β values first: β + β = ΒΌ (0.01 in binary), which can be stored. The result would be the bit pattern 00111000. Now add this to 2Β½, and we get 2ΒΎ = 01101011, which is accurate. This demonstrates that order is importantβadding small quantities together first might give a significant quantity to be added to a large quantity.
π Definition β Intelligent Processing: A strategy for reducing truncation errors by reordering arithmetic operations, specifically adding small quantities together first before adding them to larger quantities. π Strategy: When adding multiple values with different magnitudes, add the smallest values first to create a more significant quantity that can be accurately added to the larger value. π Example: Instead of computing (2Β½ + β ) + β (which truncates), compute β + β = ΒΌ (2 bits), then 2Β½ + ΒΌ = 2ΒΎ (accurate in 4-bit mantissa).
π‘ Why this matters: By being aware of the order of operations, programmers can avoid truncation errors without needing more hardware bits, making floating-point arithmetic more accurate in critical applications.
β Key Takeaways
The most critical concepts from this lecture are: normalized form requires the mantissa to start with a leading 1, ensuring each nonzero value has a unique representation and zero is represented by all 0s. Truncation errors occur when the mantissa field is too small to hold the complete binary representation of a value, causing part of the value to be lost. These errors can be reduced by using longer mantissa fields, but nonterminating binary expansions (like 1/10) require alternative approaches, such as using integer representation in smaller units. When adding multiple values, the order of operations matters: adding small quantities together first before adding them to large quantities can prevent truncation. Finally, a one-byte floating-point system with a 4-bit mantissa is too limited for real-world applications, which is why modern computers use 32-bit or longer formats.
π§ Quick Revision Questions
- What is normalized form in floating-point representation, and why is it important?
- How does a truncation error occur when storing the value 2β in an 8-bit floating-point system with a 4-bit mantissa?
- Which values are more likely to have nonterminating expansions in binary compared to decimal notation, and what is a practical solution when dealing with currency?
- When adding 2Β½ + β + β , why does the order of addition matter in a system with limited mantissa bits?
- What bit pattern results from correctly ordering the addition of 2Β½ + β + β , and what value does it represent?
π Lecture 28 β Data Compression: Generic Techniques
π Overview: This lecture introduces the fundamental concepts of data compression, distinguishing between lossless and lossy techniques. It explores four major compression methodsβrun-length encoding, frequency-dependent encoding, relative encoding, and dictionary encodingβalong with their applications and trade-offs, providing a foundational understanding of how data is efficiently stored and transmitted.
ποΈ Topics Covered
The lecture covers two categories of data compression: lossless and lossy. It then details four specific compression techniques: run-length encoding for sequences of repeated values; frequency-dependent encoding (including Huffman codes) for variable-length representations; relative encoding (differential encoding) for data with small changes between consecutive units; and dictionary encoding (including adaptive LZW encoding) for encoding data using references to a dictionary of building blocks.
π Lecture Summary
Lossless vs. Lossy Compression
Data compression schemes fall into two categories. Lossless schemes do not lose information in the compression process. Lossy schemes may lead to the loss of information. Lossy techniques often provide more compression than lossless ones and are popular in settings where minor errors can be tolerated, such as images and audio.
Run-length Encoding
In cases where data consist of long sequences of the same value, the compression technique called run-length encoding is used. This is a lossless method. It is the process of replacing sequences of identical data elements with a code indicating the element that is repeated and the number of times it occurs in the sequence.
π Example: Less space is required to indicate that a bit pattern consists of 253 ones, followed by 118 zeros, followed by 87 ones than to actually list all 458 bits.
Frequency-Dependent Encoding
Another lossless data compression technique is frequency-dependent encoding, a system in which the length of the bit pattern used to represent a data item is inversely related to the frequency of the itemβs use. Such codes are examples of variable-length codes, meaning items are represented by patterns of different lengths.
π Definition β Huffman codes: David Huffman discovered an algorithm commonly used for developing frequency-dependent codes. Most frequency-dependent codes in use today are Huffman codes.
π Example: In English text, the letters e, t, a, and I are used more frequently than letters like z, q, and x. A code for English text can use short bit patterns for frequent letters and longer bit patterns for rare ones, resulting in shorter representations than uniform-length codes.
Relative Encoding
In cases where the stream of data consists of units that differ only slightly from the preceding one, techniques using relative encoding (also known as differential encoding) are helpful. These techniques record the differences between consecutive data units rather than entire units. Each unit is encoded in terms of its relationship to the previous unit.
π‘ Why this matters: Relative encoding can be implemented in either lossless or lossy form, depending on whether differences between consecutive data units are encoded precisely or approximated. Consecutive frames of a motion picture are a typical application.
Dictionary Encoding
Dictionary encoding techniques are popular compression systems. The term dictionary refers to a collection of building blocks from which the message is constructed. The message itself is encoded as a sequence of references to the dictionary.
π Definition β Dictionary encoding: Usually considered a lossless system, but when dictionary entries are only approximations of correct data elements, it results in a lossy compression system.
π Example: Dictionary encoding can be used by word processors to compress text documents. An entire word can be encoded as a single reference to the spell-check dictionary. A typical dictionary with 25,000 entries means an entry can be identified by a pattern of only 15 bits. In contrast, a six-letter word would require 48 bits using UTF-8 character-by-character encoding.
Adaptive Dictionary Encoding (LZW)
A variation of dictionary encoding is adaptive dictionary encoding (also known as dynamic dictionary encoding). In this system, the dictionary is allowed to change during the encoding process.
π Definition β Lempel-Ziv-Welsh (LZW) encoding: A popular example of adaptive dictionary encoding, named after its creators Abraham Lempel, Jacob Ziv, and Terry Welsh.
π Example: To encode a message using LZW, one starts with a dictionary containing basic building blocks (e.g., individual characters, digits, and punctuation marks for English text). As larger units (like words) are found in the message, they are added to the dictionary. Future occurrences of those units can be encoded as single dictionary references. The dictionary grows during encoding, but only the original small dictionary is needed for decoding, because the decoding process encounters the same units and adds them to the dictionary.
π Example: Consider applying LZW encoding to the message "xyx xyx xyx xyx" starting with a dictionary with three entries: the first being x, the second being y, and the third being a space. Encoding "xyx" produces "121", meaning the message starts with the first entry, followed by the second, followed by the first. Then the space is encoded to produce "1213". Having reached a space, the preceding string of characters forms a unit that can be added to the dictionary for future encoding.
β Key Takeaways
Lossless compression preserves all data while lossy compression sacrifices some detail for greater compression. Run-length encoding is effective for sequences of repeated values. Frequency-dependent encoding (like Huffman codes) uses shorter bit patterns for more frequent items. Relative encoding compresses by storing differences between consecutive data units. Dictionary encoding (including adaptive LZW) compresses by referencing building blocks, with the dictionary growing during encoding but only the initial small dictionary needed for decoding.
π§ Quick Revision Questions
- What is the key difference between lossless and lossy compression, and in what situations is each preferred?
- How does run-length encoding work, and what type of data is it most effective for?
- Explain the principle of frequency-dependent encoding and why Huffman codes are a common implementation.
- What is relative encoding and how does it handle consecutive data units that are similar?
- How does adaptive dictionary encoding (LZW) differ from standard dictionary encoding, and what dictionary is needed for decoding?
π Lecture 30 β Data Compression: Compressing Images
π Overview: This lecture explores various compression schemes designed specifically for image representations, focusing on how images can be efficiently stored and transmitted. It covers the fundamental techniques of GIF, JPEG, and TIFF formats, explaining their mechanisms for reducing file sizes while managing the trade-off between compression ratio and image quality loss.
ποΈ Topics Covered
This lecture covers the GIF (Graphic Interchange Format) compression system, which uses a palette of 256 colors and adaptive LZW dictionary encoding; the JPEG standard developed by the Joint Photographic Experts Group, particularly its baseline standard that exploits human eye limitations through chrominance averaging and discrete cosine transform; and the TIFF format, which is primarily used as a standardized storage format rather than a compression method.
π Lecture Summary
Data Compression: Compressing Images
Numerous compression schemes have been developed specifically for image representations. One system known as GIF (short for Graphic Interchange Format and pronounced βGiffβ by some and βJiffβ by others) is a dictionary encoding system that was developed by CompuServe. It approaches the compression problem by reducing the number of colors that can be assigned to a pixel to only 256. The red-green-blue combination for each of these colors is encoded using three bytes, and these 256 encodings are stored in a table (a dictionary) called the palette. Each pixel in an image can then be represented by a single byte whose value indicates which of the 256 palette entries represents the pixelβs color. (Recall that a single byte can contain any one of 256 different bit patterns.) Note that GIF is a lossy compression system when applied to arbitrary images because the colors in the palette may not be identical to the colors in the original image.
GIF can obtain additional compression by extending this simple dictionary system to an adaptive dictionary system using LZW techniques. In particular, as patterns of pixels are encountered during the encoding process, they are added to the dictionary so that future occurrences of these patterns can be encoded more efficiently. Thus, the final dictionary consists of the original palette and a collection of pixel patterns.
One of the colors in a GIF palette is normally assigned the value βtransparent,β which means that the background is allowed to show through each region assigned that βcolor.β This option, combined with the relative simplicity of the GIF system, makes GIF a logical choice in simple animation applications in which multiple images must move around on a computer screen. On the other hand, its ability to encode only 256 colors renders it unsuitable for applications in which higher precision is required, as in the field of photography.
π Definition β GIF (Graphic Interchange Format): A dictionary encoding system that compresses images by reducing the number of assignable colors to 256, storing these color encodings in a palette, and representing each pixel with a single byte.
π Definition β Palette: A table (dictionary) containing 256 encodings of red-green-blue combinations, where each entry is encoded using three bytes.
π Definition β Lossy compression: A compression system where the decompressed data may not be identical to the original data, as occurs with GIF when palette colors differ from original image colors.
π‘ Why this matters: GIF's trade-off of color precision for compression efficiency makes it ideal for simple animations but unsuitable for photography where higher color accuracy is required.
Another popular compression system for images is JPEG (pronounced βJAY-pegβ). It is a standard developed by the Joint Photographic Experts Group (hence the standardβs name) within ISO. JPEG has proved to be an effective standard for compressing color photographs and is widely used in the photography industry, as witnessed by the fact that most digital cameras use JPEG as their default compression technique.
The JPEG standard actually encompasses several methods of image compression, each with its own goals. In those situations that require the utmost in precision, JPEG provides a lossless mode. However, JPEGβs lossless mode does not produce high levels of compression when compared to other JPEG options. Moreover, other JPEG options have proven very successful, meaning that JPEGβs lossless mode is rarely used. Instead, the option known as JPEGβs baseline standard (also known as JPEGβs lossy sequential mode) has become the standard of choice in many applications.
π Definition β JPEG (Joint Photographic Experts Group): A compression standard developed by the Joint Photographic Experts Group within ISO that encompasses several methods of image compression, with the baseline standard being most commonly used.
Image compression using the JPEG baseline standard requires a sequence of steps, some of which are designed to take advantage of a human eyeβs limitations. In particular, the human eye is more sensitive to changes in brightness than to changes in color. So, starting from an image that is encoded in terms of luminance and chrominance components, the first step is to average the chrominance values over two-by-two-pixel squares. This reduces the size of the chrominance information by a factor of four while preserving all the original brightness information. The result is a significant degree of compression without a noticeable loss of image quality.
The next step is to divide the image into eight-by-eight-pixel blocks and to compress the information in each block as a unit. This is done by applying a mathematical technique known as the discrete cosine transform, whose details need not concern us here. The important point is that this transformation converts the original eight-by-eight block into another block whose entries reflect how the pixels in the original block relate to each other rather than the actual pixel values. Within this new block, values below a predetermined threshold are then replaced by zeros, reflecting the fact that the changes represented by these values are too subtle to be detected by the human eye. For example, if the original block contained a checkerboard pattern, the new block might reflect a uniform average color. (A typical eight-by-eight-pixel block would represent a very small square within the image so the human eye would not identify the checkerboard appearance anyway.)
At this point, more traditional run-length encoding, relative encoding, and variable-length encoding techniques are applied to obtain additional compression. Altogether, JPEGβs baseline standard normally compresses color images by a factor of at least 10, and often by as much as 30, without noticeable loss of quality.
π Definition β Luminance and chrominance: Components of image encoding where luminance represents brightness information and chrominance represents color information; JPEG exploits human eye's greater sensitivity to luminance changes.
π Definition β Discrete cosine transform: A mathematical technique that converts an eight-by-eight pixel block into another block reflecting how pixels relate to each other, enabling compression by eliminating values below a threshold.
π Example: A checkerboard pattern in an eight-by-eight-pixel block might be transformed into a uniform average color because the human eye cannot distinguish such fine patterns at that small scale.
π‘ Why this matters: JPEG's baseline standard achieves compression ratios of 10:1 to 30:1 by exploiting human visual limitations and mathematical transformations, making it the dominant format for digital photography.
Still another data compression system associated with images is TIFF (short for Tagged Image File Format). However, the most popular use of TIFF is not as a means of data compression but instead as a standardized format for storing photographs along with related information such as date, time, and camera settings. In this context, the image itself is normally stored as red, green, and blue pixel components without compression.
π Definition β TIFF (Tagged Image File Format): A standardized format for storing photographs along with metadata such as date, time, and camera settings, typically storing uncompressed red, green, and blue pixel components.
β Key Takeaways
The lecture demonstrates that image compression requires balancing file size reduction against quality preservation, with different formats serving different purposes. GIF uses a palette of 256 colors and adaptive LZW dictionary encoding but is lossy for arbitrary images and limited to simple animations. JPEG's baseline standard achieves high compression by exploiting human visual limitations through chrominance averaging and discrete cosine transform, routinely achieving 10:1 to 30:1 compression ratios. TIFF serves primarily as a standardized storage format rather than a compression method, often storing uncompressed image data with metadata. The choice of compression format depends on the application's requirements for color precision, compression ratio, and acceptable quality loss.
π§ Quick Revision Questions
- How does GIF achieve compression and why is it considered lossy when applied to arbitrary images?
- What is the role of the palette in GIF compression and how many colors can it represent?
- How does JPEG's baseline standard exploit the limitations of the human eye in its compression process?
- What is the discrete cosine transform and why is it important in JPEG compression?
- Why is TIFF not typically used as a compression method despite being associated with image storage?