📘 Lecture 40 — Recap of example of PDA corresponding to CFG, CFG corresponding to PDA. Theorem, HERE state, Definition of Conversion form, different situations of PDA to be converted into conversion form
📖 Overview: This lecture revisits the relationship between Context-Free Grammars (CFGs) and Pushdown Automata (PDAs), demonstrating how to convert a CFG into a corresponding PDA. It then introduces a theorem stating that for every PDA, there exists an equivalent CFG, and presents the necessary steps to convert a PDA into a standard "conversion form" by defining the HERE state and modifying the PDA structure to meet six specific conditions.
🗂️ Topics Covered
The lecture covers an example of converting a CFG to CNF and then to a PDA; the theorem that a CFG can be generated from any PDA; the definition and purpose of the HERE state; the six conditions for a PDA to be in conversion form; and a detailed explanation of the different situations that arise when modifying a PDA to satisfy each condition.
📝 Lecture Summary
Example
Consider the following CFG: S → XY, X → aX | bX | a, Y → Ya | Yb | a. First, convert the CFG to be in CNF (Chomsky Normal Form). Introduce the nonterminals A and B as A → a and B → b. The resulting CFG in CNF is: S → XY, X → AX | BX | a, Y → YA | YB | a, A → a, B → b. The PDA corresponding to this CFG will have states including PUSH X, PUSH A, PUSH B, and PUSH Y, directly mirroring the productions.
Theorem
Theorem: Given a PDA that accepts the language L, there exists a CFG that generates exactly L. Before the CFG corresponding to the given PDA is determined, the PDA is converted into a standard form called the conversion form. Before the PDA is converted into conversion form, a new state called HERE is defined, which is placed in the middle of any edge. Like READ and POP states, HERE states are also numbered (e.g., HERE₇).
Conversion form of PDA
Definition: A PDA is in conversion form if it fulfills the following conditions:
- There is only one ACCEPT state.
- There are no REJECT states.
- Every READ or HERE is followed immediately by a POP (i.e., every edge leading out of any READ or HERE state goes directly into a POP state).
- No two POPs exist in a row on the same path without a READ or HERE between them (whether or not there are any intervening PUSH states — the POP states must be separated by READs or HEREs).
- All branching, deterministic or nondeterministic, occurs at READ or HERE states, none at POP states, and every edge has only one label.
- Even before we get to START, a bottom-of-STACK symbol $ is placed on the STACK. If this symbol is ever popped in processing, it must be replaced immediately. The STACK is never popped beneath this symbol. Right before entering ACCEPT, this symbol is popped out and left. The PDA must begin with the sequence PUSH $. The entire input string must be read before the machine can accept the word.
🔑 Definition — Conversion Form: A standardized PDA structure that satisfies six specific conditions (single ACCEPT, no REJECT, READ/HERE followed by POP, no consecutive POPs, branching only at READ/HERE, and a $ bottom-of-stack symbol), which is necessary to construct an equivalent CFG.
Different situations of a PDA to be converted into conversion form are discussed as follows
To satisfy condition 3: Every READ or HERE must be followed immediately by a POP. An edge from READ₃ reading "b" that goes directly to another state must be modified by inserting a POP state between them. 📌 Example: A transition from READ₃ reading "b" becomes READ₃ → POP → (next state).
To satisfy condition 4: No two POPs can appear in a row. If a path has two consecutive POP states, a HERE state must be inserted between them. 📌 Example: POP → POP becomes POP → HERE → POP.
To satisfy condition 5: Branching must occur only at READ or HERE states, not at POP states. A POP state that has multiple outgoing edges must be restructured. 📌 Example: A POP with two outgoing edges becomes POP → READ₃ → (two branches).
To satisfy condition 5 (another case): A POP state that both reads and branches must be separated. 📌 Example: A POP that directly goes to two different states becomes POP → HERE → (two branches).
To satisfy condition 6: The STACK is initially configured with the $ symbol at the bottom. The machine must start with PUSH $, and the $ symbol is popped only right before ACCEPT. 💡 Why this matters: These modifications ensure the PDA is in a predictable, standardized form, which is essential for the theorem that a CFG can be systematically derived from it.
⭐ Key Takeaways
The most critical concept is that CFGs and PDAs are equivalent in power: a CFG can be converted into a PDA (as shown in the example), and every PDA can be converted into an equivalent CFG by first transforming the PDA into conversion form. The conversion form imposes six strict structural rules (single ACCEPT, no REJECT, READ/HERE immediately followed by POP, no consecutive POPs without an intervening READ/HERE, branching only at READ/HERE, and a $ bottom-of-stack marker). The HERE state is introduced as a neutral state to insert between conflicting transitions (like consecutive POPs) to satisfy these conditions. For the exam, you must be able to modify a given PDA diagram to meet each of the six conversion form criteria.
🧠 Quick Revision Questions
- What is the purpose of converting a CFG into CNF before constructing its corresponding PDA?
- State the theorem discussed in this lecture regarding the relationship between PDAs and CFGs.
- What are the six conditions that a PDA must satisfy to be in conversion form?
- What is the role of the "HERE" state in the conversion process?
- Explain how condition 3 (every READ/HERE followed by a POP) modifies an existing PDA transition.
📘 Lecture 41 — Recap of PDA in conversion form, example of PDA in conversion form, joints of the machine, new pictorial representation of PDA in conversion form, summary table, row sequence, row language
📖 Overview: This lecture introduces the conversion form of a Pushdown Automaton (PDA), a standardized representation that simplifies analysis and language generation. It covers the strict conditions a PDA must satisfy to be in conversion form, demonstrates the conversion process with an example, and introduces the concepts of joints, summary tables, row sequences, and row language as tools for describing PDA behavior.
🗂️ Topics Covered
The lecture begins by defining the seven conditions for a PDA to be in conversion form. It then presents a concrete example of a PDA for the language {a²ⁿbⁿ : n = 1,2,3,...} and its conversion to conversion form. The concept of joints (START, READ, HERE, ACCEPT states) is introduced, along with their role in defining path segments. A summary table is constructed from the converted PDA, and the notion of row sequence is used to represent paths. Finally, row language is defined as the set of all joint-consistent and STACK-consistent row sequences.
📝 Lecture Summary
Conversion form of PDA
A PDA is in conversion form if it fulfills the following conditions:
- There is only one ACCEPT state.
- There are no REJECT states.
- Every READ or HERE is followed immediately by a POP — i.e. every edge leading out of any READ or HERE state goes directly into a POP state.
- No two POPs exist in a row on the same path without a READ or HERE between them whether or not there are any intervening PUSH states (i.e. the POP states must be separated by READs or HEREs).
- All branching, deterministic or nondeterministic occurs at READ or HERE states, none at POP states and every edge has only one label.
- Even before we get to START, a "bottom of STACK" symbol $ is placed on the STACK. If this symbol is ever popped in the processing it must be replaced immediately. The STACK is never popped beneath this symbol. Right before entering ACCEPT this symbol is popped out and left.
- The PDA must begin with the sequence START — POP — PUSH $ — READ. The entire input string must be read before the machine can accept the word.
🔑 Definition — Conversion Form of PDA: A standardized form of a PDA that satisfies seven strict conditions to ensure consistent structure for analysis and language generation.
Example
Consider the following PDA accepting the language {a²ⁿbⁿ : n = 1,2,3,...}:
[Pictorial representation shows states POP1, READ2, POP2, POP3 with transitions on a, b, Δ]
Which may be converted to:
[Converted PDA shows START → POP4 → PUSH$ → READ1 with transitions through POP5, POP6, READ2, POP1, POP2, POP3, HERE, and ACCEPT states]
The above PDA accepts exactly the same language.
💡 Why this matters: The conversion process preserves the language while making the PDA structure uniform and easier to analyze mathematically.
Note
It may be noted that any PDA which is in conversion form can be considered to be the collection of path segments, where each path segment is of the following form:
| FROM | TO | READ What | POP Exactly one STACK character | PUSH Any string onto the STACK |
|---|---|---|---|---|
| START | READ | No input | Any character | Any string |
| READ | READ | One letter | Any character | Any string |
| READ | HERE | One letter | Any character | Any string |
| READ | ACCEPT | One letter | Any character | Any string |
| HERE | READ | No input | Any character | Any string |
| HERE | HERE | No input | Any character | Any string |
| HERE | ACCEPT | No input | Any character | Any string |
START, READ, HERE and ACCEPT states are called the joints of the machine. Between two consecutive joints on a path exactly one character is popped and any number of characters can be pushed.
The PDA which is in the conversion form can be supposed to be the set of joints with path segments in between, similar to a TG.
The above entire machine can be described as a list of all joint-to-joint path segments, called summary table.
🔑 Definition — Joint: A state of a PDA in conversion form that serves as a connection point between path segments. Joints are START, READ, HERE, and ACCEPT states.
🔑 Definition — Path Segment: The transition between two consecutive joints where exactly one character is popped from the STACK and any number of characters may be pushed onto the STACK.
🔑 Definition — Summary Table: A table listing all joint-to-joint path segments of a PDA in conversion form, containing FROM, TO, READ, POP, and PUSH columns.
The PDA converted to the conversion form has the following summary table:
| FROM | TO | READ | POP | PUSH | ROW Number |
|---|---|---|---|---|---|
| START | READ1 | Λ | $ | $ | 1 |
| READ1 | READ1 | a | $ | a$ | 2 |
| READ1 | READ1 | a | a | aa | 3 |
| READ1 | HERE | b | a | -- | 4 |
| HERE | READ2 | Λ | a | -- | 5 |
| READ2 | HERE | b | a | -- | 6 |
| READ2 | ACCEPT | Λ | $ | -- | 7 |
Consider the word aaaabb. This word is accepted by the above PDA through the following path:
START → POP4 → PUSH$ → READ1 → POP6 → PUSH$ → PUSH a → READ1 → POP5 → PUSH a → PUSH a → READ1 → POP5 → PUSH a → PUSH a → READ1 → POP5 → PUSH a → PUSH a → READ1 → POP1 → HERE → POP2 → READ2 → POP1 → HERE → POP2 → READ2 → POP3 → ACCEPT
The above path can also be expressed by the following path in terms of sequence of rows:
Row1 – Row2 – Row3 – Row3 – Row3 – Row4 – Row5 – Row6 – Row5 – Row7
It can be observed that the above path is not only joint-to-joint consistent but STACK consistent as well.
It may be noted that in FAs, paths correspond to strings of letters, while in PDAs, paths correspond to strings of rows from the summary table.
🔑 Definition — Row Sequence: A sequence of row numbers from the summary table that represents a path through the PDA.
Note
It may be noted that since the HERE state reads nothing from the TAPE, therefore Λ is kept in the READ what column.
It may also be noted that the summary table contains all the information of the PDA which is in the pictorial representation. Every path through the PDA is a sequence of rows of the summary table. However, not every sequence of rows from the summary table represents a viable path, i.e. every sequence of rows may not be STACK consistent.
It is very important to determine which sequences of rows do correspond to possible paths through the PDA, because the paths are directly related to the language accepted, e.g. Row4 cannot be immediately followed by Row6 because Row4 leaves in HERE, while Row6 begins in Read2. Some information must be kept about the STACK before rows are concatenated.
To represent a path, a sequence of rows must be:
- Joint-consistent (the rows meet up end to end)
- STACK-consistent (when a row pops a character it should be there at the top of the STACK)
The next target is to define row language whose alphabet is Σ = {Row1, Row2, ..., Row7} i.e. the alphabet consists of the letters which are the names of the rows in the summary table.
🔑 Definition — Row Language: The language over the alphabet of row names whose words trace joint-to-joint and STACK-consistent paths through the PDA.
Note
It may be noted that the words of the row language trace joint-to-joint and STACK consistent paths, which shows that all the words of this language begin with Row1 and end in Row7.
Consider the following row sequence: Row5 Row5 Row3 Row6
This is string of 4 letters, but not a word of the row language because:
- It does not represent a path starting from START and ending in ACCEPT state.
- It is not joint consistent.
- It is not STACK consistent.
Before the CFG that generates the language accepted by the given PDA is determined, the CFG that generates the row language is to be determined. For this purpose new nonterminals are to be introduced that contain the information needed to ensure joint and STACK consistency.
It is not needed to maintain any information about what characters are read from the TAPE.
⭐ Key Takeaways
The conversion form of a PDA imposes seven strict conditions that standardize the machine's structure, making it possible to represent all paths as sequences of rows from a summary table. The key insight is that each path segment between joints (START, READ, HERE, ACCEPT) pops exactly one character and pushes any number, and these segments are cataloged in the summary table with row numbers. Not every sequence of rows is valid — a sequence must be both joint-consistent (consecutive rows connect properly) and STACK-consistent (the character popped is at the top of the stack). The row language consists of all valid row sequences that begin with Row1 and end with Row7, directly corresponding to accepted input strings. Understanding row sequences and row language is the critical precursor to constructing a CFG that generates the language accepted by the PDA.
🧠 Quick Revision Questions
- What are the seven conditions a PDA must satisfy to be in conversion form?
- What is a joint in a PDA in conversion form, and what are the four types of joints?
- How is the summary table constructed from a PDA in conversion form, and what information does each row contain?
- Explain the difference between joint consistency and STACK consistency for a sequence of rows.
- Why is the row sequence "Row5 Row5 Row3 Row6" not a valid word of the row language, and what two conditions does it violate?
📘 Lecture 42 — Row Language, Nonterminals Defined from Summary Table, Productions Defined by Rows, Rules for Defining Productions, All Possible Productions of CFG for Row Language of the Example Under Consideration, CFG Corresponding to the Given PDA
📖 Overview: This lecture explains how to construct a Context-Free Grammar (CFG) for the row language derived from a PDA’s summary table. It defines nonterminals, introduces production rules based on STACK behavior, and demonstrates the complete CFG for the PDA accepting the language {a²ⁿ bⁿ}. This bridges the gap between PDA behavior and formal grammar generation.
🗂️ Topics Covered
The lecture covers the concept of row language and the use of Net(X, Y, Z) nonterminals to represent path segments between joints. It defines how to determine true Net sentences by examining summary table rows, then provides three formal rules for generating CFG productions from those rows. Finally, it lists all possible productions for the example PDA and shows a leftmost derivation of a shortest word.
📝 Lecture Summary
Row Language and Nonterminals Defined from Summary Table
The row language has an alphabet Σ = {Row₁, Row₂, ..., Row₇} for the example PDA. Nonterminals are introduced in the form Net(X, Y, Z), where X and Y are joint states and Z is any STACK character. A Net(X, Y, Z) represents a path segment between two joints such that the net STACK effect is the same as popping Z. Not all such sentences are true; each row of the summary table is examined to determine whether the net effect of popping is exactly one letter.
🔑 Definition — Net sentence: A sequence of rows in the summary table where the net effect on the STACK is to pop exactly one character, represented by the nonterminal Net(X, Y, Z).
📌 Example: Consider Row₄ from the PDA for {a²ⁿ bⁿ}:
| FROM Where | TO Where | READ What | POP What | PUSH What | ROW Number |
|---|---|---|---|---|---|
| READ₁ | HERE | b | a | -- | 4 |
The nonterminal corresponding to this row is Net(READ₁, HERE, a), meaning Row₄ itself is a single Net row.
Productions Defined by Rows
If a row pushes multiple characters onto the STACK, it cannot itself be a Net sentence. Instead, such rows can be concatenated with other Net sentences. For example, an arbitrary row:
| FROM Where | TO Where | READ What | POP What | PUSH What | ROW Number |
|---|---|---|---|---|---|
| READ₉ | READ₃ | b | b | abb | 11 |
Row₁₁ is not a Net sentence because it pops one b but pushes two letters (a, b, b), adding net two letters. However, Row₁₁ can be concatenated as:
Row₁₁ Net(READ₃, READ₇, a) Net(READ₇, READ₁, b) Net(READ₁, READ₈, b)
This gives the nonterminal Net(READ₉, READ₈, b), leading to the production:
Net(READ₉, READ₈, b) → Row₁₁ Net(READ₃, READ₇, a) Net(READ₇, READ₁, b) Net(READ₁, READ₈, b)
Three Rules for Defining All Possible Productions of CFG for Row Language
Rule 1: The trip from START to ACCEPT with net effect Net(START, ACCEPT, $) gives the production:
S → Net(START, ACCEPT, $)
Rule 2: For any row that pops a single character and pushes nothing (i.e., — in PUSH column):
| FROM Where | TO Where | READ What | POP What | PUSH What | ROW Number |
|---|---|---|---|---|---|
| X | Y | anything | z | -- | i |
The production is:
Net(X, Y, z) → Rowᵢ
Rule 3: For any row that pushes a string of characters onto the STACK:
| FROM Where | TO Where | READ What | POP What | PUSH What | ROW Number |
|---|---|---|---|---|---|
| READₓ | READᵧ | u | w | m₁m₂...mₙ | i |
Then for any sequence of joint states S₁, S₂, ... Sₙ, the production is:
Net(READₓ, Sₙ, w) → Rowᵢ Net(READᵧ, S₁, m₁) Net(S₁, S₂, m₂) ... Net(Sₙ₋₁, Sₙ, mₙ)
💡 Why this matters: This rule introduces new productions, but not every production necessarily leads to a word in the language.
All Possible Productions of CFG for Row Language of the Example Under Consideration
For the PDA accepting {a²ⁿ bⁿ}, using rules 1, 2, and 3:
Rule 1:
S → Net(START, ACCEPT, $)
Rule 2 (from rows that pop exactly one letter):
Net(READ₁, HERE, a) → Row₄
Net(HERE, READ₂, a) → Row₅
Net(READ₂, HERE, a) → Row₆
Net(READ₂, ACCEPT, $) → Row₇
Rule 3 (from Row₁ — READ₁, pops $, pushes $):
Net(START, READ₁, $) → Row₁ Net(READ₁, READ₁, $)
Net(START, READ₂, $) → Row₁ Net(READ₁, READ₂, $)
Net(START, HERE, $) → Row₁ Net(READ₁, HERE, $)
Net(START, ACCEPT, $) → Row₁ Net(READ₁, ACCEPT, $)
Rule 3 (from Row₂ — pushes a$):
Net(READ₁, READ₁, $) → Row₂ Net(READ₁, READ₁, a) Net(READ₁, READ₁, $)
Net(READ₁, READ₁, $) → Row₂ Net(READ₁, READ₂, a) Net(READ₂, READ₁, $)
Net(READ₁, READ₁, $) → Row₂ Net(READ₁, HERE, a) Net(HERE, READ₁, $)
Net(READ₁, READ₂, $) → Row₂ Net(READ₁, READ₁, a) Net(READ₁, READ₂, $)
Net(READ₁, READ₂, $) → Row₂ Net(READ₁, READ₂, a) Net(READ₂, READ₂, $)
Net(READ₁, READ₂, $) → Row₂ Net(READ₁, HERE, a) Net(HERE, READ₂, $)
Net(READ₁, HERE, $) → Row₂ Net(READ₁, READ₁, a) Net(READ₁, HERE, $)
Net(READ₁, HERE, $) → Row₂ Net(READ₁, READ₂, a) Net(READ₂, HERE, $)
Net(READ₁, HERE, $) → Row₂ Net(READ₁, HERE, a) Net(HERE, HERE, $)
Net(READ₁, ACCEPT, $) → Row₂ Net(READ₁, READ₁, a) Net(READ₁, ACCEPT, $)
Net(READ₁, ACCEPT, $) → Row₂ Net(READ₁, READ₂, a) Net(READ₂, ACCEPT, $)
Net(READ₁, ACCEPT, $) → Row₂ Net(READ₁, HERE, a) Net(HERE, ACCEPT, $)
Rule 3 (from Row₃ — pushes aa):
Net(READ₁, READ₁, a) → Row₃ Net(READ₁, READ₁, a) Net(READ₁, READ₁, a)
Net(READ₁, READ₁, a) → Row₃ Net(READ₁, READ₂, a) Net(READ₂, READ₁, a)
Net(READ₁, READ₁, a) → Row₃ Net(READ₁, HERE, a) Net(HERE, READ₁, a)
Net(READ₁, READ₂, a) → Row₃ Net(READ₁, READ₁, a) Net(READ₁, READ₂, a)
Net(READ₁, READ₂, a) → Row₃ Net(READ₁, READ₂, a) Net(READ₂, READ₂, a)
Net(READ₁, READ₂, a) → Row₃ Net(READ₁, HERE, a) Net(HERE, READ₂, a)
Net(READ₁, HERE, a) → Row₃ Net(READ₁, READ₁, a) Net(READ₁, HERE, a)
Net(READ₁, HERE, a) → Row₃ Net(READ₁, READ₂, a) Net(READ₂, HERE, a)
Net(READ₁, HERE, a) → Row₃ Net(READ₁, HERE, a) Net(HERE, HERE, a)
Net(READ₁, ACCEPT, a) → Row₃ Net(READ₁, READ₁, a) Net(READ₁, ACCEPT, a)
Net(READ₁, ACCEPT, a) → Row₃ Net(READ₁, READ₂, a) Net(READ₂, ACCEPT, a)
Net(READ₁, ACCEPT, a) → Row₃ Net(READ₁, HERE, a) Net(HERE, ACCEPT, a)
CFG Corresponding to the Given PDA
Rule 4 completes the CFG by converting each row into a terminal symbol. Each row of the summary table defines a production of the form Rowᵢ → a, where in Rowᵢ the READ column consists of the letter a.
For the example PDA accepting {a²ⁿ bⁿ}, Rule 4 adds:
Row₁ → Λ
Row₂ → a
Row₃ → a
Row₄ → b
Row₅ → Λ
Row₆ → b
Row₇ → Δ
This means the row language word Row₁ Row₂ Row₃ Row₄ Row₅ Row₇ is converted to:
Λ a a b Λ Δ = a a b = aab
Which corresponds to the PDA word aab (i.e., a² b¹).
Leftmost derivation of the shortest word:
S ⇒ Net(START, ACCEPT, $) ... using 1
⇒ Row₁ Net(READ₁, ACCEPT, $) ... using 9
⇒ Row₁ Row₂ Net(READ₁, READ₂, a) Net(READ₂, ACCEPT, $) ... using 20
⇒ Row₁ Row₂ Row₃ Net(READ₁, HERE, a) Net(READ₂, HERE, a) Net(READ₂, ACCEPT, $) ... using 27
⇒ Row₁ Row₂ Row₃ Row₄ Net(HERE, READ₂, a) Net(READ₂, ACCEPT, $) ... using 2
⇒ Row₁ Row₂ Row₃ Row₄ Row₅ Net(HERE, ACCEPT, $) ... using 3
⇒ Row₁ Row₂ Row₃ Row₄ Row₅ Row₇ ... using 5
This yields the shortest word Row₁ Row₂ Row₃ Row₄ Row₅ Row₇ in the whole row language.
⭐ Key Takeaways
The most critical points to remember: (1) Net(X, Y, Z) nonterminals represent path segments whose net STACK effect is popping exactly one character Z between joints X and Y. (2) Three rules generate all CFG productions: Rule 1 gives the start symbol S → Net(START, ACCEPT, $); Rule 2 handles rows that pop one character and push nothing; Rule 3 handles rows that push strings, creating concatenations of Net nonterminals. (3) Rule 4 converts each Rowᵢ to a terminal letter from the READ column, allowing row language words to generate actual PDA-accepted words. (4) Not every production necessarily leads to a word; the system generates all possibilities, and only joint-consistent and STACK-consistent derivations yield valid words. (5) The complete CFG for the example PDA shows that the shortest word Row₁ Row₂ Row₃ Row₄ Row₅ Row₇ corresponds to the string aab of the original language.
🧠 Quick Revision Questions
- What does Net(READ₁, HERE, a) represent in terms of STACK behavior and joint states?
- Under what condition does a row in the summary table yield a single Net sentence (Rule 2)?
- When applying Rule 3, why must the sequence of joint states S₁, S₂, ... Sₙ be arbitrary?
- The shortest word in the row language for {a²ⁿ bⁿ} is Row₁ Row₂ Row₃ Row₄ Row₅ Row₇. What is the corresponding word in the original language after applying Rule 4?
- Why does the leftmost derivation shown produce exactly the shortest word in the row language? What constraints ensure no shorter derivation exists?
📘 Lecture 43 — Non-Context-Free Languages and Pumping Lemma for CFLs
📖 Overview: This lecture establishes that not all languages are Context-Free, introducing the concept of Non-Context-Free Languages. It presents the Pumping Lemma for Context-Free Languages as a tool to prove that certain languages are not CFLs, along with live and dead productions, self-embedded nonterminals, and detailed examples demonstrating how to apply the lemma.
🗂️ Topics Covered
The lecture covers Non-Context-Free languages, live productions (nonterminal → two nonterminals) and dead productions (nonterminal → terminal), a theorem about finite word generation in CNF grammars, the self-embedded nonterminal concept, the Pumping Lemma for CFLs with its proof and applications, examples showing how to apply the lemma to languages like {aⁿbⁿcⁿ} and {aⁿbᵐaⁿbᵐ}, a comparison with the Pumping Lemma for regular languages, and the second version of the Pumping Lemma for CFLs with a worked example of the EVENA language grammar conversion.
📝 Lecture Summary
Non-Context-Free language
Not all languages are Context-Free. Languages that are not Context-Free are called Non-CFL. To prove this claim, we need to study how words are produced from grammars.
🔑 Definition — Non-CFL: A language that cannot be generated by any Context-Free Grammar.
Live production & Dead production
A live production is a production of the form nonterminal → string of two nonterminals. A dead production is a production of the form nonterminal → terminal. Every CFG in Chomsky Normal Form (CNF) has only these types of productions.
Theorem
If a CFG is in CNF and there is a restriction to use each live production at most once, then only finite many words can be generated. Every time a live production is applied during derivation, it increases the number of nonterminals by one. Applying a dead production decreases nonterminals by one. To generate a word, one more dead production is applied than live productions.
Example: S → XY → aY → aa Here one live production (S→XY) and two dead productions (X→a, Y→a) are used.
📐 Formula: If a CFG in CNF has p live and q dead productions, then all words generated without repeating any live production have at most (p+1) letters.
Theorem
If a CFG is in CNF with p live and q dead productions, and w is a word generated by the CFG having more than 2^p letters, then any derivation tree for w has a nonterminal z which is used twice, where the second z is descended from the first z. The generation tree of word w has more than p rows.
Self-embedded nonterminal
A self-embedded nonterminal occurs when, in a given derivation of a word, a nonterminal ever occurs as a tree descendant of itself.
🔑 Definition — Self-embedded nonterminal: A nonterminal X is self-embedded if X → ... → ...X... appears in the derivation tree, meaning X produces a string that eventually yields another X.
Example with grammar: S → AB, A → BC, C → AB, A → a, B → b For word bbabbb, the derivation tree shows A is self-embedded. The part of tree in the upper triangle is identical to that in the lower triangle, allowing replacement of A by the same sequence of productions.
💡 Why this matters: Self-embedded nonterminals are the key to deriving infinite sets of words and form the basis for the pumping lemma.
Pumping lemma for CFLs
Theorem: If G is any CFG in CNF with p live productions, then every word w of length more than 2^p can be partitioned into five substrings as w = uvxyz, where x is not null string and v and y are not both null string. Then all words of the form uvⁿxyⁿz, n = 1,2,3,... can also be generated by G.
Example: Consider CFG in CNF: S → PQ, Q → QS|b, P → a, and word abab. The derivation tree shows: u = a, v = λ, x = b, y = ab, z = λ Repeating the triangle from the second Q descending from the first Q gives uvvxyyz = aλλbababλ = ababab Thus words of form uvⁿxyⁿz belong to the language.
Note
The pumping lemma is satisfied by all CFLs. Languages that don't hold this pumping lemma cannot be Context-Free — such languages are non-CFLs.
Example: Consider L = {aⁿbⁿcⁿ : n=1,2,3,...}. Let w = a²⁰⁰b²⁰⁰c²⁰⁰ (length > 2^p). No matter what choices are made for u,v,x,y,z, uv²xy²z cannot belong to L because all words in aⁿbⁿcⁿ have:
- Only one substring ab
- Only one substring bc
- No substring ac, ba, ca, or cb
If v or y is not a single letter or λ, then uv²xy²z may contain two or more substrings ab or bc. If v and y are single letters or λ, then one or two of letters a,b,c will be increased while the others remain unchanged. Thus pumping lemma is not satisfied, so L is non-CFL.
Comparison with Pumping Lemma for Regular Languages
The pumping lemma for infinite regular languages decomposes w into three parts (w=xyz) such that xyⁿz belong to L. The pumping lemma for CFLs decomposes w into five parts (w=uvxyz) such that uvⁿxyⁿz belong to L.
The proof for regular languages requires a path long enough to contain a circuit. The proof for CFLs requires a derivation long enough to contain a sequence of productions that can be repeated. The regular language pumping lemma acts on machines (FA), while the CFL pumping lemma acts on algebraic structures (grammars).
Second Version of Pumping Lemma
Theorem: If L is a CFL in CNF with p live productions, then any word w in L of length more than 2^p can be decomposed as w = uvxyz such that:
- length(vxy) ≤ 2^p
- length(x) > 0
- length(v) + length(y) > 0 Then words of the form uvⁿxyⁿz : n=1,2,3,... belong to L.
Example: Consider L = {aⁿbᵐaⁿbᵐ : m,n=1,2,3,...} = {abab, aabaab, abbabb, aabbaabb, ...}. The first version of pumping lemma may be satisfied, but applying the second version: let w be decomposed into uvxyz where length(vxy) < 2^p. This shows v and y can't be single letters separated by clumps of other letters (separator letter longer than vxy), so uvvxyyz is not in L. Thus L is non-CFL.
Example — EVENA language: EVENA = (aa)ⁿ = a²ⁿ = {aa, aaaa, aaaaaa, ...}. Grammar: S → SS|aa. CNF: S → SS|AA, A → a. A PDA is constructed, converted to conversion form, and a summary table with 9 rows is created. From the summary table, 44 productions are defined plus 9 row productions completing the CFG.
📐 Formula: Net(START, ACCEPT, $) → productions defined from summary table rows, where X and Y are corresponding joints.
⭐ Key Takeaways
The most critical points to remember: Non-Context-Free languages exist and cannot be generated by any CFG. The Pumping Lemma for CFLs states that for any CFL G in CNF with p live productions, every word longer than 2^p can be written as uvxyz, and all uvⁿxyⁿz belong to the language. A self-embedded nonterminal (one that appears as a descendant of itself in the derivation tree) is essential for infinite language generation. To prove a language is non-CFL, show it violates the pumping lemma — like aⁿbⁿcⁿ where pumping either creates extra substrings or unbalanced letter counts. The second version of the lemma adds the constraint length(vxy) ≤ 2^p, making it more powerful for proving non-CFL status, as demonstrated with aⁿbᵐaⁿbᵐ.
🧠 Quick Revision Questions
- What is the difference between a live production and a dead production in CNF, and how many of each are needed to generate a word?
- State the Pumping Lemma for CFLs and explain why the word a²⁰⁰b²⁰⁰c²⁰⁰ cannot satisfy it for the language {aⁿbⁿcⁿ}.
- What is a self-embedded nonterminal and why is it crucial to the proof of the Pumping Lemma for CFLs?
- In the second version of the Pumping Lemma for CFLs, what additional constraints are placed on the decomposition w = uvxyz, and why are these constraints important?
- For the language L = {aⁿbᵐaⁿbᵐ}, explain step by step why the second version of the Pumping Lemma shows it is not a CFL.
📘 Lecture 44 — Decidability, Parsing Techniques
📖 Overview: This lecture addresses decidability problems for Context-Free Grammars (CFGs), providing algorithms to determine emptiness, finiteness, and membership. It also introduces parsing techniques, particularly top-down parsing, with detailed examples using the PLUS-TIMES grammar to demonstrate how derivations are constructed.
🗂️ Topics Covered
Decidability problems for CFGs including emptiness (whether a CFG generates any word), uselessness (whether a nonterminal is used in derivation), finiteness (whether a CFL is finite), and membership (whether a given string is generated). Algorithms for each problem are presented with examples. The lecture concludes with parsing techniques, focusing on top-down parsing with the PLUS-TIMES grammar.
📝 Lecture Summary
Decidability
The following are decidable problems with respect to CFG:
- Whether or not the given CFG generates any word? — Problem of emptiness of CFL.
- Whether or not the given CFG generates the finite language? — Problem of finiteness.
- Whether or not the given string w can be generated by the given CFG? — Problem of membership.
Three algorithms show that the answers to these questions are yes.
Algorithm 1 (Emptiness)
If the given CFG contains a production of the form S → Λ, then the corresponding CFL is not empty. If the CFG contains a production of the form S → t, where t is a terminal or string of terminals, then t is a word of the corresponding CFL and the CFL is not empty.
If the CFG contains no such production:
- For each nonterminal N with N → t, pick one production for N (if there are more than one) and replace N by t in the right side of each production wherever it appears.
- Remove all such productions from the CFG.
- Repeat the process until either S is eliminated or no new nonterminal is eliminated.
If S has been eliminated, the CFG generates some words; otherwise not.
📌 Example:
S → AB, A → BSB, B → CC
C → SS
A → a|b
C → b|bb
Step (1): Pick A → a, C → b:
S → aB
A → BSB
A → bb
B → aaS
B → bb
C → SS
Step (2): Pick B → bb and A → bb:
S → abb
A → bbSbb
B → aaS
C → SS
Since S → abb has been obtained, abb is a word in the corresponding CFL.
💡 Why this matters: This algorithm provides a systematic method to determine if a CFG produces any terminal strings at all, which is the first question to ask about any grammar.
Algorithm 2 (Uselessness)
To determine whether a nonterminal X is ever used in the derivation of a word from the given CFG:
- Find all unproductive nonterminals (a nonterminal that cannot produce a string of terminals).
- Eliminate all productions involving unproductive nonterminals.
- Paint all X's blue.
- If any nonterminal is on the left side of a production with any blue nonterminal in the right side, paint that nonterminal blue at all occurrences throughout the grammar.
- Repeat step 4 until no new nonterminal is painted.
- If S is blue, then X is a useful member of the CFG; otherwise not.
🔑 Definition — Useless: A nonterminal is called useless if it cannot be used in a production of some word.
📌 Example: Consider the CFG:
S → Aba | bAZ | b
A → Xb | bZa
B → bAA
X → aZa | aaa
Z → ZAbA
To determine whether X is ever used: First, Z is unproductive. Eliminating productions involving Z:
S → Aba | b
A → Xb
B → bAA
X → aaa
X is blue → A is blue → B and S are blue. Since S is blue, X can be used to generate certain words from the given CFG.
💡 Why this matters: Many CFGs contain nonterminals that appear in productions but can never actually contribute to generating terminal strings. This algorithm identifies and removes such useless symbols.
Algorithm 3 (Finiteness)
To determine whether the given CFG generates a finite language:
- Determine all useless nonterminals and eliminate all productions involving them.
- For each remaining nonterminal, determine whether they are self-embedded.
- Stop if a self-embedded nonterminal is discovered.
To test whether X is self-embedded:
- (a) Change all X's on the left side of productions into a Greek letter Ψ and keep all X's on the right side as such.
- (b) Paint all X's blue.
- (c) If Y is any nonterminal on the left side of a production with X in the right side, paint Y blue.
- (d) Repeat step (c) until no new nonterminal is painted.
- (e) If Ψ is painted, then X is self-embedded; otherwise not.
If any nonterminal left in the grammar after step 1 is self-embedded, then the language generated is infinite; otherwise finite.
📌 Example: Consider the CFG:
S → ABa | bAZ | b
A → Xb | bZa
B → bAA
X → aZa | bA | aaa
Z → ZAbA
Nonterminal Z is useless. Eliminating productions involving Z:
S → ABa | b
A → Xb
B → bAA
X → bA | aaa
Starting with nonterminal X: Replace X on the left side by Ψ:
S → ABa | b
A → Xb
B → bAA
Ψ → bA | aaa
X is blue → A is blue → Ψ is blue. Since A is blue → B is blue → S is blue. Since Ψ is blue, X is self-embedded, and the CFG generates an infinite language.
💡 Why this matters: Self-embedded nonterminals create recursive structures that allow strings of unbounded length, making languages infinite. This is crucial for understanding language properties.
Algorithm 4 (The CYK Algorithm)
This algorithm was invented by John Cocke and later published by Tandao Kasami and Daniel H. Younger.
- Convert the given CFG to CNF (Chomsky Normal Form).
- Let the string x have the form x = x₁x₂x₃...xₙ where all xᵢ's may not be different.
- List all nonterminals in the given CFG (e.g., S, N₁, N₂, ...).
- List nonterminals that generate single-letter substrings of x.
- List nonterminals that generate substrings of length 2.
- Continue listing for substrings of length 3, 4, ..., n.
- If S is among the set of all producing nonterminals for x₁x₂x₃...xₙ, then x can be generated; otherwise not.
📌 Example: Consider the CFG in CNF:
S → AA
A → AA
A → a
Let x = aaa (x₁ = x₂ = x₃ = a).
According to the CYK algorithm:
| Substring | All producing nonterminals |
|---|---|
| x₁ = a | A |
| x₂ = a | A |
| x₃ = a | A |
| x₁x₂ | S, A |
| x₂x₃ | S, A |
| x = x₁x₂x₃ | S, A |
Since S is in the list of producing nonterminals, aaa can be generated by the given CFG.
Parsing Techniques
Recall the CFG for arithmetic expression:
S → S+S | S*S | number
The word 3+4*5 created ambiguity (value either 23 or 35). To remove this ambiguity, the CFG was modified with parentheses. A new CFG can be defined without parentheses using operator hierarchy (* before +). The PLUS-TIMES grammar is:
S → E
E → T+E | T
T → F*T | F
F → (E) | i
Where i stands for any identifier (number or variable name).
📌 Derivation of i+i*i:
S ⇒ E ⇒ T+E ⇒ F+E ⇒ i+E ⇒ i+T ⇒ i+F*T ⇒ i+i*T ⇒ i+i*F ⇒ i+i*i
🔑 Definition — Parsing: The process of finding the derivation of a word generated by a particular grammar is called parsing.
There are different parsing techniques, including:
- Top-down parsing
- Bottom-up parsing
- Parsing technique for particular grammar of arithmetic expression
Top-Down Parsing
In top-down parsing, parsing starts from the nonterminal S and develops a structure similar to the total language tree. The branches of the tree are extended until the required word is found as a branch. Unwanted branches (those that don't lead to the required word) are dropped.
📌 Example (PLUS-TIMES grammar, word i+i*i):
Starting from S, the tree branches:
S
├── E
├── T+E
│ ├── F*T+E → (E)*T+E (branch 1) → drop
│ ├── F+E → (E)+E (branch 3) → drop
│ └── ...
├── T (branch to T without +E)
└── ...
After dropping unwanted branches (1, 3, 5, 7, and 8), the tree shows:
S
└── E
├── T+E
│ ├── F*T+E → i*T+E (branch 2) → drop (first two letters don't match i+i*i)
│ └── F+E → i+E (branch 4)
└── T → F*T → i*T (branch 6) → drop
From branch (4) using leftmost derivation:
i+E → i+T+E (branch 9) → drop (more than five letters)
→ i+T
→ i+F*T+E (branch 10) → drop (contains two +)
→ i+F*T
→ i+(E)*T (branch 13) → drop
→ i+i*T
→ i+i*F*T (continue) → i+i*(E) → drop
→ i+i*F → i+i*i ✓
The tree confirms the required derivation:
S ⇒ E ⇒ T+E ⇒ F+E ⇒ i+E ⇒ i+T ⇒ i+F*T ⇒ i+i*T ⇒ i+i*F ⇒ i+i*i
💡 Why this matters: Top-down parsing provides a systematic tree-based approach to finding derivations, which is fundamental to compiler design and natural language processing.
⭐ Key Takeaways
The key concepts from this lecture are the four decidability algorithms for CFGs: the emptiness algorithm determines if any word is generated by repeatedly replacing nonterminals with terminal strings until S is eliminated; the uselessness algorithm identifies nonterminals that can never contribute to word generation by painting connected nonterminals and checking if S is painted; the finiteness algorithm tests for self-embedded nonterminals (which indicate infinite language generation) by replacing left-side occurrences of a nonterminal with a Greek letter and checking if that letter gets painted through connections; and the CYK membership algorithm works on CNF grammars by building a table of nonterminals that generate all substrings of increasing length. Additionally, top-down parsing begins from the start symbol S and builds a derivation tree downward until the target string is found, dropping branches that cannot lead to the desired word.
🧠 Quick Revision Questions
-
What are the three decidability problems for CFGs mentioned in this lecture?
-
In the emptiness algorithm, what condition indicates that a CFG generates at least one word?
-
How do you determine whether a nonterminal X is useful or useless in a CFG?
-
What does it mean for a nonterminal to be self-embedded, and what does it imply about the language generated?
-
In the CYK algorithm, what condition must be satisfied for the given string to be generated by the CFG?