📘 Lecture 15 — Examples of Kleene’s theorem part III (method 3), NFA, examples, avoiding loop using NFA, example, converting FA to NFA, examples, applying an NFA on an example of maze
📖 Overview: This lecture continues the discussion of Kleene’s theorem Part III (method 3) and introduces the Nondeterministic Finite Automaton (NFA) as an intermediate structure between FA and TG. It demonstrates how NFAs can eliminate loops from FAs, how to convert an FA to an equivalent NFA, and presents a practical application of NFA in artificial intelligence using a maze example.
🗂️ Topics Covered
The lecture covers observations about initial states in Kleene’s theorem, the formal definition of NFA with observations, examples of NFAs accepting specific languages, the technique of eliminating loops at states by converting FA to NFA, examples of converting FA to equivalent NFA, and an application of NFA in solving a maze problem in artificial intelligence.
📝 Lecture Summary
Note (Observation about Initial State in Kleene’s Theorem)
It is to be noted that as observed in the examples discussed in previous lecture, if at the initial state of the given FA, there is either a loop or an incoming transition edge, the initial state corresponds to the final state and a non-final state as well, of the required FA. Otherwise, the initial state of given FA will only correspond to a single state of the required FA (i.e. the initial state which is final as well).
Nondeterministic Finite Automaton (NFA) — Definition
An NFA is a TG with a unique start state and a property of having single letter as label of transitions. An NFA is a collection of three things:
- Finite many states with one initial and some final states
- Finite set of input letters, say, Σ = {a, b, c}
- Finite set of transitions, showing where to move if a letter is input at certain state (Λ is not a valid transition), there may be more than one transition for certain letters and there may not be any transition for certain letters.
Observations
It may be observed, from the definition of NFA, that the string is supposed to be accepted, if there exists at least one successful path, otherwise rejected. It is to be noted that an NFA can be considered to be an intermediate structure between FA and TG.
Example (NFA accepting a and ab)
An NFA is presented (in the lecture) that accepts the language consisting of a and ab.
Example (NFA accepting strings containing aa)
An NFA is presented that accepts the language of strings, defined over Σ = {a, b}, containing aa.
Note (Eliminating Loops Using NFA)
It is to be noted that NFA helps to eliminate a loop at certain state of an FA. This process is done by converting the loop into a circuit. But during this process the FA remains no longer FA and is converted to a corresponding NFA.
Example (Eliminating a Loop at State 7)
Consider a part of an FA with alphabet Σ = {a,b,c,d} that has a loop at state 7. To eliminate the loop at state 7, the corresponding NFA may be constructed, where the loop is replaced by a circuit structure, introducing nondeterminism.
Converting an FA to an equivalent NFA
It is to be noted that according to Kleene’s theorem, if a language can be accepted by an FA, then there exists a TG accepting that language. Since an NFA is a TG as well, therefore there exists an NFA accepting the language accepted by the given FA. In this case these FA and NFA are said to be equivalent to each other.
Example (Converting an FA for (a+b)*b to NFA)
Consider the following FA corresponding to *(a+b)b:
- (Diagram of FA with states, transitions for a and b, and a final state)
The above FA may be equivalent to the following NFA:
- (Diagram of NFA with nondeterministic transitions)
Question: Can the structure of above NFA be compared with the corresponding RE?
Example (Converting another FA to NFA)
Consider the following FA:
- (Diagram of another FA)
The above FA may be equivalent to the following NFA:
- (Diagram of corresponding NFA)
Question: Can the structure of above NFA be compared with the corresponding RE?
Application of an NFA — Maze Example
There is an important application of an NFA in artificial intelligence, which is discussed in the following example of a maze:
A maze is presented with labeled positions:
- 1, 2, 3 (top row)
- 4, L, 5, O (second row)
- 6, M, 7, P (third row)
- 8, N, 9 (bottom row)
- - at start position 1
- + at an exit/goal position
An NFA can model the possible paths through the maze, where each state represents a location and transitions represent allowable moves. The NFA accepts if there exists at least one successful path from the start (-) to the goal (+).
💡 Why this matters: This demonstrates how NFAs, with their nondeterministic nature, can efficiently model search problems and pathfinding in AI, where multiple possibilities must be explored simultaneously.
⭐ Key Takeaways
The critical points to remember are: (1) NFA is a TG with single-letter transitions and a unique start state, allowing multiple or zero transitions for a given letter; (2) a string is accepted by an NFA if there exists at least one successful path from start to final state; (3) NFAs serve as intermediate structures between FA and TG and are useful for eliminating loops from FAs; (4) every FA can be converted to an equivalent NFA by replacing loops with circuits, introducing nondeterminism; (5) NFAs have practical applications in artificial intelligence, such as modeling mazes and search problems where multiple paths must be explored.
🧠 Quick Revision Questions
- What are the three components that define an NFA?
- How does the acceptance condition of an NFA differ from that of a deterministic FA?
- Why can NFA be considered an intermediate structure between FA and TG?
- Explain the process of eliminating a loop at a state in an FA by converting it to an NFA.
- In the maze application, why is an NFA particularly suitable for modeling pathfinding problems compared to a deterministic FA?
📘 Lecture 16 — Applying an NFA on an example of maze, NFA with null string, examples, RE corresponding to NFA with null string (task), converting NFA to FA (method 1,2,3) examples
📖 Overview: This lecture demonstrates the practical application of an NFA to solve a maze problem and formally introduces NFA with null string (NFA-Λ). It then details three distinct methods for converting any NFA into an equivalent finite automaton (FA) , providing step-by-step examples for each approach. Understanding these conversions is essential for simplifying automata and building deterministic models.
🗂️ Topics Covered
The lecture covers applying an NFA to model paths in a maze, defines NFA-Λ with examples, discusses the relationships between FA, NFA, NFA-Λ, and TG, and presents three methods for converting NFA to FA: using regular expressions (Method 1), building a transition diagram with an empty state (Method 2), and constructing a transition table with combined states (Method 3).
📝 Lecture Summary
Application of an NFA
An important application of NFA in artificial intelligence is demonstrated using a maze of labeled boxes (1-9, L, M, N, O, P, +, -). The initial state is - and the final state is +. Movement is allowed only between boxes not labeled L, M, N, O, P. An NFA using only the single letter a is constructed to model all possible paths. In this NFA, each acceptable move corresponds to reading an 'a'.
It is observed that the shortest path from initial to final state consists of six steps, meaning the shortest accepted string is aaaaaa. The next longer accepted string is aaaaaaaa (eight a's). If this NFA is treated as a TG, the corresponding regular expression is aaaaaa(aa)*, which shows there are infinitely many required ways (all strings of a's with an even number of a's after the first six).
🔑 Definition — NFA (Nondeterministic Finite Automaton): An automaton where, for a given state and input symbol, there may be zero, one, or multiple possible next states.
💡 Why this matters: This maze example shows how NFAs can model computational problems with multiple possible paths, a core idea in AI pathfinding and state-space search.
NFA with Null String
Definition: If in an NFA, Λ (null string) is allowed to be a label of an edge, then the NFA is called NFA with Λ (NFA-Λ).
An NFA-Λ is a collection of three things:
- Finite many states with one initial and some final states.
- Finite set of input letters, say Σ = {a, b, c}.
- Finite set of transitions, showing where to move if a letter is input at certain state. There may be more than one transition for a certain letter and there may not be any transition for a certain letter. The transition of Λ is also allowed at any state.
📌 Example 1: An NFA-Λ is presented that accepts the language of strings, defined over Σ = {a, b}, ending in b.
📌 Example 2: An NFA-Λ is presented that accepts the language of strings, defined over Σ = {a, b}, ending in a.
Note: Every FA may be considered to be an NFA-Λ as well, but the converse may not be true. Similarly, every NFA-Λ may be considered to be a TG as well, but the converse may not be true.
🔑 Definition — NFA-Λ (NFA with Null String): An NFA where transitions labeled with the null string (Λ) are permitted, allowing state changes without consuming any input symbol.
NFA to FA — Method 1
Method 1: Since an NFA can be considered to be a TG as well, a regular expression (RE) corresponding to the given NFA can be determined (using Kleene’s theorem). Again using the methods discussed in the proof of Kleene’s theorem, an FA can be built corresponding to that RE. Hence, for a given NFA, an FA can be built equivalent to the NFA. Examples have, indirectly, been discussed earlier.
NFA to FA — Method 2
Method 2: Since in an NFA, there may be more than one transition for a certain letter and there may not be any transition for a certain letter, starting from the initial state corresponding to the initial state of the given NFA, the transition diagram of the corresponding FA can be built by:
- Introducing an empty state (Ø) for a letter having no transition at a certain state.
- Introducing a state corresponding to the combination of states (e.g., {1,2}) for a letter having more than one transitions.
📌 Example 1: Consider a given NFA. Using Method 2, the above NFA is converted to an equivalent FA. The FA includes an empty state to handle missing transitions.
📌 Example 2: A simple NFA that accepts the language of strings defined over Σ = {a,b}, consisting of bb and bbb. The NFA is converted to the following FA, which includes an empty state to handle transitions for 'a' and 'b' that are not defined in the NFA.
NFA to FA — Method 3
Method 3: As discussed earlier, in an NFA, there may be more than one transition for a certain letter and there may not be any transition for certain letter. Starting from the initial state corresponding to the initial state of the given NFA, the transition table along with new labels of states of the corresponding FA can be built by:
- Introducing an empty state for a letter having no transition at a certain state.
- Introducing a state corresponding to the combination of states for a letter having more than one transitions.
Further examples of this method are discussed in the next lecture.
⭐ Key Takeaways
The key takeaway is that NFAs are not just theoretical constructs but have practical applications, such as modeling pathfinding problems in AI using a regular expression like aaaaaa(aa)*. The lecture formally defines an NFA-Λ, which adds null transitions, and places it in the hierarchy: FA ⊆ NFA ⊆ NFA-Λ ⊆ TG. Most critically, three distinct methods for converting any NFA (with or without null) into an equivalent FA are presented: Method 1 converts to a TG then to an RE to build an FA; Method 2 builds a transition diagram directly using a new empty state for missing transitions and combined states for multiple transitions; Method 3 builds a transition table similarly, creating new state labels from combinations. These methods are essential for obtaining deterministic models from nondeterministic specifications.
🧠 Quick Revision Questions
- What regular expression describes the language of paths in the maze example, and what does it imply about the number of possible paths?
- What is the formal definition of an NFA with null string (NFA-Λ), and how does it differ from a standard NFA?
- What is the process for converting an NFA to an FA using Method 1?
- Describe the two key strategies used in Method 2 to handle missing transitions and multiple transitions in an NFA when building the equivalent FA.
- What is the fundamental difference between Method 2 and Method 3 for converting an NFA to an FA?
📘 Lecture 17 — Converting NFA to FA (Method 3), NFA and Kleene’s Theorem
📖 Overview: This lecture covers Method 3 for converting an NFA to an FA by introducing an empty state and combination states. It then explores how NFAs help prove Kleene’s Theorem Part III, demonstrating two methods for constructing FAs corresponding to regular expressions through decomposition, union, concatenation, and closure.
🗂️ Topics Covered
Method 3 for NFA-to-FA conversion with a worked example, transition table construction, and transition diagram; NFA and Kleene’s theorem (Part III) with two methods: Method 1 builds FAs for simple languages {a}, {b}, {Λ}, then Method 2 constructs NFAs for union, concatenation, and closure of FAs—illustrated by an NFA corresponding to the union of two FAs.
📝 Lecture Summary
Method 3: Converting NFA to FA
In an NFA, there may be more than one transition for a certain letter or none at all. To convert it to an FA, starting from the initial state of the NFA, build a transition table with new state labels by introducing an empty state (for letters with no transition) and a combination state (for letters with multiple transitions).
Example: Consider the following NFA that accepts the language of strings containing bb.
The transition table for the corresponding FA is constructed as:
| Old States | New States after reading a | New States after reading b |
|---|---|---|
| z₁ - ≡ x₁ | x₁ ≡ z₁ | (x₁, x₂) ≡ z₂ |
| z₂ ≡ (x₁, x₂) | (x₁, ∅) ≡ x₁ ≡ z₁ | (x₁, x₂, x₃) ≡ z₃ |
| z₃ + ≡ (x₁, x₂, x₃) | (x₁, x₃) ≡ z₄ | (x₁, x₂, x₃) ≡ z₃ |
| z₄ + ≡ (x₁, x₃) | (x₁, x₃) ≡ z₄ | (x₁, x₂, x₃) ≡ z₃ |
The corresponding transition diagram follows as:
- z₁⁻ — on 'a' goes to z₁, on 'b' goes to z₂
- z₂ — on 'a' goes to z₁, on 'b' goes to z₃⁺
- z₃⁺ — on 'a' goes to z₄⁺, on 'b' goes to z₃⁺
- z₄⁺ — on 'a' goes to z₄⁺, on 'b' goes to z₃⁺
🔑 Definition — Empty State (∅): A state introduced when a letter has no transition from a given NFA state; it behaves as a sink state in the FA. 📐 Method 3 Rule: For each letter, the new state is the set of all NFA states reachable (including ∅ if none) → represents deterministic behavior.
NFA and Kleene’s Theorem
Kleene’s Theorem Part III states that there exists an FA corresponding to any given regular expression (RE). For simple REs like r = aa+bbb or r = a(a+b)*, FAs can be constructed directly. For complicated REs, the RE is decomposed into simple REs, and FAs for sum, concatenation, and closure are combined. NFAs also help in proving this theorem.
💡 Why this matters: NFAs provide a systematic way to construct FAs for complex regular expressions by breaking them into smaller, manageable parts.
NFA and Kleene’s Theorem — Method 1
Build NFAs for simple languages, then convert them to equivalent FAs.
Example: Construct FAs for languages L₁ = {a}, L₂ = {b}, and L₃ = {Λ}.
Step 1: Build NFA₁, NFA₂, and NFA₃:
- NFA₁: Start state → on 'a' → final state (accepts {a})
- NFA₂: Start state → on 'b' → final state (accepts {b})
- NFA₃: Start and final state are the same (accepts {Λ})
Step 2: Since every NFA has an equivalent FA, convert the NFAs to FAs:
- FA₁: Accepts {a} (single transition on 'a' from start to final)
- FA₃: Accepts {Λ} (start state is final)
NFA and Kleene’s Theorem — Method 2
If an NFA can be built corresponding to union, concatenation, and closure of FAs for REs, converting that NFA to an FA yields the FA for the given RE.
NFA Corresponding to Union of FAs
Method: Introduce a new start state and connect it with the states originally connected with the old start state using the same transitions. Then remove the -ve sign (non-final status) from the old start state. This creates non-determinism, resulting in an NFA.
Example: Consider FA₁ (accepts language with 'b') and FA₂ (accepts language with 'a').
FA₁: States with transitions on 'a' and 'b' FA₂: States with transitions on 'a' and 'b'
To construct the NFA equivalent to FA₁ ∪ FA₂:
- Introduce a new start state (-)
- From the new start state, add transitions: on 'a' to the states reachable from the old start of FA₁ on 'a' and from the old start of FA₂ on 'a'; on 'b' similarly.
- Remove the -ve sign from the old start states.
- This creates multiple transitions from the new start state, introducing non-determinism.
The resulting NFA accepts strings that are in either FA₁ or FA₂.
⭐ Key Takeaways
- Method 3 for NFA-to-FA conversion uses combination states (sets of NFA states) and an empty state (∅) to handle multiple or missing transitions, ensuring deterministic behavior.
- Kleene’s Theorem Part III can be proven by decomposing complex REs into simple components and using NFAs for union, concatenation, and closure operations.
- Method 1 builds NFAs for atomic languages {a}, {b}, {Λ}, then converts them to equivalent FAs.
- Method 2 constructs NFAs for union by introducing a new start state with transitions to states connected to old start states, creating non-determinism.
- The transition table method systematically tracks all possible NFA state combinations as single FA states, ensuring the final FA is deterministic.
🧠 Quick Revision Questions
- In Method 3 for NFA-to-FA conversion, what does the empty state (∅) represent, and when is it introduced?
- How does the transition table in the example handle the case where reading 'a' from state x₂ leads to no transition?
- In Method 1 of Kleene’s theorem, what are the three atomic languages for which NFAs are first built?
- When constructing an NFA for the union of two FAs, what is the purpose of introducing a new start state?
- Why is the NFA in Method 2 considered non-deterministic after adding the new start state?
📘 Lecture 18 — Theory of Automata
📖 Overview: This lecture covers the construction of NFAs corresponding to the union, concatenation, and closure (Kleene star) of finite automata. It provides systematic methods and detailed examples for building complex automata from simpler ones, which is fundamental for understanding regular operations and expressions.
🗂️ Topics Covered
NFA corresponding to union of FAs with an example, NFA corresponding to concatenation of FAs with methodology and three detailed examples (no FA accepts Null string, FA₂ accepts Null string, both FAs accept Null string), and NFA corresponding to closure of an FA with methodology and two examples.
📝 Lecture Summary
Example (Union of FAs)
The lecture begins by presenting two FAs, FA₁ and FA₂, and showing the construction of an NFA equivalent to FA₁ ∪ FA₂. FA₁ has states p- (initial) and q, with transitions on a and b between them. FA₂ has states 1-, 2, 3, 4, 5, and 6+, with a more complex transition structure. The resulting NFA merges both automata by connecting the initial states to a new start state with ε-transitions.
🔑 Definition — Union of FAs: A construction that yields an automaton accepting strings that are accepted by either FA₁ or FA₂.
NFA corresponding to Concatenation of FAs
Method: Introduce additional transitions for each letter connecting each final state of the first FA with the states of second FA that are connected with the initial state of second FA corresponding to each letter of the alphabet. Remove the +ve sign of each final state of first FA and –ve sign of the initial state of second FA. This creates non-determinism at final states of first FA, yielding the required NFA.
Note: If first FA accepts the Null string, then every string accepted by second FA must be accepted by the concatenation. If second FA accepts Null string, then every string accepted by first FA must be accepted — in this case, the +ve sign of final states of first FA will not be removed. If both FAs accept the Null string, then the Null string must be accepted by the required FA.
Example (No FA accepts Null string)
FA₁ has states 1– (initial), 2, 3, 4, and 5+ with transitions on a and b. FA₂ has states p- (initial), q, and r+ with transitions on a and b. The resulting NFA equivalent to FA₁FA₂ is constructed by:
- Removing the + sign from state 5 and the – sign from state p
- Adding transitions from state 5 (former final of FA₁) to states reachable from p in FA₂
📌 Example: FA₁ accepts strings ending in 'a' (state pattern), FA₂ accepts strings containing 'b'. Their concatenation accepts strings where a string ending in 'a' is followed by a string containing 'b'.
Example (FA₂ accepts Null string)
FA₁ has states p- (initial) and q with transitions on a and b. FA₂ has states with final state r+ that accepts all strings (a,b at r). Since FA₂ accepts the Null string, the +ve sign of final states of FA₁ is not removed. The resulting NFA equivalent to FA₁FA₂ preserves the final status of FA₁'s accepting states.
Example (Both FAs accept Null string)
FA₁ has states 1± (initial and final) with transitions on a,b to 2 and 3, with 3 being final. FA₂ has states with initial state and final state. Both accept the Null string. The resulting NFA equivalent to FA₁FA₂ shows the construction where final sign of first FA's states remains, and transitions connect states appropriately.
NFA corresponding to the Closure of an FA
Method: Introduce an initial state which should be final as well (so that the Null string is accepted). Connect it with the states originally connected with the old start state with the same transitions as the old start state. Remove the –ve sign of old start state. Introduce new transitions, for each letter, at each of the final states (including new final state) with those connected with the old start state. This creates non-determinism and results in the required NFA.
Example (Closure)
Consider an FA with a single initial state. The FA* accepts only the additional string which is the Null string. Simply making the initial state final would allow unwanted strings to be accepted. The required NFA introduces a new initial state with appropriate transitions.
Example (Another Closure)
Consider an FA with states and transitions on a and b. The FA* accepts only the additional Null string. As observed, the required NFA is constructed by introducing the new initial state shown in the diagram.
💡 Why this matters: The closure construction is essential because simply making the initial state final would incorrectly accept strings that the original FA would reject.
⭐ Key Takeaways
The most critical concepts from this lecture are: (1) For union, NFAs combine FAs using a new start state with ε-transitions to the original initial states. (2) For concatenation, transitions from final states of the first FA are added to states reachable from the initial state of the second FA, with careful handling of the + and – signs based on whether each FA accepts the Null string. (3) For closure (Kleene star), a new initial state that is also final must be introduced, with transitions mimicking the old start state and additional transitions from all final states. (4) The Null string acceptance rules are critical: if the second FA accepts ε, final signs of the first FA are preserved; if both accept ε, the concatenation also accepts ε. (5) The closure construction prevents acceptance of unwanted strings that would occur if the original initial state were simply made final.
🧠 Quick Revision Questions
- When constructing the NFA for concatenation of two FAs, under what conditions do you not remove the + sign from the final states of the first FA?
- What additional transitions are introduced in the closure (Kleene star) construction for an FA, and why must a new initial state be introduced?
- In the union construction, how do you connect the two separate FAs to form the NFA equivalent to FA₁ ∪ FA₂?
- If FA₁ accepts the Null string but FA₂ does not, what modification is needed in the concatenation construction?
- Why can't you simply make the initial state of an FA final to construct its closure (FA*)?
📘 Lecture 19 — NFA corresponding to Closure of FA, Examples, Memory required to recognize a language, Example, Distinguishing one string from another, Example, Theorem, Proof
📖 Overview: This lecture explores how to construct NFAs corresponding to the closure (Kleene star) of an FA, and examines the memory requirements for recognizing different languages. It introduces the crucial concept of distinguishable strings, which forms the basis for a fundamental theorem about the minimum number of states required in any FA that recognizes a given language.
🗂️ Topics Covered
The lecture covers NFA construction for the closure of FA with examples, memory requirements to recognize languages including detailed analysis for L₃ and L₂₀ languages, the concepts of distinguishable and indistinguishable strings with examples, and the theorem stating that if n strings are pairwise distinguishable with respect to a language L, then any FA recognizing L must have at least n states, along with its complete proof.
📝 Lecture Summary
Example (Closure of FA)
Consider the following FA that accepts all strings. It can be observed that FA* not only accepts the Null string but every other string as well. Here we don’t need separate initial and final state. Hence an NFA corresponding to FA* may be constructed as shown.
📌 Example: For the given FA with states 1± and 2+ (both initial and final), the NFA for the closure keeps the same structure but allows transitions without needing distinct initial/final state separation.
Memory required to recognize a language
Memory required to recognize a language means to look at the machine which can recognize a language. An FA can be considered to be a simple model of computation, and every regular language is associated with a certain FA. To recognize a language, there is a restriction that there is a single pass from left to right for any string to decide whether it belongs to a certain language. This helps to remember information about the initial part of the string read so far.
Consider L = {w ∈ {a,b}* : w neither ends in ab nor in ba}. i.e. L is the language of strings, defined over Σ = {a,b}, consisting of Λ, a, b and strings ending in aa or bb. As seen in the above FA, seven states are required to recognize the language L, while on the other hand it is very hard to recognize the language PALINDROME.
Now consider another language L₃ of strings of length three or more, defined over Σ = {a,b}, and the third letter from the right is a. As discussed by Martin, there is a straightforward method to build an FA recognizing L₃: a distinct state for every possible substring of length less than or equal to 3. It is obvious that for each length i, i=0,1,2,3, of substring, the number of states are 2ⁱ and thus total number of states required to recognize the language L₃ are 2⁰+2¹+2²+2³ = 2³⁺¹-1 = 15 (using 2⁰+2¹+2²+...+2ⁿ = 2ⁿ⁺¹-1).
📐 Formula: Total states = 2ⁿ⁺¹ - 1, where n is the position from the right being checked.
Remark: Let L₂₀ be the language of strings of length 20 or more, defined over Σ = {a,b}, and the 20th letter from the right is 1, then following the previous method, number of states for the corresponding FA is 2²⁰⁺¹-1 = 2,097,151. However, it may be noted that any portion of memory of a computer that can accommodate 21 bits can be in 2²¹ possible states i.e. 2²¹ possible choices for the informational content.
💡 Why this matters: This shows the exponential growth in states required for languages defined by a fixed position from the right, and how this relates to computer memory requirements.
Distinguishable strings and Indistinguishable strings
Two strings x and y, belonging to Σ*, are said to be distinguishable w.r.t a language L ⊆ Σ* if there exists a string z belonging to Σ* such that xz ∈ L but yz ∉ L or xz ∉ L but yz ∈ L.
Two strings x and y, belonging to Σ*, are said to be indistinguishable with respect to a language L ⊆ Σ* if for every string z belonging to Σ*, either both xz and yz ∈ L or both don’t belong to L.
🔑 Definition — Distinguishable strings: Strings x and y are distinguishable w.r.t. L if there is some suffix z that causes one concatenation to be in L and the other not.
📌 Example: Let L be the language of strings, defined over Σ = {0,1}, ending in 01. The strings 110 and 010011 are distinguishable w.r.t L, as there exists 1 belonging to Σ* such that 1101 belongs to L but 0100111 doesn’t belong to L. But 111 and 010011 are indistinguishable, for 1 belonging to Σ* such that both 1111 and 010011 don’t belong to L i.e. for every z belonging to Σ*, either both 111z and 01001z belong to L, or both don’t belong to L.
Theorem
Statement: If L is a language over an alphabet Σ and for integer n there are n strings from Σ, any two of which are distinguishable* w.r.t. language L, then any FA that recognizes L must have at least n states. (Note: There may not exist any FA which recognizes the given language.)
🔑 Definition — Theorem implication: This provides a lower bound on the number of states needed for any FA recognizing L.
📐 Proof: Let S be set of strings, any two of which are distinguishable w.r.t. language L. Let F₁ be the FA which recognizes the language L. To prove the theorem, it is sufficient to show that any two strings under F₁ must end in different states i.e. corresponding to each string x belonging to S, F₁ ends in distinct states. Thus if S has n strings then it is to be shown that F₁ has at least n states.
Let x and y be any two strings from S. By supposition any two strings of S are distinguishable w.r.t. L, so there exists a string z belonging to Σ* such that only one of xz and yz belongs to L i.e. F₁ ends in a final state either for xz or yz which shows that F₁ ends in distinct states for xz and yz.
Let F₁ be ended in same state for both the strings x and y, which shows that F₁ ends in same state for both xz and yz, a contradiction as x and y being distinguishable implies xz and yz are ended at distinct states of F₁. Hence F₁ does not end in a same state for both strings x and y, which shows that each pair of strings belonging to S ends in different states. Hence F₁ must contain at least n states.
💡 Why this matters: This theorem provides the theoretical foundation for proving that certain languages cannot be recognized by FAs with fewer than a specific number of states, and is essential for understanding the Myhill-Nerode theorem.
⭐ Key Takeaways
The most critical concepts to remember are: 1) The number of states required for an FA recognizing a language defined by a specific position from the right (like the nth letter) is 2ⁿ⁺¹ - 1, which grows exponentially; 2) Two strings are distinguishable with respect to a language L if there exists some suffix that separates them (one concatenation is in L, the other is not); 3) If n strings are pairwise distinguishable w.r.t. L, then any FA recognizing L requires at least n states — this is proven by contradiction, showing each distinguishable pair must end in different states; 4) Indistinguishable strings always behave the same way when any suffix is appended; 5) This theorem provides a lower bound on FA size and can prove that certain languages are not regular if infinite distinguishable strings exist.
🧠 Quick Revision Questions
-
How many states are required for an FA recognizing the language where the 5th letter from the right is 'a' over alphabet {a,b}?
-
Why can't two distinguishable strings end in the same state in an FA that recognizes language L?
-
In the language L ending in "01", are the strings "101" and "010" distinguishable or indistinguishable? Justify your answer.
-
What is the relationship between the number of states in an FA and the number of pairwise distinguishable strings with respect to the language it recognizes?
-
If we have 100 pairwise distinguishable strings for language L, what can we conclude about any FA that recognizes L?
📘 Lecture N0. 20 — Theory of Automata
📖 Overview: This lecture begins by applying a previous theorem to show the minimum number of states required for an FA accepting a language defined by the 20th letter from the right. It then introduces the concept of finite automata with output, specifically defining and demonstrating the Moore machine, a model that produces an output character for each state entered.
🗂️ Topics Covered
The lecture covers an example demonstrating that any FA accepting the language L20 (strings where the 20th letter from the right is 1) must have at least 2^20 states, followed by a note on the memory required. The main topic shifts to finite automata with output, introducing the Moore machine, its formal definition, and providing a complete example with a transition table, transition diagram, input string, and output string.
📝 Lecture Summary
Example
The lecture examines the language L₂₀ = {w ∈ {0,1}* : |w| ≥ 20 and the 20th letter of w, from right is, 1}. Let S be the set of all strings of length 20, any two of which are distinguishable with respect to L₂₀. There are 2²⁰ strings in S. Let x and y be any two distinct strings in S, differing in the i-th letter from the left.
If i=1, they differ by the first letter from left. By definition of L₂₀, one string is in L₂₀ while the other is not, as shown below: 0... . 1... .
For z = Λ (empty string), one of xz and yz belongs to L₂₀, so x and y are distinguishable.
If i=2, they differ by the 2nd letter from left. For any z ∈ Σ* with |z|=1, either xz or yz belongs to L₂₀, because the 20th letter from the right of xz and yz is the 2nd letter from left of x and y, as shown below: . 0 ... . z . 1 ... . z
Continuing this process, any pair of strings x and y in S will be distinguishable with respect to L₂₀. Since S contains 2²⁰ strings, all pairwise distinguishable, using the theorem, any FA accepting L₂₀ must have at least 2²⁰ states. 💡 Why this matters: This demonstrates a powerful lower bound on the number of states required for a language defined by a distant character, showing exponential state requirements for simple-looking conditions.
Note
Using Martin’s method, there exists an FA having 2²⁰⁺¹ - 1 = 2,097,151 states. This indicates the memory required to recognize L₂₀ is the memory of a computer that can accommodate 21-bits, i.e., the computer can be in 2²¹ possible states.
Finite Automaton with output
So far, finite automata are associated only with regular expressions or languages. A key question is whether an FA can generate an output string corresponding to each input string. The answer is yes. Such machines are called machines with output. There are two types: Moore machine and Mealy machine.
Moore machine
A Moore machine consists of the following:
- A finite set of states q₀, q₁, q₂, ... where q₀ is the initial state.
- An alphabet of letters Σ = {a,b,c,...} from which input strings are formed.
- An alphabet Γ = {x,y,z,...} of output characters from which output strings are generated.
- A transition table that shows for each state and each input letter what state is entered next.
- An output table that shows what character is printed by each state as it is entered.
🔑 Definition — Moore machine: A finite automaton that produces an output character for each state it enters, including the initial state before reading any input.
Note: In a Moore machine, no state is designated to be a final state, so there is no question of accepting any language. However, the relation between an input string and the corresponding output string may be identified. The initial state is not especially important, as restarting the machine from the state it was left off is possible.
Example
Consider a Moore machine with states q₀, q₁, q₂, q₃ where q₀ is the start state, Σ = {a,b}, and Γ = {0,1}.
The transition table is:
| Old States | New States after reading a | New States after reading b | Characters to be printed |
|---|---|---|---|
| q₀- | q₁ | q₃ | 1 |
| q₁ | q₃ | q₁ | 0 |
| q₂ | q₀ | q₃ | 0 |
| q₃ | q₃ | q₂ | 1 |
The transition diagram corresponding to the previous transition table may be:
[Diagram with states labeled: q₀ (1), q₁ (0), q₂ (0), q₃ (1). Transitions: q₀ --a--> q₁, q₀ --b--> q₃; q₁ --a--> q₃, q₁ --b--> q₁; q₂ --a--> q₀, q₂ --b--> q₃; q₃ --a--> q₃, q₃ --b--> q₂]
It is to be noted that the states are labeled along with the characters to be printed. Running the string abbabbba over the above machine, the corresponding output string will be 100010101, which can be determined by the following table as well:
| Input | a | b | b | a | b | b | b | a |
|---|---|---|---|---|---|---|---|---|
| State | q₀ | q₁ | q₁ | q₁ | q₃ | q₂ | q₃ | q₂ |
| output | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
It may be noted that the length of output string is 1 more than that of input string, as the initial state prints out the extra character 1 before the input string is read. 📌 Example: Given the Moore machine above and the input string "a", the sequence of states is q₀ → q₁. The output is the character printed by q₀ (which is '1') followed by the character printed by q₁ (which is '0'), resulting in the output string "10". For an input string "ab", the states are q₀ → q₁ → q₁. The outputs are: q₀ prints 1, q₁ prints 0, and then q₁ prints 0, giving output "100".
⭐ Key Takeaways
The key takeaway is that the number of states required for an FA can grow exponentially with simple language constraints, as shown by the 2²⁰ state lower bound for L₂₀. A Moore machine is a fundamental type of finite automaton with output that produces an output character every time a state is entered, including the initial state. Unlike standard FAs, Moore machines do not accept languages, but rather define a mapping from input strings to output strings. The length of the output string from a Moore machine is always one greater than the length of the input string. When reading an input string, the next state is determined by the current state and the input letter, and the output of that next state is printed.
🧠 Quick Revision Questions
- Why must any FA accepting L₂₀ have at least 2²⁰ states?
- What is the key difference between a finite automaton that accepts a language and a Moore machine?
- What are the four essential components that define a Moore machine?
- For the Moore machine in the example, what is the output string for the input string "ba"?
- Why is the output string from a Moore machine always one character longer than the input string?
📘 Lecture 21 — Theory of Automata
📖 Overview: This lecture introduces the Mealy machine, compares it with the Moore machine, and demonstrates how to construct specific Mealy machines such as the complementing machine and the incrementing machine. Understanding these machines is crucial for grasping how finite automata can generate output sequences based on input strings.
🗂️ Topics Covered
Example of Moore machine showing relation between input substrings and output characters; definition and working of Mealy machine with examples; complementing machine that produces 1's complement of input; construction of the incrementing machine that adds 1 to a binary number read from right to left, including overflow situation.
📝 Lecture Summary
Example
To identify the relation between the input strings and the corresponding output strings in the given Moore machine, if the string bbbabaabbaa is run, the output string will be 000010000010.
It can be observed from the Moore machine that q₃ is the only state which prints out the character 1, which shows that the moment the state q₃ is entered, the machine will print out 1. To enter the state q₃, starting from q₀, the string must contain bba. It can also be observed that to enter the state q₃ once more, the string must contain another substring bba. In general, the input string will visit the state q₃ as many times as the number of substring bba occurs in the input string. Thus the number of 1's in an output string will be the same as the number of substring bba occurs in the corresponding input string.
📌 Example: Input b b b a b a a b b a a → Output 0 0 0 0 1 0 0 0 0 0 1 0
Mealy machine
A Mealy machine consists of the following:
- A finite set of states q₀, q₁, q₂, ... where q₀ is the initial state.
- An alphabet of letters Σ = {a,b,c,...} from which the input strings are formed.
- An alphabet Γ={x,y,z,...} of output characters from which output strings are generated.
- A pictorial representation with states and directed edges labeled by an input letter along with an output character. The directed edges also show how to go from one state to another corresponding to every possible input letter.
⚠️ Note: It is not possible to give a transition table in this case.
💡 Why this matters: Unlike Moore machines where output is associated with states, Mealy machines produce output on transitions, making them more efficient for certain applications.
Note: Similar to Moore machine, in Mealy machine no state is designated to be a final state, so there is no question of accepting any language by Mealy machine. However, in some cases the relation between an input string and the corresponding output string may be identified by the Mealy machine. Moreover, the state to be initial is not important, as if the machine is used several times and is restarted after some time, the machine will be started from the state where it was left off.
Example
Consider the Mealy machine having the states q₀, q₁, q₂, q₃, where q₀ is the start state and Σ = {a,b}, Γ={0,1}. Running the string abbabbba over the machine, the corresponding output string will be 11011010.
It may be noted that in Mealy machine, the length of output string is equal to that of input string.
📌 Example: Input a b b a b b b a → Output 0 1 1 1 1 0 1 0
Example
Consider the Mealy machine having the states q₀, q₁, q₂, where q₀ is the start state and Σ = {a,b}, Γ={0,1}. It is observed that if in the output string the nth character is 1, it shows that the nth letter in the input string is the second in the pair of double letter. For babaababba as input string the machine will print 0000100010.
Example
Consider the Mealy machine having the only state q₀ as the start state and Σ = {0,1}, Γ= {0,1} with transitions 0/1, 1/0. If 0011010 is run on this machine then the corresponding output string will be 1100101. This machine is called Complementing machine.
Constructing the incrementing machine
In the previous example of complementing machine, it has been observed that the input string and the corresponding output string are 1's complement of each other. There is a question whether the Mealy machine can be constructed so that the output string is increased, in magnitude, by 1 than the corresponding input string? The answer is yes. This machine is called the incrementing machine.
Before constructing the incrementing machine, consider how 1 is added to a binary number. Since if two numbers are added, the addition is performed from right to left, so while increasing the binary number by 1, the string (binary number) must be read by the corresponding Mealy machine from right to left, and hence the output string (binary number) will also be generated from right to left.
Consider the following additions: a) 100101110 + 1 = 100101111 b) 1001100111 + 1 = 1001101000
It may be observed that:
- If the rightmost bit of binary number to be incremented is 0, the output binary number can be obtained by converting the rightmost bit to 1 and remaining bits unchanged.
- If the rightmost bit of binary number is 1, then the output can be obtained by converting that 1 along with all its concatenated 1's to 0's, then converting the next 0 to 1 and remaining bits unchanged.
The observations help to construct the Incrementing (Mealy) machine. The Mealy machine has the states q₀, q₁, q₂, where q₀ is the start state and Σ = {0,1}, Γ={0,1}.
It may be observed that, in the incrementing machine, if 0 is read at initial state q₀, that 0 is converted to 1 and a no change state q₁ (no carry state) is entered where all 0's and all 1's remain unchanged. If 1 is read at initial state, that 1 is converted to 0 and the state q₂ (owe carry state) is entered, where all 1's are converted to 0's and at that state if 0 is read, that 0 is converted to 1 and the machine goes to no change state.
📌 Example: If the strings 100101110 and 1001100111 are run over this machine, the corresponding output strings will be 100101111 and 1001101000 respectively.
Note: It is to be noted that if the string 111111 is run over the incrementing machine, the machine will print out 000000, which is not increased in magnitude by 1. Such a situation is called an overflow situation, as the length of output string will be the same as that of input string.
💡 Why this matters: The overflow situation demonstrates a fundamental limitation — the machine cannot represent a carry beyond the most significant bit because input and output strings must have equal length.
It may also be noted that there exists another incrementing machine with two states.
⭐ Key Takeaways
The most critical points to remember are: (1) In a Moore machine, output is associated with states (each state prints a character when entered), while in a Mealy machine, output is associated with transitions (each edge is labeled with input/output). (2) The number of 1's in the output of the given Moore machine equals the number of occurrences of substring "bba" in the input, demonstrating how Moore machines can count pattern occurrences. (3) Mealy machines always produce an output string of the same length as the input string, and no state is designated as final. (4) The complementing machine is a simple one-state Mealy machine that inverts each bit (0→1, 1→0). (5) The incrementing machine reads binary numbers from right to left and adds 1, handling carries through states q₁ (no carry) and q₂ (owe carry), but suffers from overflow when the input is all 1's.
🧠 Quick Revision Questions
- What is the fundamental difference between how Moore machines and Mealy machines associate output with input?
- In the example Moore machine, what substring must occur in the input for the machine to output a 1, and why?
- For the incrementing machine, why must the input be read from right to left rather than left to right?
- What happens when the string "111111" is run on the incrementing machine, and what is this situation called?
- In the Mealy machine example with states q₀, q₁, q₂ (where nth output character 1 indicates double letter), what would the output be for input string "aabbaa"?