What are common examiner traps in Lexical Analysis?

Treating hybrid strings like `23mm` or `14A` as a single invalid token or single identifier rather than applying maximal munch to produce two separate tokens (`number` + `id`). Confusing lexical errors (malformed tokens like unclosed string literals) with syntactic errors (grammar rule violations) or semantic errors (type mismatches). Assuming the symbol table is discarded after parsing/syntax analysis; it is required through intermediate code generation, semantic analysis, and target code generation. Selecting an NFA that fails to allow alternating transitions between character classes (e.g., getting trapped in a `letter`-only or `digit`-only loop). Forgetting that whitespace and comments typically do not produce output tokens unless explicitly asked.

Lexical Analysis

7 Qs · since 2011 · 7 marks · 0.3 marks/paperStandard yield

Take topic test

Lexical Analysis in GATE CS is consistently tested as a 1-mark foundational topic covering tokenization mechanics, formal automaton models, error classification, and compiler data… Guide

Type

GATE CSE 2026 Set 1Q27MSQ1 markEasyConsider the following C statements: char *str1 = "Hello; /* Statement S1 */ char *str2 = "Hello;"; /* Statement S2 */ int *str3 = "Hello"; /* Statement S3 */ Which of the following options is/are correct?
GATE CSE 2026 Set 2Q35NAT1 markMediumA lexical analyzer uses the following token definitions • $letter \rightarrow [A - Z a - z]$ • $digit \rightarrow [0 - 9]$ • $id \rightarrow letter \text{ } (letter \mid digit)*$ • $number \rightarrow digit^+$ • $ws \rightarrow (blank \mid tab \mid newline)^+$ For the string given below, $x1\ \ 23mm\ 78\ y\ 7z\ zz5\ 14A\ 8H\ AaYcD$ …
GATE CSE 2025 Set 1Q12MCQ1 markEasyWhich ONE of the following statements is FALSE regarding the symbol table?
GATE CSE 2024 Set 2Q21MCQ1 markEasyDiagramConsider the following two sets: Which one of the following options is the CORRECT match from Set X to Set Y ?
GATE CSE 2023Q19MCQ1 markEasyConsider the following definition of a lexical token id for an identifier in a programming language, using extended regular expressions: $\begin{aligned} \text{letter} &\rightarrow [\text{A-Za-z}] \\ \text{digit} &\rightarrow [0\text{-}9] \\ \mathbf{id} &\rightarrow \text{letter} (\text{letter} \mid \text{digit})^* \end{aligned}$ …
GATE CSE 2011Q1MCQ1 markEasyIn a compiler, keywords of a language are recognized during
GATE CSE 2011Q19MCQ1 markEasyThe lexical analysis for a modern computer language such as Java needs the power of which one of the following machine models in a necessary and sufficient sense?

Topic guide

What does GATE test in Lexical Analysis?

Lexical Analysis in GATE CS is consistently tested as a 1-mark foundational topic covering tokenization mechanics, formal automaton models, error classification, and compiler data structure lifecycle. Recent years have shifted from purely definitional phase-matching questions toward practical token segmentation using the maximal munch rule and formal construction of NFA/DFA models from regular expressions.

Core concepts

Tokens, Patterns, and Lexemes
Maximal Munch / Longest Match rule
Finite State Automata as the computational engine for lexical analysis
NFA with epsilon-transitions constructed from regular expressions
Lexical errors (e.g., unclosed string literals, unrecognized characters) vs Syntactic and Semantic errors
Symbol table initialization during lexical analysis and its cross-phase persistence
Handling of whitespace and comments

Recurring question patterns

Token Counting with Maximal Munch (Longest Match)
common · NAT · 1 marks · 2026
Given regular expression definitions for tokens (identifiers, numbers, whitespace) and an input string, compute the exact number of tokens recognized by applying greedy prefix matching.
Error Classification Across Compiler Phases
occasional · MSQ · 1 marks · 2026
Given snippets of code with errors (e.g., unclosed string literals, invalid identifiers, type mismatches, missing brackets), classify which phase detects which error (Lexical, Syntactic, or Semantic).
Automata and Regular Expressions for Tokens
common · MCQ · 1 marks · 2023, 2011
Mapping token definitions (extended regular expressions) to corresponding Finite State Automata (NFA with $\epsilon$ -transitions or DFA) or identifying the theoretical power needed (FSA).
Role of Lexer and Symbol Table Interactions
common · MCQ · 1 marks · 2025, 2024, 2011
Conceptual verification of the lexical analyzer's duties (keyword identification, token stream generation) and symbol table management (creation at lexical phase, persistence across code generation).

Formulas GATE uses

Identifier Regular Definition

\text{id} \rightarrow \text{letter}(\text{letter} \mid \text{digit})^*

Used to specify valid identifier lexical patterns and evaluate corresponding NFA/DFA transitions.

Maximal Munch Token Split

\text{Token}_{i} = \operatorname{arg max}_{w \in \text{Prefix}(S)} (w \in L(R))

Used when scanning an input stream without whitespace delimiters where a prefix matches one token class and the remainder starts a new token (e.g., splitting `23mm` into `number` 23 and `id` mm).

Examiner traps in Lexical Analysis

Treating hybrid strings like `23mm` or `14A` as a single invalid token or single identifier rather than applying maximal munch to produce two separate tokens (`number` + `id`).
Confusing lexical errors (malformed tokens like unclosed string literals) with syntactic errors (grammar rule violations) or semantic errors (type mismatches).
Assuming the symbol table is discarded after parsing/syntax analysis; it is required through intermediate code generation, semantic analysis, and target code generation.
Selecting an NFA that fails to allow alternating transitions between character classes (e.g., getting trapped in a `letter`-only or `digit`-only loop).
Forgetting that whitespace and comments typically do not produce output tokens unless explicitly asked.

How questions are asked

Count total tokens produced by maximal munch for a given string
Classify error types in C statements as Lexical, Syntactic, or Semantic
Identify correct NFA/DFA diagram recognizing a token regular expression
Match compiler phases to output artifacts or optimization techniques
Select true/false conceptual statements about the symbol table and lexer

Paper trends

Transition from direct recall questions (e.g., matching phases or identifying FSA as the formal model) to constructive and evaluation-based questions (tracing maximal munch splits and NFA transitions).
2026, 2023, 2011
Introduction of error localization across phases using MSQ format and token-counting using NAT format.
2026

Difficulty notes

Easy items test theoretical models (FSA), symbol table lifecycle, standard phase matching, or straightforward error identification. Medium items test edge cases in token segmentation under greedy scanning (maximal munch) involving adjacent alphanumeric strings or recognizing precise NFA state transitions.

Lexical Analysis

7 Qs · since 2011 · 7 marks · 0.3 marks/paperStandard yield

Take topic test

Lexical Analysis in GATE CS is consistently tested as a 1-mark foundational topic covering tokenization mechanics, formal automaton models, error classification, and compiler data… Guide

Type

Topic guide

What does GATE test in Lexical Analysis?

Core concepts

Tokens, Patterns, and Lexemes
Maximal Munch / Longest Match rule
Finite State Automata as the computational engine for lexical analysis
NFA with epsilon-transitions constructed from regular expressions
Lexical errors (e.g., unclosed string literals, unrecognized characters) vs Syntactic and Semantic errors
Symbol table initialization during lexical analysis and its cross-phase persistence
Handling of whitespace and comments

Recurring question patterns

Token Counting with Maximal Munch (Longest Match)
common · NAT · 1 marks · 2026
Given regular expression definitions for tokens (identifiers, numbers, whitespace) and an input string, compute the exact number of tokens recognized by applying greedy prefix matching.
Error Classification Across Compiler Phases
occasional · MSQ · 1 marks · 2026
Given snippets of code with errors (e.g., unclosed string literals, invalid identifiers, type mismatches, missing brackets), classify which phase detects which error (Lexical, Syntactic, or Semantic).
Automata and Regular Expressions for Tokens
common · MCQ · 1 marks · 2023, 2011
Mapping token definitions (extended regular expressions) to corresponding Finite State Automata (NFA with $\epsilon$ -transitions or DFA) or identifying the theoretical power needed (FSA).
Role of Lexer and Symbol Table Interactions
common · MCQ · 1 marks · 2025, 2024, 2011
Conceptual verification of the lexical analyzer's duties (keyword identification, token stream generation) and symbol table management (creation at lexical phase, persistence across code generation).

Formulas GATE uses

Identifier Regular Definition

\text{id} \rightarrow \text{letter}(\text{letter} \mid \text{digit})^*

Used to specify valid identifier lexical patterns and evaluate corresponding NFA/DFA transitions.

Maximal Munch Token Split

\text{Token}_{i} = \operatorname{arg max}_{w \in \text{Prefix}(S)} (w \in L(R))

Used when scanning an input stream without whitespace delimiters where a prefix matches one token class and the remainder starts a new token (e.g., splitting `23mm` into `number` 23 and `id` mm).

Examiner traps in Lexical Analysis

Treating hybrid strings like `23mm` or `14A` as a single invalid token or single identifier rather than applying maximal munch to produce two separate tokens (`number` + `id`).
Confusing lexical errors (malformed tokens like unclosed string literals) with syntactic errors (grammar rule violations) or semantic errors (type mismatches).
Assuming the symbol table is discarded after parsing/syntax analysis; it is required through intermediate code generation, semantic analysis, and target code generation.
Selecting an NFA that fails to allow alternating transitions between character classes (e.g., getting trapped in a `letter`-only or `digit`-only loop).
Forgetting that whitespace and comments typically do not produce output tokens unless explicitly asked.

How questions are asked

Count total tokens produced by maximal munch for a given string
Classify error types in C statements as Lexical, Syntactic, or Semantic
Identify correct NFA/DFA diagram recognizing a token regular expression
Match compiler phases to output artifacts or optimization techniques
Select true/false conceptual statements about the symbol table and lexer

Paper trends

Transition from direct recall questions (e.g., matching phases or identifying FSA as the formal model) to constructive and evaluation-based questions (tracing maximal munch splits and NFA transitions).
2026, 2023, 2011
Introduction of error localization across phases using MSQ format and token-counting using NAT format.
2026