All operators are tokens, hence ++ is taken as a single token.

Correct if wrong.

Correct if wrong.

The Gateway to Computer Science Excellence

+7 votes

Best answer

Preprocessor follows greedy approach to tokenize, This can be done in TWO steps

1 => Separated by space (A+ ++B) => Have 4 tokens(Space is not considered as token), here tokens are A, +, ++, B(In order of expression). If any operator doesn't have any space ,

then ,

2 => Greedy approach (A+++B) for longest possible valid operator in the Grammer => Like we have '++' > '+' so here the tokens are A,++, +, B(In order of expression).

1 => Separated by space (A+ ++B) => Have 4 tokens(Space is not considered as token), here tokens are A, +, ++, B(In order of expression). If any operator doesn't have any space ,

then ,

2 => Greedy approach (A+++B) for longest possible valid operator in the Grammer => Like we have '++' > '+' so here the tokens are A,++, +, B(In order of expression).

0

@Harish Kumar 2 I think there is nothing like priority in tokenization.

While performing tokenization, Compiler use greedy approach and tries to include more token if possible,

0

Ok, so after Lexical analysis then it checks then the compiler checks which operator to be formed?

Like for "A+++B"(if used some where) then, how many tokens? and how will it use this. Or space is used to seperate these now. Kindly clarify.

Like for "A+++B"(if used some where) then, how many tokens? and how will it use this. Or space is used to seperate these now. Kindly clarify.

0

Then tokens are :-

A

++

+

B

Even you can run on ide

int main() { int a,b; a = 10; b = 10; printf("Hello World %d \nval of a = %d\nval of b = %d", (a+++b), a, b); return 0; }

Output

Hello World 20

val of a = 11

val of b = 10

0

So then priority plays role in getting tokens, right? Because if not, then by greedy approach it should have considered it as '+' only, Shouldn't it???

0

For tokenization, greedy is used and while solving the expression,

like a+++b

priority of operators are used because unary operator have higher precedence than binary operator.

like a+++b

priority of operators are used because unary operator have higher precedence than binary operator.

- All categories
- General Aptitude 1.9k
- Engineering Mathematics 7.5k
- Digital Logic 2.9k
- Programming and DS 4.9k
- Algorithms 4.3k
- Theory of Computation 6.2k
- Compiler Design 2.1k
- Databases 4.1k
- CO and Architecture 3.4k
- Computer Networks 4.1k
- Non GATE 1.5k
- Others 1.5k
- Admissions 595
- Exam Queries 576
- Tier 1 Placement Questions 23
- Job Queries 72
- Projects 17

50,644 questions

56,505 answers

195,555 comments

101,052 users