The Gateway to Computer Science Excellence
First time here? Checkout the FAQ!
+7 votes
When do we count ++(post increment or pre increment) as 1 or 2 token ?
asked in Compiler Design by Loyal (8.8k points)
edited by | 267 views
All operators are tokens, hence ++ is taken as a single token.

Correct if wrong.

This will be consider as single token, because compiler apply greedy approach for performing tokenization, it will try to include more character in token if possible.


@ Shubhanshu Thanks it helps :) /\

2 Answers

+6 votes
Best answer
Preprocessor follows greedy approach to tokenize,  This can be done in TWO steps

1 => Separated by space (A+ ++B) => Have 4 tokens(Space is not considered as token), here tokens are A, +, ++, B(In order of expression). If any operator doesn't have any space ,

then ,

2 => Greedy approach (A+++B) for longest possible valid operator in the Grammer => Like we have '++' > '+' so here the tokens are A,++, +, B(In order of expression).
answered by Junior (795 points)
selected by

@Harish Kumar 2 I think there is nothing like priority in tokenization.

While performing tokenization, Compiler use greedy approach and tries to include more token if possible,


Ok, so after Lexical analysis then it checks then the compiler checks which operator to be formed?

Like for "A+++B"(if used some where) then, how many tokens? and how will it use this. Or space is used to seperate these now. Kindly clarify.

Then tokens are :-





Even you can run on ide

int main()
    int a,b;
    a = 10;
    b = 10;
    printf("Hello World %d \nval of a = %d\nval of b = %d", (a+++b), a, b);
    return 0;


Hello World 20

val of a = 11

val of b = 10

So then priority plays role in getting tokens, right? Because if not, then by greedy approach it should have considered it as '+' only, Shouldn't it???
For tokenization, greedy is used and while solving the expression,

like a+++b

priority of operators are used because unary operator have higher precedence than binary operator.
Ok thanks Shubhansu, now I have got it.

And i have updated the answer, can you please look into that now and provide feedback.

@Harish Kumar 2 now it is correct.

Can anyone explain no. of tokens in case of unary operator such as the below line

int a = -3;

Is -3 is taken as single token or two tokens?
0 votes
I think 2 tokens because count is an identifier and ++ is an operator.
answered by (131 points)

Quick search syntax
tags tag:apple
author user:martin
title title:apple
content content:apple
exclude -tag:apple
force match +apple
views views:100
score score:10
answers answers:2
is accepted isaccepted:true
is closed isclosed:true

36,089 questions
43,532 answers
42,762 users