Number of tokens in the C program

Question

Given answer is 25, while my answer is 26, they have considered -10.0 as single token, which is wrong?

Comments

yes, it should be 26.

---

1. Keywords               (int, while,etc),
2. Identifiers               ( main, total,etc),
3. Constants              ( 10, 20),
4. Strings                    ( “total”, “hello”),
5. Special symbols  ( (,), {,} etc),
6. Operators              ( +, /,-,*,etc)

So yes 26. Minus is included.

---

But floating point constants are also allowed right @manu00x , @Joshi_nItIsh , @Warlock lord ?

---

https://stackoverflow.com/questions/26529711/how-to-differentiate-operator-from-a-negative-number-for-a-tokenizer

Answer 1

Ans is 25

i think you done mistak -10.0 take tow token but only 1.

Lexical analyzer checks for a longest valid lexeme which matches the pattern in case of "<=", it is the longest valid pattern so it will see "-10.0" as one pattern (as it is also a longest valid pattern).

Comments

why will -10.0 be counted as one token? and from where you copied this solution?? - is operator and 10.0 is a constant.

---

@manu00x

it should be 26 only...

'-10.0' will never be counted as a single token.

---

@joshi_nitish

25 is right

Lexical analyzer checks for a longest valid lexeme which matches the pattern in case of "<=", it is the longest valid pattern so it will see "-10.0" as one pattern (as it is also a longest valid pattern).

---

Bhargav D Dave 6

Do you really understand what you are talking?

---

yes

---

operator and operand cannot come under single token

---

@Shaik Masthan @Na462 What is your opinion regarding this answer?

---

as per me also it is 26,

first number ( consecutive digits ) found in lexical analysis, at syntax phase it is derived, and at semantic phase it appends '-' operator to the number and check the semantics.

---

Thanks :)

---

i also counted 26 but

listen one thing

given     x<=-10.0

here      <=     is one token  means this is compresion  with -10.0

i think  -10.0  when we represent  it  in 2complement  form  then -10.0 will act as one no

what is your view on this  ??

---

Yes 26 is correct and -10.0 will be counted as 2 tokens..

Can you find the no. of tokens in given fonction---

main()

{ 

in/*comment t x;

floa/*comment*/ t gate;

}

 

---

i think 9

---

16

---

@gurdeep in above question we are takig 10.0 as single token ?

---

@Prince Sindhiya  16 How??

---

@Prince Sindhiya yes 10.0  is single token

---

gurdeep means 10.0 is a constant but why 10,.,0 all are not 3 tokens?

---

@Prince Sindhiya can you tell me the category of , token 

---

@VERMA ASHISH SEE https://gateoverflow.in/262335/number-of-tokens

---

10-CONSTANT . OPERATOR , 0 CONSTANT

please correct me if i am wrong

---

Lexical analyzer(scanner) follows greedy approach..
It try to find lexeme with the maximum length..
Like  ++ counted as single token as long as  both '+' operators are not seperated by whitespace..
if it is 10  .  0 then 3 tokens counted..

---

Prince Sindhiya if you do like that then how are you going to get a floating number ???

Using Lexical Analysis we get token by recognizing the pattern, floating number has a decimal point. we need to recognize the whole string to get a floating number.

---

are you considering             .=*           ?

---

Gurdeep Saini

The category of comma(,) is separator.

 separators   ; | , | . | ( | ) | { | } | [ | ]

---

@Prince Sindhiya

main()

{ 

in/*comment t x;

floa/*comment*/ t gate;

}

 I think here tokens are 9.

It is different qiestion than the link what you shared.

---

@Shaik Masthan

lexical analysis, syntax analysis and symantic checking

all happens in lexer

right?

Parser doesnot do this

rt?

---

mam, recheck your comment !

---

srestha

lexical analysis, syntax analysis and symantic checking

all happens in lexer ??

 

Why? then what is the use of parser if lexer will do syntax analysis.

and parsers with SDD does semantic checking right ? Shaik Masthan

---

you are right @Aakash_

---

yes

lexical analysis for lexical error

parser for syntax error

semantic analyzer for semantic error

now right?