The Gateway to Computer Science Excellence
+1 vote
203 views

which one of the following string can definitely said to be a token without looking at the next input:

  1. +(++,+=)
  2. return(return a)
  3. *(*=)
  4. =(==)
  5. ++ , ( ) ‘ ;    option e is: plusplus, comma, bracket open, bracket close, single quote ;
in Compiler Design by (195 points) | 203 views
0
I think the question should also specify the programming language.
0
what is meaning of option b)??
0
Suppose if we have c language

return(return a)

if after ) it is ;  then it will be considered as token but wht if "return(return a)" inside printf ?

because for declaring a lexeme as token the compiler first checks the next character and then tells about the previous expression
0

@Satbir so in this question only in option e) a semicolon is coming therefore the lexeme pattern is matched here and it becomes a token? and in previous question no semicolon or no other separator is comg so it becomes necessary for the compiler to scan the next input. am i coorect here? also to which pattern does the option e match? it is simply operators separated by commas and ended with semi colon. please tell 

0
You can't say for since the programming language is not defined.
Like in python we don't have statements ending with semicolon.

https://gateoverflow.in/297978/self-doubt
0

language is c @Satbir

0

i got it. just one doubt. when we see semicolon then we dont need to see the character after it and we declare it as token. is this right?

and if this is right then do we do the same with comma or bracket close ) 

please clear this rest is all clear @Satbir

+2
No

printf( " No; No)" ); there are 5 tokens only.
0
im not getting this. in return(return a) if after ) we have an alphabet or a number then why wont ) be considered as a token?
0
If lets assume you consider ) as token then statement becomes right option.

then why you are not considering a,c and d options also  ?
0

can you explain the options in details? @Satbir

0
I am also little confused , but if we have to solve it we can do it by giving counter examples.

like in a+ we can't say that +  is a token without looking at next symbol because the next symbol could also be + , so ++ will be the token.

In the given options all the options are correct if we don't care what the next symbol is.

we have to generate counter examples to eliminate the options. I am not able to do it till now.
0
this seems to be a pretty confusing qquestion. because in all the above options a,b,c,d the ) can be said to be a token because there is no possibility after ) for it to make it another token(just like in ++, on seeing a + there is a possiblity of seeing another for it to be made as ++ token and not +) will remain as atoken whatever comes after it. and in e the string is terminated so ; is also a token whatever comes after it it will be considered next line.
0

@Arjunsir please help

0
Answer will be option e
0
Why ?
0
@Arjun sir please help
0

@SuvasishDutta why option e)?? 

it takes ++ and which depends on next input. ++ Cannot evaluate as token with single +.

0

@srestha mam, lexical analyzer, on seeing first + operator , will see the next character which is also + operator. After seeing the second + operator, lexical analyser will group the ++ operator into a lexeme without seeing the next character. Because after ++ operator , there is no character which will together form a single lexeme. Thus lexical analyser will generate token for lexeme ++(increment operator).

Why token is generated for ++ and not for +?

This is because lexical analyser generated a token for longest sequence that can form a token. 

Similar explanation for , ; ( ) ' operators in option e.

 

 

0
but why a,b,c,d are incorrect?
0

@srestha mam ,

see my answer. There i have discussed why option a,b are not correct. Option c and d have same explanation as a and b. 

2 Answers

+1 vote
For option a, lexical analyzer will see the next character after + , because it will choose the longest possible group as a lexeme. If after + no valid character is seen then only the lexical analyzer will generate token for + operator. Thus option a is not the answer.

For option b, on seeing the 'return' character sequence, lexical analyzer will generate token i.e. keyword for lexeme return only if there is no character except a delimiter, otherwise if the next character follows the pattern of identifier, then token id will be generated instead of keyword. Thus option b is no answer.

Similar explanation for option c and option d.

For option e, lexical analyzer, on seeing first + operator , will see the next character which is also + operator. After seeing the second + operator, lexical analyzer will group the ++ operator into a lexeme without seeing the next character. Because after ++ operator , there is no character which will together form a single lexeme. Thus lexical analyzer will generate token for lexeme ++(increment operator).

Why token is generated for ++ and not for +?

This is because lexical analyzer generated a token for longest sequence that can form a token.

Similar explanation for , ; ( ) ' operators in option e. Thus option e is the answer.
by Active (1.3k points)
0

@srestha mam ,

see my answer above . There i have discussed why option a,b are not correct. Option c and d have same explanation as a and b. 

if any doubt, please ask

0
In option + itself is a valid token. Is it not?
0
+ is a valid token when it is alone i.e. there is a delimiter after +. Then lexical analyzer will generate token for + only.

But here this is not the case. After + there is another + and not a delimiter. That is why lexical analyzer will generate token ++ and not for +. Here ++ together forms one lexeme and not +. But in the previous case, + was the lexeme and so lexical analyzer generated token for +.

For generating a token, always check whether the whole sequence till a delimiter is encountered matches with any pattern. If matches, then lexical analyzer will generate token for it else it will give error
0 votes
Why are you confusing yourself with hypothetical questions?

 

The longest lexeme needs to be matched by the lexer. You need to scan as much as possible, there is no question of not knowing the next character.

Also, do mention which lexical analyser you are referrin to: is it C or C++ or Java or Python or your own. If its your own, do provide all the regex you are using, only then one can answer such hypothetical question. Lets not confuse ourselves in this simple topic, please.
by Junior (551 points)

Related questions

Quick search syntax
tags tag:apple
author user:martin
title title:apple
content content:apple
exclude -tag:apple
force match +apple
views views:100
score score:10
answers answers:2
is accepted isaccepted:true
is closed isclosed:true
50,650 questions
56,236 answers
194,264 comments
95,870 users