Log In
3 votes
How are constants recogonized by the lexical analyser like

interger 1234     floating point  1234.56789

and what happens when we have -1234  and -1234.56789 how many tokens are generated in this case(each of this case) ?

is "-" considered seperately ?
in Compiler Design 190 views
"-" is considered as a separate token, it is an unary operator. Rest of the constant is taken as a single token, i.e. -1234.56789 are two token: "-" and "1234.56789".
but - is not unary  --  and -= are unary right ?
but i too think it will generate as it is valid operator
If "-" is a separate token in "-2.5" then we need not have a representation for negative numbers :)

1 Answer

1 vote
For generating the number of tokens, following are considered as one:

1.) Keywords: if, else, while, do, for, float,..,etc.

2.) Identifiers: letter followed by letter or underscore or digit.

3.) Constant(consider any real number): 4,10,5.5

4.) Punctuation Symbols: :, ;, , , {, }, [, ], ?

5.) Operators: Relational Operator-> < , >, <=, >=, ==, etc

                       Logical Operator-> &&, ||, !

                       Bitwise Operatore-> &, |, ^(Ex-OR), <<, >>

6.) Few more: +, ++, -, --,/,*,+=,-=,=,etc

Now according to your question integer 1234 will be considered as ONE TOKEN (refer constant discussed above)

and , -1234.56789 will be considered as two token( first for - sign and  second for 1234.56789)

Hope, this clears your doubt and help in understanding of the concept.

Related questions

2 votes
1 answer
The above diagram is Transition Diagrams for identifiers. As we can see that the identifier is said to be accepted if it starts with a letter and ends with a valid delimiter, which includes blank symbol, arithmetic, logical operator, left parenthesis, right parenthesis, +, ... ends with a delimiter and + is a valid delimiter and the error in declaration will not be detected at this stage...
asked Jun 9, 2019 in Compiler Design Hirak 403 views
0 votes
4 answers
How many tokens in this a>>=1; and a! , Will >>= and a! be treated as a single token ?
asked Jan 21, 2019 in Compiler Design kman30 366 views
1 vote
0 answers
Once the parser makes the list of legitimate tokens available to the error recovery routine then this routine can decide whether a remaining input’s prefix matches one of these tokens closely enough to be treated as that token. This helps the lexical analyzer.
asked Oct 6, 2017 in Compiler Design Venkat Sai 167 views
2 votes
1 answer
I came across few Compiler Design Doubts, Please provide your cent. $Q_1 :$ Can lexical analyser detect some/any type of errors? I think yes, because while scanning to identify tokens, it may be the cases that a string pattern doesn't match with any keyword (or) entry in ... (Left linear) $\color{navy}{A \rightarrow aA}$(Right Linear). What can be said about $\color{navy}{A \rightarrow aAb}$
asked Nov 1, 2016 in Compiler Design mcjoshi 464 views