# Recent questions tagged compiler-tokenization

1
The above diagram is Transition Diagrams for identifiers. As we can see that the identifier is said to be accepted if it starts with a letter and ends with a valid delimiter, which includes blank symbol, arithmetic, logical operator, left parenthesis, right parenthesis, +, ... ends with a delimiter and + is a valid delimiter and the error in declaration will not be detected at this stage...
1 vote
2
which one of the following string can definitely said to be a token without looking at the next input: +(++,+=) return(return a) *(*=) =(==) ++ , ( ) ‘ ; option e is: plusplus, comma, bracket open, bracket close, single quote ;
3
Q1: Consider G :- $X \rightarrow X + Y | Y$ $Y \rightarrow Y*Z|Z$ $Z \rightarrow (X)$ $Z \rightarrow id$ if LR(1) parser is used to parse the above grammar, then total how many lookahead will be present in the dfa state I0 for the items $X \rightarrow .Y \text{ and } Z \rightarrow .id$. My Answer: ... --------------------------------------------------------------- Someone verify all these answers.
4
1 vote
5
How will the compiler detect whether "if" is a keyword or an identifier? Please tell me the concept behind this.
6
The number of tokens in the fortran statements DO 10 I = 1.3 is ?
1 vote
7
Find no of tokens in below program. #include<stdio.h> main() { int I; int *pi = &I; //parent pointer scanf("%d",pi); printf("%d\n", I+5); }
1 vote
8
Whether a given pattern constitutes a token or not (a) depends on the source language (b) depends on the target language (c) depends on the compiler (d) none of the above comment is true
1 vote
9
void main() { int total-value, Num=2,sum=5,var1; 5=Num; var1=2; sum=num*1; if(sum=var1) { sum=sum+1 } } Here, is $\text{"total-value"}$ a lexical error?
10
If I have the grammar : E->E*F | F+E |F F->id , Now here although + and * are defined at the same level , with different associativity but this grammar produces 2 different parse trees for the string id+id*id , so here we can't make any generalization regarding precedence of the operators although they are defined at the same level . So what conclusion can be drawn from this ?
11
Number of tokens in $\text{int a[5];}$
1 vote
12
# DEFINE M 100 int main() { // declare variables int n = 2020; return n%M; } The number of tokens in this program?
13
How many tokens are there? 1) int a /*b 2) printf ("%
14
What are different set of characters which are taken as single token in lexical analysis in compiler design? For eg. Charaters under double quotes are taken as single token, post-increment and pre-increment is taken as single token etc.
1 vote
15
Which of the following not a token of C program? a)1.02e+2 b)#define c)MAX d)123.33
16
if there is miss spelling in some keyword in a program then this misspelled keyword will be treated as lexical errors or it will be treated as a new identifier and accepted as a token ?? ex - whiel (1) ; here while is misspelled as whiel
17
int main() { print("%d",a); return 0; } If i execute,i will get error as:- prog.c:112:17: error: 'a' undeclared (first use in this function) printf("%d",a); Now as this is the semantic error,so how does semantic analyzer knows the row and ... I know lexical analyzer can give the row number and column number of error as it tokenize the entire program but is this also possible in sematic analyzer?
18
How many tokens are there in the following statement: x >> = 2; In general, are shorthand operators counted as a single token as ">>=" or double as ">>" and "=". Or anything else?
19
This screenshot is token from the book Ullman, How can following be a lexical error? because "elipseSize" should have a token recorded as an identifier.
20
I am writing down some errors and its type with a reason.If there something wrong please correct it with a valid reason. 1.fi(a==10) this is not a lexical error, because lexical analyzer cannot tell whether fi is a misspelling of the keyword if or an undeclared function ... is not a valid token. but this says it is streams of valid tokens 6.int d=10.10.10 lexical error as invalid num token.
1 vote
21
What will be the tokens formed for ',' and '{' ?
22
Find the type of error produced by the following C code. main() { in/*comment t x; floa/*comment*/t gate; } Lexical error syntax error both a) and b) None of these
23
24
Consider the following C program: int main (void) { in/*this is an example*/z; double/*is it an error?*/y; print( “This is simple” ); return 0; } - How many Different tokens are there in the above code.
25
Consider the following C program: How many different tokens are there for the same program
26
Given explanation Couldn't understand the explanation. Please explain
27
I think answer should be 13. We got 13 tokens printf ( "string" ++ & & * * * a ) ;