14 votes 14 votes Consider the following C program: int main (void) { in/*this is an example*/z; double/*is it an error?*/y; print( “This is simple” ); return 0; } - How many Different tokens are there in the above code. Compiler Design compiler-design compiler-tokenization lexical-analysis + – dd asked Dec 13, 2016 dd 3.4k views answer comment Share Follow See 1 comment See all 1 1 comment reply dd commented Dec 13, 2016 reply Follow Share please categorize the tokens. 0 votes 0 votes Please log in or register to add a comment.
Best answer 22 votes 22 votes int main ( void ) { in z ; double y ; print( “This is simple” ); return 0 ; } All Colored ones are different Tokens = 16 Types of tokens- keywords - int, void , double, return. Identifiers - main , in, z, y, print. Strings - “This is simple”. Constant - 0. Special- symbols / Delimiter / Punctuator / - ( , ), ; , { , } , comma. NOTE: ( 6 types of c tokens ) Kapil answered Dec 13, 2016 • edited Dec 13, 2016 by dd Kapil comment Share Follow See all 8 Comments See all 8 8 Comments reply Injila commented Apr 27, 2017 i reshown by Injila Apr 27, 2017 reply Follow Share Why remaining three semicolons are not counted? 0 votes 0 votes Injila commented Apr 27, 2017 reply Follow Share Why remaining three semicolons are not get counted? 0 votes 0 votes Kapil commented Apr 27, 2017 reply Follow Share Question says number of distinct tokens and not just (count the total number of tokens) . 5 votes 5 votes Vishal Goyal commented May 17, 2017 reply Follow Share main is a function type na why did you take it a identifier "main(" is considered as fuction i think 0 votes 0 votes vamp_vaibhav commented Dec 20, 2017 reply Follow Share I think in and z are not taken as two separate tokens after comment it would work as single one isn't?? Plz clear this doubt 0 votes 0 votes Venkat Sai commented Jan 17, 2018 reply Follow Share i have run this on a compiler it takes in and z as seperate tokens ..they will be identifiers 0 votes 0 votes `JEET commented Sep 3, 2019 reply Follow Share I am getting 17 as the answer by counting in the best answer given above. Can someone help where I am going wrong?? I counted beginning from keyword and my counts are 1. -> 4 2 -> 5 3 -> 1 4 -> 1 5 -> 6 0 votes 0 votes Mohitdas commented Jun 15, 2020 reply Follow Share the answer is right you added up wrong it's 17 you counted only 16 0 votes 0 votes Please log in or register to add a comment.
0 votes 0 votes "int","main","(","void",")"-5 tokens "{"-1 token "inz",";"-2 tokens "double","y",";"-3 tokens "printf","(","String literal",")",";"-5 tokens "return","0",";"- 3 tokens "}"- 1 token Total 20 tokens int,void-keyword "inz","y"-identifier ";"-delimeter srestha answered Dec 13, 2016 • edited Dec 13, 2016 by srestha srestha comment Share Follow See all 11 Comments See all 11 11 Comments reply Show 8 previous comments Anu007 commented Dec 20, 2017 reply Follow Share no token need not be seperated by white space like () = 2 token but no space , white space need when we define by own i.e. not known to computer 1 votes 1 votes vamp_vaibhav commented Dec 20, 2017 reply Follow Share Anu.. Yes I am getting your point but the example you are giving is two different separators which is considered to be two tokens definitely.. But what about above example.. Plz Cross check it would be very helpful.. If any reference From book or anywhere.. 0 votes 0 votes Anu007 commented Dec 20, 2017 reply Follow Share 21 token but distinct are 16 as selected answer explain. 4 votes 4 votes Please log in or register to add a comment.