Three main approaches to build a lexical analyzer

1. Write a formal description of the token patterns of the language using descriptive languages related to regular expression. These descriptions can be used as inputs to a software tool such as Yac, Lex, that automatically generate a lexical analyzer.
2. Design a state transition diagram that describes the token patterns of the language and write a program that implements the diagram.
3. Hand construct a table driven implementation of the state diagram.