[Back to TEXTFILE SWAG index] [Back to Main SWAG index] [Original]

    {     Right now I'm writing an interpreter For a language that I
    developed, called "Isaac".  (It's Physics oriented).  I'd be very
    interested in you publishing this inFormation regarding Pascal
    Compilers, though I would likely not have time to do the excercises
    right away.

       Ok, Gavin. I'll post the lister (not Really anything exceptional,
       but it'll get this thing going in Case anyone joins in late.)

       Here's the lister Program:
    }
    {$I-}
    Program Lister;

    Uses Dos;

    {$I PTypeS.inC}
    {Loacted in the SOURCE\MISC Directory.}

    Function LeadingZero(w:Word): String;{convert Word to String With 0's}
       Var s :String;
       begin
          Str(w:0,s);
          if Length(s) < 2 then s := '0'+s;
          LeadingZero := s;
          if Length(s) > 2 then Delete(s,1,Length(s)-2);
       end;

    Function FormatDate :String; { get system date and pretty it up }
       Const
          months : Array[1..12] of String[9] =
          ('January', 'February', 'March', 'April', 'May', 'June', 'July',
           'August', 'September', 'October', 'November', 'December');
       Var s1,fn : String; y,m,d,dow : Word;
       begin
          GetDate(y,m,d,dow);
          s1 := leadingZero(y);
          fn := LeadingZero(d);
          s1 := fn+' '+s1;
          fn := months[m];
          s1 := fn+' '+s1;
          FormatDate := s1;
       end;

    Function FormatTime :String; { get system time and pretty it up }
       Var s1, fn : String; h,m,s,s100 : Word;
       begin
          GetTime(h,m,s,s100);
          fn := LeadingZero(h);
          s1 := fn+':';
          fn := LeadingZero(m);
          FormatTime := s1+fn;
       end;

    Procedure Init(name:String);
       Var t,d        :String;
       begin
          line_num := 0; page_num := 0; level := 0;
          line_count := MAX_LinES_PER_PAGE;
          source_name := name;
          Assign(F1, name);      { open sourceFile - terminate if error }
          Reset(F1);
          if Ioresult>0 then
          begin
             Writeln('File error!');
             Halt(1);
          end;
          { set date/time String }
          d := FormatDate;
          t := FormatTime;
          date := d+'  '+t;
       end;

    Procedure Print_Header;
       Var s, s1 :String;
       begin
          Writeln(F_FEED);
          Inc(page_num);
          Str(page_num, s1);
          s := 'Page '+s1+'   '+source_name+'  '+date;
          Writeln(s);
       end;

    Procedure PrintLine(line :String);
       begin
          Inc(line_count);
          if line_count>MAX_LinES_PER_PAGE then
          begin
             print_header;
             line_count := 1;
          end;
          if ord(line[0])>MAX_PRinTLinE_LEN then
             line[0] := Chr(MAX_PRinTLinE_LEN);
          Writeln(line);
       end;

    Function GetSourceLine :Boolean;
       Var print_buffer :String[MAX_SOURCELinE_LEN+9];
           s            :String;
       begin
          if not(Eof(F1)) then begin
             Readln(F1, source_buffer);
             Inc(line_num);
             Str(line_num:4, s);
             print_buffer := s+' ';
             Str(level, s);
             print_buffer := print_buffer+s+': '+source_buffer;
             PrintLine(print_buffer);
             GetSourceLine := True;
          end else GetSourceLine := False;
       end;

    begin  { main }
       if ParamCount=0 then begin
          Writeln('Syntax: LISTER <Filename>');
          Halt(2);
       end;
       init(ParamStr(1));
       While GetSourceLine do;
    end.

    {
       Now that the task of producing a source listing is taken care of,
       we can tackle the scanners main business: scanning. Our next job
       is to produce a scanner that, With minor changes, will serve us
       For the rest of this "course".

       The SCANNER will do the following tasks:

       ø scan Words, numbers, Strings and special Characters.
       ø determine the value of a number.
       ø recognize RESERVED WordS.

       LOOKinG For toKENS

       SCANNinG is reading the sourceFile and breaking up the Text of a
       Program into it's language Components; such as Words, numbers,
       and special symbols. These Components are called toKENS.

       You want to extract each each token, in turn, from the source
       buffer and place it's Characters into an empty Array, eg.
       token_String.

       At the start of a Word token, you fetch it's first Character and
       each subsequent Character from the source buffer, appending each
       Character to the contents of token_String. As soon as you fetch a
       Character that is not a LETTER, you stop. All the letters in
       token_String make up the Word token.

       Similarly, at the start of a NUMBER token, you fetch the first
       digit and each subsequent digit from the source buffer. You
       append each digit to the contents of token_String. As soon as you
       fetch a Character that is not a DIGIT, you stop. All digits
       within token_String make up the number token.

       Once you are done extracting a token, you have the first
       Character after a token. This Character tells you that you have
       finished extracting the token. if the Character is blank, you
       skip it and any subsequent blanks Until you are again looking at
       a nonblank Character. This Character is the start of the next
       token.

       You extract the next token in the same way you extracted the
       previous one. This process continues Until all the tokens have
       been extracted from the source buffer. Between extracting tokens,
       you must reset token_String to null String to prepare it For the
       next token.

       PASCAL toKENS

       A scanner For a pascal Compiler must, of course, recognize Pascal
       tokens. The Pascal language contains several Types of tokens:
       identifiers, reserved Words, numbers, Strings, and special
       symbols.

       This next exercise is a toKENIZER that recognizes a limited
       subset of Pascal tokens. The Program will read a source File and
       list all the tokens it finds. This first version will recognize
       only Words, numbers, and the Pascal "end-of-File" period - but it
       provides the foundation upon which we will build a full Pascal
       scanner in the second version.

       Word: A Pascal Word is made up of a LETTER followed by any number
       of LETTERS and DIGITS (including 0).

       NUMBER: For now, we'll restrict a number token to a Pascal
       unsigned Integer, which is one or more consecutive digits. (We'll
       handle signs, decimals, fractions, and exponents later) and,
       we'll use the rule that an input File *must* have a period as
       it's last token.

       The tokenizer will print it's output in the source listing.

       EXERCISE #2

       Use the following TypeS and ConstANTS to create a SCANNER as
       described above:

    -------------------------------------------------------------------

    Type
       Char_code    = (LETTER, DIGIT, SPECIAL, Eof_CODE);
       token_code   = (NO_toKEN, Word, NUMBER, PERIOD,
                       end_of_File, ERRor);
       symb_Strings :Array[token_code] of String[13] =
                      ('<no token>','<Word>','<NUMBER>','<PERIOD>',
                       '<end of File>','<ERRor>');

       literal_Type = (Integer_LIT, String_LIT);

       litrec = Record
          l :LITERAL_Type;
          Case l of

             Integer_LIT: value :Integer;
             String_LIT:  value :String;
          end;
       end;

    Const
       Eof_Char = #$7F;

    Var
       ch             :Char;        {current input Char}
       token          :token_code;  {code of current token}
       literal        :litrec;      {value of current literal}
       digit_count    :Integer;     {number of digits in number}
       count_error    :Boolean;     {too many digits in number?}
       Char_table     :Array[0..255] of Char_code;{ascii Character map}

    The following code initializes the Character map table:

    For c := 0 to 255 do
       Char_table[c] := SPECIAL;
    For c := ord('0') to ord('9') do
       Char_table[c] := DIGIT;
    For c := ord('A') to ord('Z') do
       Char_table[c] := LETTER;
    For c:= ord('a') ro ord('z') do
       Char_table[c] := LETTER;
    Char_table[ord(Eof_Char)] := Eof_CODE;

    -------------------------------------------------------------------

       You can (and should) use the code from your source listing
       Program to start your scanner. if you have just arrived, use my
       own code posted just previously.

[Back to TEXTFILE SWAG index] [Back to Main SWAG index] [Original]