WO2001097027A1

WO2001097027A1 - Multiphase single pass interpreter

Info

Publication number: WO2001097027A1
Application number: PCT/US2000/016458
Authority: WO
Inventors: Bruce Hodge
Original assignee: Bruce Hodge
Priority date: 2000-06-15
Filing date: 2000-06-15
Publication date: 2001-12-20
Also published as: AU2000257401A1

Abstract

A multiphase single pass interpreter (100) switches modes of execution. The interpreter scans a program file in a scan phase (204) until a tag signaling the presence of a script language is encountered (206). The interpreter then switches its mode to full parse phase (21) where each line scanned is tokenized and parsed. Tokens are replaced with their dynamic content, if any. When the interpreter detects logic blocks, the interpreter changes its mode to p-code generation phase (216) and generates p-code for the lines of code within the logic block. The generated p-code is then executed by the interpreter in its p-code execution phase (222). The interpretation is performed in a single pass through the program file.

Description

MULTIPHASE SINGLE PASS INTERPRETER

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to computer programming language interpreter and particularly, to system and method for multiphase single pass interpreter.

BACKGROUND OF THE INVENTION

An interpreter typically carries out a series of commands in a program or a script one command at a time by requesting services from an operating system. In general, script languages which are interpreted by interpreters are easier and faster to code than the more structured and compiled languages such as C. The script, however, takes longer time to execute than a compiled program because each instruction requires interpretation before the basic machine instruction processing can begin.

Particularly, most interpreters use the two pass method of interpreting. On the first pass, the existing interpreters convert the entire program or code into a simplistic micro-code that performs conditional and unconditional jumps. The micro-code is generally known as p-code. On the second pass, these prior art interpreters execute the precompiled p-code. Therefore, it is highly desirable to have an interpreter that would maintain all the advantages of the interpreter but at the same time reduce the long run-time duration that is typically associated existing prior art program interpreters. SUMMARY OF THE INVENTION

The present invention is directed to a scripting language interpreter that changes its mode of execution in a single pass as lines of code are read and interpreted. The interpreter of the present invention scans a program file while in scan phase, also referred hereinafter as a linear mode, parsing and writing out what is scanned. Also in the scan phase, if the interpreter encounters dynamic variables that need to be resolved, the interpreter resolves them with their corresponding literal content. When a predetermined symbol that signals the beginning of a script is encountered, for example, a DynaScript^® start tag, the interpreter changes its mode of execution to full parse phase. In full parse phase, the interpreter tokenizes and parses lines of code in their entirety, executing the lines of code as they are parsed. The interpreter remains in full parse phase until a script stop tag is detected. Upon detecting the script stop tag, the interpreter switches its mode back to the scan phase. While in full parse phase, the interpreter of the present invention may switch its mode to ^' a p-code generation and execution phase. The p-code generation and execution phase is activated when the interpreter detects an existence of a conditional logic block while operating in the full parse phase. In p-code generation phase, the interpreter generates lines of p-code associated with the conditional logic block. After the conditional logic block is translated into lines of p- code, the interpreter switches its mode to p-code execution phase, wherein the interpreter executes the generated p-code. The generated p-code is executed using macro instructions that include basic conditional and unconditional jumps or branching instructions. After the interpreter completes processing the conditional logic block, the interpreter switches its mode back to the full parse phase and continues its processing according to the method described hereinabove with reference to the full parse phase.

The interpreter remains in full parse phase until a script stop tag is detected. Upon detecting the script stop tag, the interpreter switches its mode back to the scan phase and continues its processing according to the method described hereinabove with reference to the scan phase . The script start and stop tags may include DynaScript^® tags. Briefly, DynaScript^® is an interpreter implementing an object type declaration syntax prefix and is described in greater detail in co- pending U.S. Patent Application Serial No. 09/583,672 (Attorney Docket No. 56129050-3) entitled OBJECT TYPE- DECLARATION PREFIX SYNTAX, filed on May 31, 2000.

The interpreter of the present invention includes a lexan or lexical analyzer that analyzes each character and generates tokens based on the characters analyzed. A parser performs semantic validation to determine if the generated tokens are semantically correct. A p-code generator generates the p-code based on the parsed tokens. Each p-code generated is inserted into a p-code execution list. The generated p-code is then executed by resetting the pointer to the beginning of the p-code execution list and executing the p-code one line at a time. A scanner scans the lines of code and replaces symbolic variables with their dynamic content before each character is analyzed by the lexical analyzer. This novel interpreting technique is significantly faster to run and uses less memory. Further features and advantages of the present invention as well as the structure and operation of various embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers indicate identical or functionally similar elements .

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 is a logic diagram illustrating the components of the interpreter of the present invention in one embodiment ;

Figure 2 is a high-level flow diagram illustrating the method of the present invention in one embodiment;

Figure 3 illustrates an example of a code listing which may be interpreted with the multiphase single pass interpreter of the present invention;

Figure 4 illustrates another example of a script and corresponding p-code that is generated by the interpreter of the present invention,-

Figure 5 illustrates an example of a symbol table created by the interpreter of the present invention;

Figure 6 illustrates a p-code execution list generated by the p-code generator of the present invention;

Figure 7 illustrates a text output that is generated by the scripting code example of Figure 3; and

Figure 8 illustrates a web page output that is generated by the scripting code example of Figure 3. DETAILED DESCRIPTION OF INVENTION

The present invention is related to co-pending U.S. Patent Application Serial No. 09/583,672 (Attorney

Docket No. 56129050-3) entitled OBJECT TYPE-DECLARATION PREFIX SYNTAX, filed on May 31, 2000, and co-pending U.S. Patent Application Serial No. 09/583,673 (Attorney Docket No. 56129050-4) entitled DYNAMIC OBJECT SYNTHESIS WITH AUTOMATIC LATE BINDING, filed on May 31, 2000, the disclosures of which are incorporated herein in their entirety by reference thereto.

Figure 1 is a block diagram illustrating the components of the interpreter of the present invention in one embodiment. The interpreter of the present invention processes the source code in a single pass by changing phases depending on the data it is processing.

It is a data driven interpreter. As shown in Figure 1, the source code is read into a source buffer 102. A scanner 118 scans a line in the source buffer 102, searching for a symbolic variable, i.e., DynaScript^® object. When a symbolic variable is found, the scanner dynamically evaluates and replaces the symbolic name with its corresponding literal content. For example, an object name "URL@Title" occurring in a line of code may be evaluated by the scanner to a title that was embedded in the URL of the web page. This information may be obtained from a symbol table 112.

The scanned line or a stream of characters 105 is passed to a lexical analyzer 106. Lexan, the lexical analyzer 106 receives a character stream 105 and generates a token stream 108. The methods by which a lexical analyzer tokenizes a character stream are known to those skilled in the art and also may be found in Writing Compiler and Interpreters by Ronald Mak, 2d Ed., John Wiley & Sons, 1996. Lexan 106 may also insert the token stream 108 into the symbol table.

Briefly, tokens represent a single character or grouping of characters that are recognized by a parser 110.

The parser 110 receives the token stream 108. From the parser 110, the token stream may be sent to the executor 116 to be executed immediately or to be generated into lines of p-code by the p-code generator/executor 114 depending on the type of the token stream received. The executor 116 of the present invention executes statements in the token stream by initiating system resource function calls associated with the statements. Examples of system resource function call include ODBC function calls. When the token stream includes an expression that may need to be evaluated, the token stream is passed to the executor 116 who builds an expression stack 122 and passes the expression stack 122 to the expression evaluator 120. The expression evaluator 120 evaluates the expression and returns the result back. The expression evaluator 120 also may insert the evaluated expressions, e.g., name/value pair into the symbol table 112. Both the executor 116 and the p-code generator/executor 114 may use the services of expression evaluator 120 to have expressions evaluated and returned.

The interpreter in one embodiment includes initialization phase, scan phase, full parse phase, p- code generation phase, p-code execution phase, and termination phase. When the interpreter starts, the interpreter is in initialization phase. In this phase, the interpreter initializes and instantiates objects used by the interpreter. Also, in this phase, the symbol table is propagated with symbolic names and their corresponding literal expressions and attributes which are known to the system at the time of initialization. As the interpreter's processing progresses, the symbol table 112 is updated with new information. For example, URL variables or name/value pairs are decoded and inserted into the symbol table .

When the initialization is complete, the interpreter enters scan phase . The interpreter remains in the scan phase until a script start tag, e.g., a DynaScript^® start token, is detected or read. In this phase, the lexan 106, in one embodiment, may only tokenize the first element in each line looking for a- script start tag, e.g., a DynaScript^® start token. Tokenizing only the first element improves performance since the rest of the line need not be analyzed. Also in this phase, all lines are scanned for DynaScript^® objects by the scanner. As described hereinabove, DynaScript^® objects, or symbolic names, are replaced with their dynamic content. All output, including the evaluated expressions is directly sent to the standard output, e.g., STDOUT stream.

When a script start tag, e.g., a DynaScript^® start token, is detected, the interpreter changes its mode to full parse phase. In this mode, the lexan 102 and the parser 110 fully process and parse each line in its entirety. The parsed commands are executed immediately unless they are wrapped in a control logic block. A control logic block includes, for example, lines within IF THEN ELSE conditional block or within loops such as the FOR, WHILE or DO loops. When control logic block is detected, the interpreter changes its mode to yet another phase, i.e., a p-code generation phase.

In p-code generation phase, the p-code generator 114 converts the lines of code that occur within a logic block into p-code. The generated p-code is stored in a p-code execution list with a corresponding micro-code index for each line of p-code generated. The micro-code index typically begins with the number zero. Branching instructions typically are padded with place holders for branching addresses until the entire conditional logic block is processed. When all the code within a logic block are generated into p-code, the interpreter then inserts a p-code end marker in the p-code execution list. The place holder are replaced with their corresponding jump indices. An example of a p-code execution list generated by the p-code generator of the present invention shown in Figure 6 and Figure 4 at 407.

Referring back to Figure 1, after generating the p- code, the interpreter changes its mode to p-code execution phase. In p-code execution phase, a micro-code index counter is set to zero for the interpreter to begin executing the p-code at the beginning of the list. The p-code is executed until a p-code end marker is encountered. In one embodiment of the present invention, the p-code is fetched by the executor 116, tokenized, parsed, and executed. In this embodiment, p- code generator/executor performs the branching instructions in the p-code and communicates to the executor 116 what commands to fetch and execute. Thus, p-code generator/executor acts as a source buffer emulator. In this way, the executor 116 need not know whether it is executing a line from the p-code execution list or a source buffer. After executing all the p-code in the p-code execution list, the interpreter changes its mode back to the full parse phase and continues processing according to the method described hereinabove. In this phase, the executor 116 fetches the character stream from the source buffer and not the p-code executor 114. When a script stop tag, e.g., DynaScript^® end token, is detected or read, the interpreter changes its mode back to the scan phase, and continues processing lines of code.

When an end of file is detected, the interpreter changes its mode into a termination phase in which the interpreter frees the allocated resources and terminates its processing. Within one file, the interpreter may switch its mode as many times as needed, according to the type of data encountered in that file. For example, to continue the example above, after having changed its mode to scan phase, the interpreter may again switch to a full parse phase if the interpreter detects another script start token.

Figure 5 illustrates an example of a symbol table created by the interpreter of the present invention. As illustrated, the symbol table includes name/value pairs as well as other information associated with a symbol.

Figure 6 illustrates a p-code execution list generated by the p-code generator 114 of the present invention.

As shown, p-code instructions include unconditional or conditional jumps to corresponding indices.

Figure 2 is a high-level flow diagram illustrating the method of the present invention in one embodiment. At step 202, the interpreter begins. At step 203, the interpreter is in initialization phase where default objects are instantiated and symbolic variables or name/value pairs, e.g., from URL, are decoded and written into the symbol table. At step 204, the interpreter is initially in scan phase. In scan phase, the scanner component of the interpreter looks for a script object, for example,

"TypeDeclaration@ObjectName" . If a script object is found, the interpreter replaces the object with its dynamic content. For instance, if an object name scanned is URL@loginid, the interpreter replaces the object name with the dynamic content of the loginid. As another example, if an object name scanned is ENV@HTTP_HOST, the interpreter replaces the object name of this environment variable, e.g., with ww . hodgepodge . com. At step 206, if the token is a script start tag, the processing continues to step 212. If the token is not a script start tag, it is tested to determine whether the token represents an end of file marker at step 208. If the token signifies end of file, the interpreter enters its termination phase 209 and terminates at step 210 after freeing all allocated memory and closing all open database connections. At step 208, if it is determined that the token does not signify an end of file, the processing continues back at step 204.

At step 206, if a script start tag token is detected signifying a start of a scripting language, e.g., DynaScript^®, the interpreter switches its mode to full parse phase. At step 212, the lexical analyzer or lexan, now in full parse phase, tokenizes the entire line scanned, executing statements immediately as each line is parsed. At step 214, if the token scanned includes a conditional logic block, the interpreter changes its mode to p-code generation phase. If no conditional logic block is detected at step 214, the interpreter determines at step 220, whether the scanned token is a script stop tag representing end of the script. If the token is a script stop tag, then the interpreter continues processing to step 208 where it is determined whether an end of file is detected. If end of file is not detected, the interpreter returns to scan phase at step 204.

At step 214, if the scanned token includes a conditional logic block, the interpreter changes its mode to p-code generation phase and at step 216 begins to generate p-code for each line within the conditional block. Each line of p-code is stored in an indexed list referred to as p-code execution list. At step 218, when a token signifying the end of the conditional block is detected, the interpreter changes its mode to p-code execution phase. If the token does not represent the end of the conditional block at step 218, the interpreter continues to generate and store at step 216, p-code corresponding to the scanned lines. At step 222, the interpreter enters the p-code execution phase and executes the p-code in the p-code execution list starting at index zero. When the interpreter completes executing the generated p-code, the interpreter returns to its full parse phase and continues its processing at step 220.

Figure 3 illustrates an example of a code listing which may be interpreted with the multiphase single pass interpreter of the present invention. After its initialization phase 301, the interpreter starts scanning a file in scan phase. For example, the interpreter scans the code at 302. When the interpreter sees the "URLOTitle" token 303 the interpreter replaces that title with the title specified in the Uniform Resource Locator (URL) of the web page being called. As described hereinabove, this information has been inserted into the symbol table 112 during the initialization phase. The replaced stream of characters having the actual title is then written out to STDOUT stream. When the interpreter detects a script start tag, e.g., <% 308, the interpreter changes its mode to full parse phase. The interpreter tokenizes and parses the statements at 304 immediately, assigning, e.-g., the token "var@cmd", with its dynamic content "select * from ClientList". As soon as the statements are tokenized and parsed, the interpreter executes the statements. When the interpreter detects a control logic block 306, e.g., IF statement, the interpreter switches its mode to p-code generation phase and execution phase, generating and executing p-code for the block 306. After processing the conditional logic block at 306, the interpreter switches its mode to full parse phase and continues processing the statements at 307 in full parse phase by having the executor execute the statements as described hereinabove with reference to full parse phase .

When the interpreter detects a stop tag, e.g., %> 310, the interpreter switches to the scan phase. The interpreter scans the lines at 312 and sends the scanned lines to a standard output file, e.g., stdout . At 314, the interpreter changes its mode to full parse phase upon detecting the start tag, e.g., <% symbol. When the interpreter scans the while loop at 316, the interpreter enters the p-code generation and execution phases, to generate and execute p-code associated with the lines of code within the while loop 316. Then the interpreter changes its mode back to the full parse phase and upon detecting the stop tag at 318, e.g., %> symbol, the interpreter changes its mode to the scan phase. The statements at 320 are scanned and when the interpreter detects the end of file tag, in this example "\html" 322, the interpreter terminates.

Figure 7 illustrates a text output that is generated by the scripting code example of Figure 3. Figure 8 illustrates a web page output that is generated by the scripting code example of Figure 3.

Figure 4 illustrates another example of a script and corresponding p-code that is generated by the interpreter of the present invention. The script including the loop logic block 402 produces the output shown at 404. The executor of the present invention fetches the instructions shown at 406 from the p-code 407 generated by the p-code generator of the present invention. The p-code 407 evaluates the "If" statement at index 0 and if true, jumps to the end. If the "If" statement at index 0 evaluates to false, the p-code in indices 1, 2 and 3 are executed. As described hereinabove, the expression evaluator of the present invention may be utilized to evaluate the IF expression. While the invention has been particularly shown and described with respect to a preferred embodiment thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

CLAIMSHaving thus described our invention, what we claim as new, and desire to secure by Letters Patent is:

1. A method of interpreting a code, comprising: scanning a stream of characters; converting the scanned stream of characters into a stream of tokens; if the stream of tokens represent a start tag of a script, scanning, parsing, and executing one or more streams of characters following the start tag; if the one or more streams of characters represent a control logic block, generating p-code associated with one or more streams of characters within the control logic block and executing the generated p-code; and repeating the scanning, parsing, and executing step and the generating p-code step until a stream of tokens representing a stop tag of the script is detected.

2. The method of interpreting a code, further including : dynamically replacing the stream of characters with associated dynamic content before the step of converting .

3. The method of interpreting a code, further including: transmitting the scanned stream of characters to a standard output stream after the step of scanning a stream of characters in a code if the start tag is not detected.

4. A multiphase single pass interpreter, comprising: a lexical analyzer to receive a character stream and to convert the character stream into a token; a parser to receive one or more of the tokens from the lexical analyzer; a p-code generator to receive the one or more of the parsed tokens to generate p-code associated with the one or more of the parsed tokens; and an executor to execute the generated p-code.

5. The multiphase single pass interpreter as claimed in claim 1, further including a scanner that scans the character stream and replaces the character stream with associated dynamic content.

6. The multiphase single pass interpreter as claimed in claim 4, wherein the executor receives parsed tokens from the parser for immediate execution.

7. A method of interpreting a code, comprising: scanning one or more characters from a code, in a scan phase; if a script start tag is scanned, switching to a full parse phase to fully scan, parse, and execute one or more instructions following the start tag; if a conditional block is scanned following the start tag, switching to p-code generation phase to generate p-code for one or more instructions within the conditional block, and switching to p-code execution phase to execute the p-code.

8. The method of interpreting a code as claimed in claim 7, further including: dynamically replacing the one or more characters with associated dynamic content.

9. The method of interpreting a code as claimed in claim 7, further including: transmitting the one or more scanned characters to a standard output stream.

10. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps of interpreting a code , comprising : scanning a stream of characters in a code; converting the scanned stream of characters into a stream of tokens; if the stream of tokens represent a start tag of a script, scanning, parsing, and executing one or more streams of characters following the start tag in the code,- if the one or more streams of characters represent a control logic block, generating p-code associated with one or more streams of characters within the control logic block and executing the generated p-code; and repeating the scanning, parsing, and executing step and the generating p-code step until a stream of tokens representing a stop tag of the script is detected.

11. The program storage device as claimed in claim 9, wherein the method steps further include: dynamically replacing the stream of characters with associated dynamic content before the step of converting.

12. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps of .interpreting a code, comprising: scanning one or more characters from a code, in a scan phase; if a script start tag is scanned, switching to a full parse phase to fully scan, parse, and execute one or more instructions following the start tag; if a conditional block is scanned following the start tag, switching to a p-code generation phase to generate p-code for one or more instructions within the conditional block, and switching to a p-code execution phase to execute the p-code.

13. The program storage device as claimed in claim 12 , wherein the method steps further include : if a script stop tag is scanned, switching back to the scan phase.