Apache JMeter

org.apache.jmeter.protocol.http.util.accesslog
Class TCLogParser

java.lang.Object
  extended by org.apache.jmeter.protocol.http.util.accesslog.TCLogParser
All Implemented Interfaces:
LogParser
Direct Known Subclasses:
SharedTCLogParser

public class TCLogParser
extends Object
implements LogParser

Description:

Currently the parser only handles GET/POST requests. It's easy enough to add support for other request methods by changing checkMethod. The is a complete rewrite of a tool I wrote for myself earlier. The older algorithm was basic and did not provide the same level of flexibility I want, so I wrote a new one using a totally new algorithm. This implementation reads one line at a time using BufferedReader. When it gets to the end of the file and the sampler needs to get more requests, the parser will re-initialize the BufferedReader. The implementation uses StringTokenizer to create tokens.

The parse algorithm is the following:

  1. cleans the entry by looking for backslash "\"
  2. looks to see if GET or POST is in the line
  3. tokenizes using quotes "
  4. finds the token with the request method
  5. gets the string of the token and tokenizes it using space
  6. finds the first token beginning with slash character
  7. tokenizes the string using question mark "?"
  8. get the path from the first token
  9. returns the second token and checks it for parameters
  10. tokenizes the string using ampersand "&"
  11. parses each token to name/value pairs

Extending this class is fairly simple. Most access logs use the same format starting from the request method. Therefore, changing the implementation of cleanURL(string) method should be sufficient to support new log formats. Tomcat uses common log format, so any webserver that uses the format should work with this parser. Servers that are known to use non standard formats are IIS and Netscape.


Field Summary
protected  boolean decode
          by default, we probably should decode the parameter values
protected  String FILENAME
           
protected  Filter FILTER
          Handles to supporting classes
static String GET
           
static String HEAD
           
protected static org.apache.log.Logger log
           
static String POST
           
protected  BufferedReader READER
           
protected  String RMETHOD
          protected members *
protected  File SOURCE
           
protected  String URL_PATH
          The path to the access log file
protected  boolean useFILE
           
 
Constructor Summary
TCLogParser()
           
TCLogParser(String source)
           
 
Method Summary
 boolean checkMethod(String text)
          The method checks for POST and GET methods currently.
 boolean checkParamFormat(String text)
          Checks the string to see if it contains "&" and "=".
 boolean checkURL(String url)
          Checks the string to make sure it has /path/file?
 String cleanURL(String entry)
          The method cleans the URL using the following algorithm.
 void close()
          close the any streams or readers.
 void convertStringToJMRequest(String text, TestElement el)
          Convert a single line into XML
 NVPair[] convertStringtoNVPair(String stringparams)
          Parse the string parameters into NVPair[] array.
 boolean decodeParameterValue()
          decode the parameter values is to true by default
protected  int parse(BufferedReader breader, TestElement el, int parseCount)
          The method is responsible for reading each line, and breaking out of the while loop if a set number of lines is given.
 int parse(TestElement el, int parseCount)
          parse the entire file.
 int parseAndConfigure(int count, TestElement el)
          parse a set number of lines from the access log.
protected  int parseLine(String line, TestElement el)
          parseLine calls the other parse methods to parse the given text.
protected  NVPair parseOneParameter(String parameter)
          Method expects name and value to be separated by an equal sign "=".
protected  List<String> parseParameters(String parameters)
          Method uses StringTokenizer to convert the string into single pairs.
 void setDecodeParameterValues(boolean decodeparams)
          by default decode is set to true. if the parameters shouldn't be decoded, call the method with false
 void setFilter(Filter filter)
          Use the filter to include/exclude files in the access logs.
 void setSourceFile(String source)
          Sets the source file.
 void setUseParsedFile(boolean file)
          Calls this method to set whether or not to use the path in the log.
 String stripFile(String url, TestElement el)
          Tokenize the URL into two tokens.
 StringTokenizer tokenize(String line, String delim)
          Parses the line using java.util.StringTokenizer.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

log

protected static final org.apache.log.Logger log

GET

public static final String GET
See Also:
Constant Field Values

POST

public static final String POST
See Also:
Constant Field Values

HEAD

public static final String HEAD
See Also:
Constant Field Values

RMETHOD

protected String RMETHOD
protected members *


URL_PATH

protected String URL_PATH
The path to the access log file


useFILE

protected boolean useFILE

SOURCE

protected File SOURCE

FILENAME

protected String FILENAME

READER

protected BufferedReader READER

FILTER

protected Filter FILTER
Handles to supporting classes


decode

protected boolean decode
by default, we probably should decode the parameter values

Constructor Detail

TCLogParser

public TCLogParser()

TCLogParser

public TCLogParser(String source)
Parameters:
source -
Method Detail

setDecodeParameterValues

public void setDecodeParameterValues(boolean decodeparams)
by default decode is set to true. if the parameters shouldn't be decoded, call the method with false

Parameters:
decodeparams -

decodeParameterValue

public boolean decodeParameterValue()
decode the parameter values is to true by default

Returns:
if paramter values should be decoded

setUseParsedFile

public void setUseParsedFile(boolean file)
Calls this method to set whether or not to use the path in the log. We may want to provide the ability to filter the log file later on. By default, the parser uses the file in the log.

Parameters:
file -

setFilter

public void setFilter(Filter filter)
Use the filter to include/exclude files in the access logs. This is provided as a convienance and reduce the need to spend hours cleaning up log files.

Specified by:
setFilter in interface LogParser
Parameters:
filter -

setSourceFile

public void setSourceFile(String source)
Sets the source file.

Specified by:
setSourceFile in interface LogParser
Parameters:
source -

parse

public int parse(TestElement el,
                 int parseCount)
parse the entire file.

Returns:
boolean success/failure

parseAndConfigure

public int parseAndConfigure(int count,
                             TestElement el)
parse a set number of lines from the access log. Keep in mind the number of lines parsed will depend the filter and number of lines in the log. The method returns the actual number of lines parsed.

Specified by:
parseAndConfigure in interface LogParser
Parameters:
count -
Returns:
lines parsed

parse

protected int parse(BufferedReader breader,
                    TestElement el,
                    int parseCount)
The method is responsible for reading each line, and breaking out of the while loop if a set number of lines is given.

Parameters:
breader -

parseLine

protected int parseLine(String line,
                        TestElement el)
parseLine calls the other parse methods to parse the given text.

Parameters:
line -

cleanURL

public String cleanURL(String entry)
The method cleans the URL using the following algorithm.
  1. check for double quotes
  2. check the request method
  3. tokenize using double quotes
  4. find first token containing request method
  5. tokenize string using space
  6. find first token that begins with "/"
Example Tomcat log entry:

127.0.0.1 - - [08/Jan/2003:07:03:54 -0500] "GET /addrbook/ HTTP/1.1" 200 1981

Parameters:
entry -
Returns:
cleaned url

checkMethod

public boolean checkMethod(String text)
The method checks for POST and GET methods currently. The other methods aren't supported yet.

Parameters:
text -
Returns:
if method is supported

stripFile

public String stripFile(String url,
                        TestElement el)
Tokenize the URL into two tokens. If the URL has more than one "?", the parse may fail. Only the first two tokens are used. The first token is automatically parsed and set at URL_PATH.

Parameters:
url -
Returns:
String parameters

checkURL

public boolean checkURL(String url)
Checks the string to make sure it has /path/file?name=value format. If the string doesn't have "?", it will return false.

Parameters:
url -
Returns:
boolean

checkParamFormat

public boolean checkParamFormat(String text)
Checks the string to see if it contains "&" and "=". If it does, return true, so that it can be parsed.

Parameters:
text -
Returns:
boolean

convertStringToJMRequest

public void convertStringToJMRequest(String text,
                                     TestElement el)
Convert a single line into XML

Parameters:
text -

convertStringtoNVPair

public NVPair[] convertStringtoNVPair(String stringparams)
Parse the string parameters into NVPair[] array. Once they are parsed, it is returned. The method uses parseOneParameter(string) to convert each pair.

Parameters:
stringparams -

parseOneParameter

protected NVPair parseOneParameter(String parameter)
Method expects name and value to be separated by an equal sign "=". The method uses StringTokenizer to make a NVPair object. If there happens to be more than one "=" sign, the others are ignored. The chance of a string containing more than one is unlikely and would not conform to HTTP spec. I should double check the protocol spec to make sure this is accurate.

Parameters:
parameter - to be parsed
Returns:
NVPair

parseParameters

protected List<String> parseParameters(String parameters)
Method uses StringTokenizer to convert the string into single pairs. The string should conform to HTTP protocol spec, which means the name/value pairs are separated by the ampersand symbol "&". Some one could write the querystrings by hand, but that would be round about and go against the purpose of this utility.

Parameters:
parameters -
Returns:
List

tokenize

public StringTokenizer tokenize(String line,
                                String delim)
Parses the line using java.util.StringTokenizer.

Parameters:
line - line to be parsed
delim - delimiter
Returns:
StringTokenizer

close

public void close()
Description copied from interface: LogParser
close the any streams or readers.

Specified by:
close in interface LogParser

Apache JMeter

Copyright © 1998-2014 Apache Software Foundation. All Rights Reserved.