Introduction

What my KEPLOG program does is process the Access Log that the APACHE server kicks out. The report that is output will tell you far more than what you'd ever want to know about your server's activity. Here is the data that the report will give:
  • It will tell total data traffic for the time period.
  • It will tell worst hour of data traffic during time period.
  • It will list all files that got hit.
  • It will tell how large each file is.
  • It will tell the percentage of your data traffic each file took up.
  • It will list all refers (links) to each file as they are listed.
  • It will list everyone who hit each file and how many times they did.
  • It will give a TOP TEN LIST of worst bandwidth hogs (Files that took up highest percentage of traffic).
  • It will list all BANDWIDTH BANDITS (People linked to your GIFs and JPGs).
  • It will then list all refers (links) to all your files.
  • Multiple reports can then be split out of base reports for sending to webpage customers.
  • Heavily hacked up example report -> (click me)

    • How Does It Work

    My program is written in Visual Basic in a pretty 16-bit version, or a fast 32-bit version. Line by line it will read the Access Log and just tally everything up. When it reaches the end, or certain limits of memory, it will kick out a report. These reports can then be merged together if you wish, and then customer reports can be split out of it. All of this can be automated into just one click of its icon. This program was optimized to work on my wife's pages, which are relatively active, hovering about 15 megs of logs to process each week.

    APACHE Setup

    In order for KEPLOG to understand your server's Access Log, it has to be in a defined format. Somewhere in your HTTPD.CONF file you must put in the following line (I put it in just after my TransferLog statement):

    Logformat "%h %l %u %t \"%r\" %s %b \"%{Referer}i \""

     

    Running KEPLOG

    Kepper Jack's APACHE Logger page 

     

    Creating Reports

    After you fire up my program, the first button at the bottom left is what you press to create some reports. Upon clicking, a popup box will ask you to enter the name of the file. Yes, you have to type it. No, there is no browse feature. Yes, you really are limited to the 8 character name and 3 character extension that the files REALLY are named. When I transfer over MY Access Log from my linux machine, I name it 'access.old' and put it in my 'linux' directory on drive 'e'. I therefore have to type in 'e:/linux/access.old'.

    The first report will be named with your 8 char filename along with an extension of just '1'. This extension will incremented if more reports are created from your logs.

    Searching Logs

    This function allows you to search for a string within a log. All occurrences of that string will cause that line of the logs to be listed until 100 are found. Then you will be asked if you wish to continue the search. Frankly, it would probably be more useful to you if you just brought the log into your favorite editor and did a search from there.

    Merging Reports

    So you have an ACCESS.1 and an ACCESS.2 that you want to merge together. No problem. All data will be combined and recalculated. You just have to type in the names of the two Reports that you want to be merged. The end result will be an ACCESS.MRG (ie. '.mrg' will be appended to your filename).

    Split Reports

    Do you have users/clients on your system? You can split off reports for them that will show the basics: filenames, sizes, Usage, Refers, and Hits. It will not split off the Top Ten list or the Bandwidth Bandits or the big Refers list at the end. In order to do this, however, you need to have a configuration file. You can name it anything you wish, but I'm trying to standardize it to a '.spt' extention. Inside this file you keep all the keywords for all the reports you want to be created. Each line specifies a seperate report, and must have the following data in comma delimited format:
  • All Data must be enclosed in double quotes (Its a VB thing).
  • The First field is the drive:path/filename that you want the report to be called.
  • The Second field is a Title that you want at the top of the report.
  • All fields after that are key strings.
    • What will happen is that KEPLOG will pass through all the files that are listed in the base report, searching for the key strings (ie. '/mike/') and if it is found will append that file into the appropriate report (You may want to defrag your disk more often). You can also use exclude strings to stop a file from entering a report. An exclude string is determined by a '-' as the first character (KEPLOG won't try to match it, of course).

    Here is an example of a .spt configuration file.
    "e:\linux\logs\dave.rpt,Daves Files,/dave/"
    "e:\linux\logs\darkwing.rpt,Darkwings Files,/dark/,-/dark/closet/"

    The AUTO Function

    So now you want to process your logs by just hitting the icon. Well fine. You can do that by using execution parameters. For your KEPLOG icon, choose properties. After the program name (keplog.exe) you need to put some parameters, seperated by spaces. The first parameter must be the word 'AUTO'. The second parameter must be the drive:path/filename of the logs that you want to process. The third paramter is optional, but can be the drive:path/filename of your split reports configuration file. KEPLOG will pass through the logs, merging all the reports together as it goes through them. The final report that you want will be the '.mrg' that you find.

    Installing KEPLOG

    I have no installation wizard, but this is a simple program with no tentacles that it needs to throw anywhere. Just create a directory for it anywhere on your system, and unzip my file into it. Then create a shortcut to it however you like. You will now have to configure the 'keplogx.dat' file...

    The keplogx.dat file has several purposes. Basically is has exclude and include strings in it that control how the base reports are created. At the top of my default file is a few remarks that explain each type of record. The type of the record is always the first character of the record, which is following by the key string.

    The '0' type records are all the users that you do NOT want listed as valid hits onto your files. Since my wife and I have our PCs on a LAN with dedicated IP addresses, we have included our IP numbers as '0' records. We do not want us listed as people who hit our files. That would be silly.

    The '1' type records are Refers that we know are hitting our .JPG and .GIF, but we don't want listed as bandits. My wife has friends that are using our server to store pictures, so we don't want to list their pages as Bandits.

    The '2' records are very important. They are the refers that we absolutely do not want counted or listed anywhere in our report. Our server is jade-leaves.com, and we have many pages on our server that point to each other. And we don't want to list ourselves as Referers, do we? Of course, not.
    Here is an example of our keplogx.dat, IP numbers masked, of course.
    * The first Character is the Type:
    * 0=Users to be excluded from all logging
    * 1=Refers to be Logged, but excluded from Bandit Suspects
    * 2=Refers to be excluded from all logging
    *
    0123.456.78.90
    0123.456.78.91
    1bugnet.com
    1zipzap.net
    1bughost/zip
    2jade-leaves
    2JADE-LEAVES

     

    Click here for my FAQ Page

     

    bar 

    Have you read through all this and still are interested? Then just click below and download the zip file. Send me some email if you are having any trouble, or even have some enhancements (no promises). Did I mention that this is Freeware?
  • v3.2 updated 9/1/99

  • v5.0 updated 10/12/99

  • v5.1 updated 01/24/00 User overflow bug fixed
    		•

     

    Download the file (keplog.zip v3.2)

     

    Download the file (keplog32.zip v5.0)

     

    bar 

    SOFTWARE LICENSE AGREEMENT

    This program is FREEWARE. It is free for PERSONAL USE and distribution, no Registration is required. It cannot be used for commercial or promotional purposes. This program must be shipped in original package (zip file). It is allowed to bundle this package as a part of other packages.

     

    LIMITED WARRANTY

    For direct or indirect damage, caused by using the program, the author is not liable. The author gives no warranty for full, correct or partly functionality of the program. You are using this program on your own risk. The author also gives no warranty, that the program is free of any virus. The user has to check it.

    bar 

    mailboxSend me some email.  

    To My Skyforge