Project Idea: LogCompanion

LogCompanion is a tool for analyzing log-files while developing and debugging software.
Logging as a technique for debugging and error analysis is heavily used in software industry. In my experience, many companies use logging as the only debugging techniques and neglect others (e.g. such exotic one as using debugger). Whether sensible or not, the fact is, that plenty of time is spent on log-file analysis. That includes finding/(re-)opening/ log-files, looking for error (e.g. ERROR:), analyzing what happened before, analyzing what happen, if error doesn’t occur, trying to compare logs, removing timestamps in order to be able to diff files, filtering out noisy lines, removing code-line information in order to be able diff files, getting an idea why error may have happened, suspecting race-condition, looking again for corresponding timestamps, looking again for corresponding code-line information, trying to identify the place in code, copying/renaming usefull log-files/sessions, and so on.

Imagine, a software would save the log-files you’ve analyzed last time, and monitor them for changes or even for error-tags. It would recognize automatically the tags in log-lines such as file-names, lines, timestamps, error-levels, messages, etc. It would use the information sensible depending on context e.g ignoring timestamps while comparing sequences of commands, opening file on specified line in IDE, showing time-spans between lines, displaying error-level as an icon, etc. Imagine, it would even detect unusual behavior e.g prohibited sequences, time-outs, invalid values etc. How much faster would you be able to develop your software then? Much faster! LogCompanion will help you in that.

I am currently working on first prototype and I hope to publish the code soon. The prototype is developed in Qt4, of cause . While the prototype is evolving, I am still collecting requirements and feature ideas. Therefore this site was created. If you have a question, an idea or a suggestions, I would appreciate your message. If you are aware of such a software, free or commercial for reasonable prize, I would appreciate, If you gave me a hint – reinventing a wheel is always a pain.

Here are the key features I am aiming to implement:

• easy definition of log format
• auto-recognition of known log formats:
  • log4cxx
  • log4cpp
  • log4j
  • log4net
  • some proprietary formats
• Automatic Error-detection in using logging APIs (extra/missing newlines etc)
• auto-recognition of log data
  • timestamps
  • file-names and line numbers
  • URIs
• outstanding filtering abilities
• comparing logs using defined filter criteria
• extensive usage of intuitive GUI-techniques
  • Drag&Drop
  • Semitransparent dialogs
  • actions on mouse over
  • gestures
• syncing several log-files using time-stamp
• configurable presentation of log entries
  • Icons
  • recognition of URIs
  • replacement of integer values by symbols
  • summing up same/similar consecutive log lines
• preview of log-data items
  • Mimes
  • Code-Chunks
• conversion to all known formats
• auto-recognition of known formats
• pre-defined configurations for all known formats
• conversion to XML
• HTML report generation
• “tail -f” replacement
• real time log transformation and forwarding (to file, network, etc.)
• log data input from network connection (e.g. piping via nc)
• log joining
• log collection
• input from several network connections
• “tailing -f” several files
• code generation for input/parsing plug-ins based on format description
• unusual sequences detection
• repetition detection
• block comparison
  • N lines beginning with specified line
  • All lines between specified lines
• Log-rotate with compression
• alarming
  • sound
  • email
• Status information
  • last update, a warning sign, if file older than specified time
• Replay of cuptured events.
• Splitting Log into several pre-defined Log-Classes. e.g. Errors, Messages of a particular developer, Strange behaviour
• Acting as multi-client proxy server for TCP/UDP protocolls and capturing data passed through.