Projects

X Ray: Coding Analysis for SCMs

XRay is an analysis tool built upon your SCMs, aimed to consolidade project statistics from many repositories in a single tool, generating code reports and development statistics obtained from log data and file versions.

CollabNet Apache Stats

This project provides some tools that enable you to explore traffic patterns on your CollabNet (CEE or CTF) site, in a deeper way, more aware of the structure of the site itself, than with generic analysis tools such as Wusage and AWStats.

squidlook

Warning: squidLook is currently under developement. SVN code is not yet usable for production use.squidLook is a fork of Mysar (see the link at the right). The main modifications are: Optimized importer, at least 40% faster than the original Grouping of users to aggregate statistics Site ... [More] white-listing to remove trusted websites from statistics in real time Preliminary i18n internationalizing support Yearly, monthly and weekly statistics for all pages Software requirements: Linux system (other systems can be suported) PHP 4 or higher Mysql 4.1 or higher Copyright (c) 2007-2008, Luca Manganelli - Comune di Trento [Less]

mysql-slow-query-log-parser

The MySQL slow query log parser converts your slow query logs in to a more usable format. It also adds some interesting information like median and average times. Here is an example of what the parsed log looks like 1 Queries Taking 4 seconds to complete Locking for 0 seconds Average time: 4 ... [More] , Median time 4 Average lock: 0, Median lock 0 DELETE FROM blah WHERE blah1 >= XXX AND blah2<= XXX; ################################################################################ 22 Queries Taking 3 3 seconds to complete Locking for 0 0 seconds Average time: 3, Median time 3 Average lock: 0, Median lock 0 select * from table1 WHERE table1.something = table.something and table1.x = XXX; ################################################################################This parser was inspired by the perl mysql_slow_log_parser written by Nathanial Hendler. [Less]

mysql-log-filter

PurposeThe script filters the MySQL Slow Query Log to show queries which impacted performance most and is intended to be used by DB admins and application developers. The log file is analyzed and processed as a stream, line after line, so there is no need to load the whole log file into memory. ... [More] --no-duplicates is a very useful option to see only necessary statistics. --incremental remembers last input file positions and statistics in a SQLite 3 database, so periodical executions on the same files run much faster. The Python version is usually 3-5 times faster than the PHP5 version. You are welcome to contribute a version translated into your favorite scripting language, just open a feature request under Issues i.e. to add a Perl version. Usage Examples# Filter slow queries executed for at least 3 seconds not from root, remove duplicates, # apply execution count as first sorting value and save first 10 unique queries to file. # In addition, remember last input file position and statistics. php mysql_filter_slow_log.php -T=3 -eu=root --no-duplicates --sort-execution-count --top=10 --incremental linux-slow.log > mysql-slow-queries.log # Start permanent filtering of all slow queries from now on: at least 3 seconds or examining 10000 rows, exclude users root and test tail -f -n 0 linux-slow.log | python mysql_filter_slow_log.py -T=3 -R=10000 -eu=root -eu=test & # (-n 0 outputs only lines generated after start of tail) # Stop permanent filtering kill `ps auxww | grep 'tail -f -n 0 linux-slow.log' | egrep -v grep | awk '{print $2}'`Filter Options-T=min_query_time -R=min_rows_examined -ih, --include-host -eh, --exclude-host -iu, --include-user -eu, --exclude-user -iq, --include-query --date=date_first-date_last Include only queries between date_first (and date_last). Input: Date Range: 13.11.2006 -> 13.11.2006 - 14.11.2006 (exclusive) 13.11.2006-15.11.2006 -> 13.11.2006 - 16.11.2006 (exclusive) 15-11-2006-11/13/2006 -> 13.11.2006 - 16.11.2006 (exclusive) >13.11.2006 -> 14.11.2006 - later 13.11.2006- -> 13.11.2006 - later earlier - 13.11.2006 (exclusive) -13.11.2006 -> earlier - 14.11.2006 (exclusive) Please do not forget to escape the greater or lesser than symbols (>= 5.1.21 (or patched): 3 seconds = 3000000 microseconds # long_query_time=3.000000 # minimum: 0.000001 (1 microsecond) # Activate the Slow Query Log slow_query_log # >= 5.1.29 # log-slow-queries # deprecated since 5.1.29 # Write to a custom file name (>= 5.1.29) # slow_query_log_file=file_name # default: /data_dir/host_name-slow.log # Log all queries without indexes # log-queries-not-using-indexes # Log only queries which examine at least N rows (>= 5.1.21) # min_examined_row_limit=1000 # default: 0 # Log slow OPTIMIZE TABLE, ANALYZE TABLE, and ALTER TABLE statements # log-slow-admin-statements # Log slow queries executed by replication slaves (>= 5.1.21) # log-slow-slave-statements # MySQL 5.1.6 through 5.1.20 had a default value of log-output=TABLE, so you should force # Attention: logging to TABLE only includes whole seconds information log-output=FILE ## Admin query for online activation is possible since MySQL 5.1 (without server restart) ## SET @@global.slow_query_log=1 ## SET @@global.long_query_time=1 ## Show current variables related to the Slow Query Log ## SHOW GLOBAL VARIABLES WHERE Variable_name REGEXP 'admin|min_examined|log_output|log_queries|log_slave|long|slow_quer' [Less]

Money Analyzer

A web tool to analyse your earnings and costs. Features: * Import your cash account statement with CSV * Using Google Graphs * Little AJAX Features to group positions * Auto-Group * and many more. Searching members to enhance.

mysql-search

Purpose The script filters the MySQL Slow Query Log to show queries which impacted performance most and is intended to be used by DB admins and application developers. The log file is analyzed and processed as a stream, line after line, so there is no need to load the whole log file into memory. ... [More] --no-duplicates is a very useful option to see only necessary statistics. --incremental remembers last input file positions and statistics in a SQLite 3 database, so periodical executions on the same files run much faster. The Python version is usually 3-5 times faster than the PHP5 version. You are welcome to contribute a version translated into your favorite scripting language, just open a feature request under Issues i.e. to add a Perl version. Usage Examples # Filter slow queries executed for at least 3 seconds not from root, remove duplicates, # apply execution count as first sorting value and save first 10 unique queries to file. # In addition, remember last input file position and statistics. php mysql_filter_slow_log.php -T=3 -eu=root --no-duplicates --sort-execution-count --top=10 --incremental linux-slow.log > mysql-slow-queries.log # Start permanent filtering of all slow queries from now on: at least 3 seconds or examining 10000 rows, exclude users root and test tail -f -n 0 linux-slow.log | python mysql_filter_slow_log.py -T=3 -R=10000 -eu=root -eu=test & # (-n 0 outputs only lines generated after start of tail) # Stop permanent filtering kill ps auxww | grep 'tail -f -n 0 linux-slow.log' | egrep -v grep | awk '{print $2}' Filter Options -T=min_query_time -R=min_rows_examined -ih, --include-host -eh, --exclude-host -iu, --include-user -eu, --exclude-user -iq, --include-query --date=date_first-date_last Include only queries between date_first (and date_last). Input: Date Range: .11.2006 -> 13.11.2006 - 14.11.2006 (exclusive) .11.2006-15.11.2006 -> 13.11.2006 - 16.11.2006 (exclusive) -11-2006-11/13/2006 -> 13.11.2006 - 16.11.2006 (exclusive) >13.11.2006 -> 14.11.2006 - later .11.2006- -> 13.11.2006 - later earlier - 13.11.2006 (exclusive) -13.11.2006 -> earlier - 14.11.2006 (exclusive) Please do not forget to escape the greater or lesser than symbols (>= 5.1.21 (or patched): 3 seconds = 3000000 microseconds # long_query_time=3.000000 # minimum: 0.000001 (1 microsecond) # Activate the Slow Query Log slow_query_log # >= 5.1.29 # log-slow-queries # deprecated since 5.1.29 # Write to a custom file name (>= 5.1.29) # slow_query_log_file=file_name # default: /data_dir/host_name-slow.log # Log all queries without indexes # log-queries-not-using-indexes # Log only queries which examine at least N rows (>= 5.1.21) # min_examined_row_limit=1000 # default: 0 # Log slow OPTIMIZE TABLE, ANALYZE TABLE, and ALTER TABLE statements # log-slow-admin-statements # Log slow queries executed by replication slaves (>= 5.1.21) # log-slow-slave-statements # MySQL 5.1.6 through 5.1.20 had a default value of log-output=TABLE, so you should force # Attention: logging to TABLE only includes whole seconds information log-output=FILE ## Admin query for online activation is possible since MySQL 5.1 (without server restart) ## SET @@global.slow_query_log=1 ## SET @@global.long_query_time=1 ## Show current variables related to the Slow Query Log ## SHOW GLOBAL VARIABLES WHERE Variable_name REGEXP 'admin|min_examined|log_output|log_queries|log_slave|long|slow_quer' [Less]

mysql-query-analyzer

The MySQL Query Analyzer is a Google Summer of Code 2008 project.