Projects tagged ‘csv’ and ‘data’

pChart is a PHP class oriented framework designed to create aliased charts. Data can be retrieved from SQL queries, CSV files, or manually provided.

Cutplace is a tool to validate that data conform to an interface control document (ICD). Cutplace works with flat data formats using delimiters (such as CSV) or a fixed format. Such formats are ... [More] commonly used to exchange data between different platforms or physically separated systems. With cutplace you can describe these data in a simple and human readable spreadsheets using popular applications like Calc or Excel. It acts as "executable specification" that cutplace can use to validate that data actually conform to it. [Less]

benerator is a framework for creating realistic and valid high-volume test data, used for testing (unit/integration/load) and showcase setup. Metadata constraints are imported from systems and/or ... [More] configuration files. Data can be imported from and exported to files and systems, anonymized or generated from scratch. Domain packages provide reusable generators for creating domain-specific data as names and addresses internationalizable in language and region. It is strongly customizable with plugins and configuration options. [Less]

The Problem If you have any dealings at all with data and databases, then you almost certainly will have have to deal with comma-separated values (CSV) data. Unfortunately, the CSV files you are ... [More] given, or are required to produce, never seem to be in quite the right format for your particular business application. And because of the structure of CSV records, using standard text procesing tools like sed, awk and perl on CSV files is not as simple as it might be. The Solution CSVfix is a command-line stream editor specifically designed to deal with CSV data. With it, you can: Convert fixed format, multi-line and DSV files to CSV Reorder, remove, split and merge fields Convert case, trim leading & trailing spaces Search for specific content using regular expressions Filter out duplicate data or data on exclusion lists Perform sed/perl style editing Enrich with data from other sources Add sequence numbers and file source information Split large CSV files into smaller files based on field contents Perform arithmetic calculations on individual fields Validate CSV data against a collection of validation rules Convert from CSV to fixed format, XML, SQL, DSV Summarise CSV data, calculating averages, modes, frequencies etc. You can see a full list of CSVfix commands here. Update: CSVfix Version 0.95 now available for download. This adds several new commands (including a new XML generating command) and the ability to use separators other than a comma in CSV input. Details of the changes are here. New: If you find CSVfix useful, please also check out CSVTest, a test data generator, at http://code.google.com/p/csvtest. [Less]

Bluebird Reports is a PHP based reporting framework, which enables you to quickly create compelling reports. Highlights include login and access control, parameterized templates, a multitude of output ... [More] options, scheduling and powerful report formatting. [Less]

This class can be used to generate line, bar and 2D/3D pie graphs from multiple data sets using PHP5. Offers many style and data formatting options and an extensive readme and examples page. You can ... [More] customize almost any option. It can be built from data defined in a programmatic way, with data import from MySQL, XML or CSV files or functions. It can use custom colors, custom data point shapes or images, custom end arrows, optional inline format, set the graph scale, etc.. The graphs can be generated in image format either to a file or serve as the current script output. Please post any requested enhancements or bugs. [Less]

parseCSV is an easy to use PHP class to read and write CSV data properly. It fully conforms to the specifications outlined on the on the Wikipedia article. It has a few advanced features which help ... [More] make your life easier when dealing with CSV data. I created this class due to the lack of built-in and third-party support for handling CSV data in PHP. Please check the Examples page for basic sample code illustrating the use of parseCSV. Features:parseCSV is the only complete and fully featured CSV solution for PHP (as far as i know). Supports enclosed values, enclosed commas, double quotes and new lines. Automatic delimiter character detection. Sort data by specific fields/columns. Easy data manipulation. Basic SQL-like conditions, offset and limit options for filtering data. Error detection for incorrectly formatted input. It attempts to be intelligent, but can not be trusted 100% due to the structure of CSV, and how different programs like Excel for example outputs CSV data. Support for character encoding conversion using PHP's iconv function (requires PHP 5). Supports both PHP 4 & 5. Planned Features:Support negative offset values to offset from end of data rather than beginning. More advanced and mature conditions input. Currently it doesn't even support enclosing values in quotes. Some decent documentation, specifically regarding conditions syntax. Credits:parseCSV is based on the concept of Ming Hong Ng's CsvFileParser class. Contact:If you have any problems, questions or suggestions please create a new issue, or feel free to drop me a mail: contact at jimeh dot me [Less]

IntroductionThis is a library for parsing text files. The records could be comma delimited, tab delimited or delimited with any set of characters. The csv_parser class is used to parse text files ... [More] to extract records and fields. We are making the following assumptions : The record terminator is only one character in length. The field terminator is only one character in length. The fields are enclosed by single characters, if any. The parser can handle documents where fields are always enclosed, not enclosed at all or optionally enclosed. When fields are strictly all enclosed, there is an assumption that any enclosure characters within the field are escaped by placing a backslash in front of the enclosure character. --- The CSV files can be parsed in 3 modes. (a) No enclosures (b) Fields always enclosed. (c) Fields optionally enclosed. For option (c) when the enclosure character is optional, if an enclosure character is spotted at either the beginning or the end of the string, it is assumed that the field is enclosed. The csv_parser::init() method can accept a character array as the path to the CSV file. Since it is overloaded, it can also accept a FILE pointer to a stream that is already open for reading. The set_enclosed_char() method accepts the field enclosure character as the first parameter and the enclosure mode as the second parameter which controls how the text file is going to be parsed. The source documentation is available here http://www.israelekpo.com/csv-parser-documentation/ Library InstallationDecompress the tarball tar -zxvf csv_parser-x.x.tar.gz cd csv_parser Configure, compile and install ./configure make make all install Linking and Compiling Source Programsg++ -Wall -O2 -g csv_parser-config --cxxflags --libs example.cpp -o example Example Usage #include /** * Example Usage of libcsv_parser++ * * These are some of the characters you may use in this program * * @li DEC is how it would be represented in decimal form (base 10) * @li HEX is how it would be represented in hexadecimal format (base 16) * * @li DEC HEX Character Name * @li 0 0x00 null * @li 9 0x09 horizontal tab * @li 10 0x0A line feed, new line * @li 13 0x0D carriage return * @li 27 0x1B escape * @li 32 0x20 space * @li 33 0x21 double quote * @li 39 0x27 single quote * @li 44 0x2C comma * @li 92 0x5C backslash */ /** * Example Program - showing usage of the csv_parser class * * In this example, we include the csv_parser header file as * * Then we declare the variables we are going to use in the program * * @li The name of the input file is "example_input.csv" * @li The field terminator is the comma character * @li The record terminator is the new line character 0x0A * @li The field enclosure character is the double quote. * * In this example we will be parsing the document as the fields are optionally enclosed. * * The first record in the CSV file will be skipped. * * Please view the source code of this file more closely for details. * * @todo Add more examples using different parsing modes and different enclosure and line terminator characters. * @toto Include an example where the filename, field_terminator char, line_terminator char, enclosure_char and other program variables are loaded from a file. * * @param int The number of arguments passed * @param argv The array of arguements passed to the main function * @return int The status of the program returned to the operating system upon termination. * * @see csv_parser * * @author Israel Ekpo */ int main(int argc, char ** argv) { // const char * filename = argv[1]; /* Declare the variables to be used */ const char * filename = "example_input.csv"; const char field_terminator = ','; const char line_terminator = '\n'; const char enclosure_char = '"'; csv_parser file_parser; /* Define how many records we're gonna skip. This could be used to skip the column definitions. */ file_parser.set_skip_lines(1); /* Specify the file to parse */ file_parser.init(filename); /* Here we tell the parser how to parse the file */ file_parser.set_enclosed_char(enclosure_char, ENCLOSURE_OPTIONAL); file_parser.set_field_term_char(field_terminator); file_parser.set_line_term_char(line_terminator); unsigned int row_count = 1U; /* Check to see if there are more records, then grab each row one at a time */ while(file_parser.has_more_rows()) { unsigned int i = 0; /* Get the record */ csv_row row = file_parser.get_row(); /* Print out each column in the row */ for (i = 0; i < row.size(); i++) { printf("COLUMN %02d : %s\n", i + 1U, row[i].c_str()); } printf("====================================================================\n"); printf("END OF ROW %02d\n", row_count); printf("====================================================================\n"); row_count++; } return 0; } [Less]

Grubby is a framework to help you quickly generate random (or pseudo random) data to use for testing, etc. The data can be generated in the form of SQL INSERTs, CSV, or LDIF. Why does it not just ... [More] load the data directly? When you have lot of data to upload (e.g. for performance testing with real world volumes) then this can simply take too long. If you have files which contain your data then you can use the optimised tools which come with your datastore to do the loadnig for you. Much nicer. If you want to help out, please contact me. Otherwise,happy loading! [Less]

Dockhand began as a tool for generating SQL create table statements and Oracle SQL*Loader control files, but it quickly expanded and now aims to be a general purpose data manipulation, extraction, and deployment utility.