Projects tagged ‘lexer’

Parse, Analyze and Manipulate Perl (without perl) The ability to read, and manipulate Perl (the language) programmatically other than with perl (the application) was one that caused difficulty for ... [More] a long time. The cause of this problem was Perl's complex and dynamic grammar. Although there is typically not a huge diversity in the grammar of most Perl code, certain issues cause large problems when it comes to parsing. Indeed, quite early in Perl's history Tom Christenson introduced the Perl community to the quote "Nothing but perl can parse Perl", or as it is more often stated now as a truism: "Only perl can parse Perl" PPI tries to solve this problem. [Less]

re2c is a tool for writing very fast and very flexible scanners. Unlike any other such tool, re2c focuses on generating high efficient code for regular expression matching. As a result this allows a ... [More] much broader range of use than any traditional lexer offers. And Last but not least re2c generates warning free code that is equal to hand-written code in terms of size, speed and quality. [Less]

Alex is a tool for generating lexical analysers in Haskell

TenDRA is a three clause BSD-licensed C and C++ compiler, with C++ STL support forthcoming.

Ragel compiles finite state machines from regular languages into executable C, C++, Objective-C, D, Java or Ruby code. Ragel state machines can not only recognize byte sequences as regular expression ... [More] machines do, but can also execute code at arbitrary points in the recognition of a regular language. Code embedding is done using inline operators that do not disrupt the regular language syntax. [Less]

DMLT (Descriptive Markup Language Toolkit) is a tollkit for creating markup lexers and parsers. They could be used to generate a HTML-Output based on a markup language, like BBCode or some other things...

Project Lestes is an ongoing effort to make a C++ compiler with comprehensive C++ source code using the most advanced algorithms seen. Other goals include: easilly retargettable compiler, compiler ... [More] that can be successfully used in teaching compiler construction, compiler that can compile multitude of languages. [Less]

AST builder for Objective-C++ language. Features: High-performance hand-crafted lexer About 5 times faster than NRefactory's C# lexer Easy to use parser Able to parse complex constructs like int ... [More] (*(int*foo())() )() Able to parse Objective-C classes Easy to use AST tree Able to produce source from AST Open Source, Free Software NOTE: Source files must be previously preprocessed Useful and bugfixing patches are welcomed! Sources currently resides in NObjective so you should checkout it from: svn checkout http://objcmapper.googlecode.com/svn/trunk/NObjectiveAST/ NObjectiveASTUsage// use lexer to produce lexems // than use parser to produce AST var translationUnit = new Parser( new Lexer( File.ReadAllText( "..\\..\\test.cpp" ) ) ).TranslationUnit; [Less]

Recursive descent parser library based on functional combinators. The primary domain for funcparserlib is parsing little languages or external DSLs (domain specific languages). The library ... [More] itself is very small. Its source code is only 0.5 KLOC, with lots of comments included. It features the longest parsed prefix error reporting, as well as a tiny lexer generator for token position tracking. [Less]

A parser/lexer generator written in Java. This is a development fork of the well known JavaCC parser generator originally developed at Sun Microsystems.