Lexer Parser

BRL-CAD needs to be able to express grammars for geometry formats in order to provide a flexible, fast, robust and comprehensive framework for conversion from one geometry format type to another. Ideally, we will use tools designed specifically to generate lexers and parsers to automate the "grunt work" of handling such formats. Candidates are listed below, with pros and cons. Decison criteria include license, portability, robustness and ease of integration with our existing build logic.


The 800 pound gorilla of open source lexers. BSD license. Modern versions require M4, which is why existing Windows versions stop at 2.54 - exploring NetBSD m4 as a possible supporting tool. GNU m4 is a possibility if NetBSD's version doesn't work for some reason.

Public domain. Seems like it can be built on Windows MSVC platforms. Searching around, it looks like re2c + lemon may be a relatively common approach for those looking to replace flex/bixon. A worked example: http://fasterparser.blogspot.com/2010/11/re2c-lexer-lemon-parser-calculator.html

C++, LGPL licensed. Author "doesn't like it to be used for military applications" although he apparently took the clause forbidding such use out of the license itself... Requres Python. Probably not viable for integration into BRL-CAD's build system, or at least not likely to be less trouble than flex itself.


800 pound gorilla for parsers. GNU GPL licensed, which pretty much makes it a non-starter for inclusion in BRL-CAD's tree.

License compatible, implements some of the Bison features of interest to us. Author has been quite helpful. Current front-runner.

License compatible, grammar definition style different from yacc. Take a look at the example here: http://osdir.com/ml/db.sqlite.general/2003-10/msg00235.html


In many ways this is the most interesting of any of these tools (polish, community size, active developoment and use, language flexibility), but unfortunately it requires Java to generate code. To use this, we would have to include generated C code in the tree, detect if it is installed, and if installed regenerate the C code (otherwise use the pre-generated code.) This doesn't improve things much over Flex/Bison - particularly given that we'd have to rewrite all our existing code - except it would be much more reasonable to hope that someone could/would install Antlr on Windows. As a long term target Antlr is very interesting, but in the short term it is probably more work for similar results.

Combination of GPL and LGPL - would have to dig to see if parts we need are usable or not. Probably not of primary interest unless other options start to look really grim.