Sign in to PrePAN

Devel::ParseXS Parse XS files into an abstract syntax tree



use Devel::ParseXS;

my $parser = Devel::ParseXS->new();
$parser->parse_file( $file );
my $ast = $parser->tree;


I have a need to manipulate XS files, and it'd be a lot easier to do so if one could parse them into an abstract format for further manipulation.

ExtUtils::ParseXS is a masterfully woven tapestry, but it because it weaves parsing, interpretation and code generation into a single pass, it's hard to use it to create an abstract representation.

This led me to develop Devel::ParseXS and it's companion modules in the Devel::XS::AST namespace. The parsing code closely follows the perlxs documentation, and where necessary I've used the existing behavior of ExtUtils::ParseXS to disambiguate a few things.

Currently the parsing is pretty complete; not everything is further translated into a useable AST object. Prior to releasing this to CPAN, I'd like to get some feedback on the module names.

Devel::ParseXS and Devel::XS::AST sound good to me.


Can you outline what the goal of the module is?

"Devel" namespace is for people developing things for Perl itself, isn't it? I don't know if that is the right place.

XS is a kind of a mess basically, no? I think your parser is a good addition.
This module is pure awesomeness. I like it a lot and it would be really cool if it were on CPAN.
I appreciate a lot the work that went into it. I will study it closer over the next days.
I'd like to ask the author what he thinks of ANTLR , Parse::Eyapp , Marpa::PP , Marpa::XS , and if he would perhaps think of porting the parser to one of those in the future. I don't think that is necessary, but it would be interesting to know.

The initial need was to allow one to dissect existing XS into its components so as to rewrite it, but I'm interested in using it to experiment with different approaches to what XS does.

This is being written as if it might one day be part of the core; my thoughts on what goes where are evolving.


When I first started, I did think of using Marpa. However, I decided to model the parsing process on the existing ExtUtils::ParseXS code so that I could better debug and understand it. Additionally, I wanted to keep this as core-friendly as possible, with few external dependencies.

At this point, I can't gauge whether it's worth the effort to bring in another framework for the parser. The desire is certainly to get it to parse all XS code, but I think (hope) that the current code can easily be tweaked to handle it.
The Devel:: namespace is quite special because perl does something special with the "-d" flag (see perldoc perlrun).
I don't think this module fits that usage.
So it would be better considering another top-level namespace than "Devel::".
@dolmen: I hadn't considered that. Thanks.
@dolmen: Looking at CPAN, past practice in the Devel namespace leads to rather a murky (at best) picture of how Devel modules are used; some are designed to interact with Perl's -d flag, others not. In general though, there's enough of a difference in focus between what they do and what this does that this shouldn't be in Devel.

Please sign up to post a review.