nd set its astgrammar to 'ABC::PAST::Grammar' (from src/PAST/Grammar.tg)
18:59 <pmichaud> that new compiler object can then be obtained via   $P99 = compreg 'ABC'
18:59 <PerlJam> The language method is just a wrapper around the compreg opcode?
18:59 <pmichaud> at the moment, yes
18:59 <pmichaud> eventually it's likely that one will be able to do the following:
18:59 <pmichaud> $P0 = new ['HLLCompiler']; $P0.'init'('language'=>'ABC', 'parsegrammar'=>'ABC::Grammar', 'astgrammar'=>'ABC::PAST::Grammar')
19:00 <PerlJam> Is there some reason to be mentioning "grammar" everywhere?
19:00 <PerlJam> "parsegrammar" and "astrgrammar" specifically
19:01 <pmichaud> allison is making a strong distinction between grammars and the items they grammarizes
19:01 <PerlJam> er, "astgrammar" even
19:01 <pmichaud> besides, we should probably make it clear that what HLLCompiler is expecting here is, in fact, a grammar
19:01 <pmichaud> there's one grammar for converting source to a parse tree, and another grammar for converting the parse tree into an ast
19:01 <PerlJam> okay
19:02 <pmichaud> so, the 'HLLCompiler' object has a built-in 'compile' method, which knows how to call the parser, ast, etc. to compile source into a
                 target
19:03 <pmichaud> so, after creating the HLLCompiler object, one could do:   $P99 = compreg 'ABC'; $P1 = $P99.'compile'(source)
19:03 <pmichaud> and it will compile the source into executable code
19:03 <pmichaud> the compiler method understands various options, thus
19:03 <Coke> << someone pelase summarize this discussion and make sure it ends up in a readme or something. Danke. >>
19:04 <pmichaud> $P1 = $P99.'compile'(source, 'target'=>'pir')    # compile source into PIR
19:04 <pmichaud> $P1 = $P99.'compile'(source, 'target'=>'past')    # return the ast from source
19:04 <pmichaud> so far so good?
19:04 <PerlJam> yep
19:05 <pmichaud> okay
19:05 <pmichaud> now then, at present the typical invocation for any hll compiler is as follows:
19:05 <pmichaud>     parrot language.pbc [options] [sourcefile]
19:05 <pmichaud> for abc, we typically do
19:05 <pmichaud>     parrot abc.pbc
19:05 <pmichaud> or
19:05 <pmichaud>     parrot abc.pbc source.abc
19:06 <pmichaud> The HLLCompiler object understands standard command line invocation, so that the compiler writer doesn't have to write a command line
                 processor
19:06 <pmichaud> this is what is happening in the 'main' subroutine of abc.pir
19:06 <pmichaud> er
19:06 <pmichaud> hmmm
19:06 <pmichaud> there is no main
19:07 <PerlJam> and now we come to a very important question ... where is main?  :)
19:07 <pmichaud> ...hey, what happened to main?
19:08 <PerlJam> I'm guessing that since abc.pir doesn't have a :main, it does nothing?
19:08 <PerlJam> (other than load stuff)
19:08 <pmichaud> I obviously forgot to include :main
19:09 <pmichaud> since the tests aren't invoking abc.pir from a parrot command line, we didn't notice :-)
19:09 <pmichaud> just a sec, fixing.
19:09 <pmichaud> (actually, what happened is I removed :main during the HLLCompiler refactor)
19:11 <pmichaud> r15906
19:11 <pmichaud> however, :main is just five lines long
19:11 -!- Irssi: Pasting 5 lines to #parrot. Press Ctrl-K if you wish to do this or Ctrl-C to cancel.
19:11 <pmichaud> .sub 'main' :main .param pmc args $P0 = compreg 'ABC' .return $P0.'command_line'(args)
19:11 <pmichaud> .end
19:11 <PerlJam> yeah, I figured it would be short (looking at command_line in Parrot/HLLCompiler.pir)
19:11 <pmichaud> grr, irssi paste didn't work
19:11 <pmichaud> but you get the point -- we just pass control to the 'command_line' method of HLLCompiler
19:12 <pmichaud> (N.B.: 'command_line' is about to go through a significant refactor, but the API remains the same)
19:12 <pmichaud> command_line handles a lot of standard options for compilers
19:12 <pmichaud> for example, the --target= option allows us to change what abc.pbc does
19:13 <pmichaud> ./parrot abc.pbc           # invoke an interactive compiler
19:13 <pmichaud> ./parrot abc.pbc file.abc         # compile and execute file.abc
19:13 <pmichaud> ./parrot abc.pbc --target=PIR  file.abc   # compile file.abc to PIR
19:14 <pmichaud> ./parrot abc.pbc --encoding=utf8 file.abc    # compile and execute file.abc, file.abc is encoded as utf8
19:14 <svnbotl> r15906 | pmichaud++ | trunk:
19:14 <svnbotl> : [abc]:
19:14 <svnbotl> : * Restore lost :main sub
19:14 <pmichaud> ./parrot abc.pbc --combine file1.abc file2.abc file3.abc   # combine three files into a single source unit and compile and execute that
19:14 <PerlJam> abc doesn't seem to know about # comments
19:15 <pmichaud> probably not.  needs a grammar change :-)
19:15 <pmichaud> (patches welcome!)
19:15 <pmichaud> so, that's the high-level view of creating a compiler
19:16 <pmichaud> the rest of abc.pir is simply including the various builtin functions, the parse grammar (compiled to pir by pgc), and the ast grammar
                 (compiled to pir by tgc)
19:16 <PerlJam> So ... I'm still fuzzy on tge.  Where does "node" come from in PASTGrammar.tg?
19:16 <pmichaud> it's automatic, kinda like 'self'
19:17 <pmichaud> each transform rule automatically initializes 'node' to be the node being transformed, and 'tree' to be the object doing the
                 transforming
19:17 <PerlJam> and it appears to be an lvalue?  (i.e. I can replace the current node with another)
19:18 <pmichaud> actually, it's a transformation, so we return the transformed node
19:18 <PerlJam> is there anything else that's just magically there like node?
19:18 <pmichaud> just node and tree
19:19 <pmichaud> in perl6, it would probably be something like   transform past ($tree: ABC::Grammar $node) { ... }
19:19 <pmichaud> i.e., tree is our "invocant", and node is the thing being transformed
19:19 <pmichaud> so, let's look at a transformation
19:20 <PerlJam> yeah, let's look at the ROOT transform  :)
19:20 <pmichaud> let's start with statement_list
19:20 <pmichaud> when we enter this transform, 'node' is initialized to the match object corresponding to a <statement_list> subrule in the grammar
19:20 <pmichaud> that rule looks like (from abc.pg)
19:21 <pmichaud> rule statement_list { <statement>? [ ; <statement>? ]* }
19:21 <pmichaud> so, a statement_list node will have $<statement>, an array of statement nodes
19:22 <pmichaud> to transform this statement_list node into past, we transform each of nodes in $<statement>, and place those as children of a
                 PAST::Stmts node
19:22 <PerlJam> And the grammat guarantees that it's an array of statements so there's no error checking in the transform?
19:22 <pmichaud> that's right
19:22 <cognominal> is pugs with PGE supposed to crash in the development version?
19:22 <pmichaud> cognominal: I have no idea
19:23 <PerlJam> cognominal: probably if the pugs people haven't kept up with the PGE changes  :-)
19:23 <cognominal> oki
19:23 <pmichaud> at the moment, there are just a few basic AST node types:  PAST::Node (base), PAST::Var, PAST::Val, PAST::Op, PAST::Stmts, and
                 PAST::Block
19:24 <pmichaud> PAST::Node is really an abstract class, so we can discount it
19:25 <pmichaud> so, the transform on ABC::Grammar::statement_list is simply creating a new PAST::Stmts node, transforming each <statement> child into
                 past and adding it to the PAST::Stmts node.  We then return the PAST::Stmts node, which is the ast representation of this particular
                 statement_list
19:26 <pmichaud> okay so far?
19:26 <PerlJam> yes, I think.
19:26 <PerlJam> What's the first arg to tree.'get'() for?
19:26 <PerlJam> It's always 'past'
19:26 <pmichaud> it's selecting which rule to apply
19:27 <PerlJam> And the third arg?
19:27 <purl> the third arg is probably char *envp[]
19:27 <pmichaud> the third arg is indicating the "type" of the second arg
19:27 <PerlJam> why?
19:27 <pmichaud> when TGE can't figure it out on its own
19:27 <pmichaud> (more detail coming)
19:27 <pmichaud> in a parse tree, all of the nodes end up with the same type (ABC::Grammar, in this case)
19:27 <pmichaud> the fact that a particular node is a 'statement_list' is actually held in that node's parent
19:28 <pmichaud> i.e., any given node in the parse tree doesn't know its own name
19:28 <pmichaud> and they're all the same type
19:28 <pmichaud> sooooo.....
19:29 <pmichaud> in such cases we have to tell TGE the "type" of the second paramater, which we just artificially create by appending the subrule name
                 to the type of the node
19:30 <pmichaud> (yes, this feels wrong.  The only alternatives I've come up with are to somehow create a separate class for every subrule (bad), or to
                 change/augment the semantics of S05 so that every node can know the name its parent has given it.)
19:30 <pmichaud> (This is also why I'm continually harping on mp6 to provide a concrete example of how this sort of transformation would be performed.)
19:30 <purl> okay, pmichaud.
19:30 <pmichaud> purl, forget This
19:30 <purl> pmichaud: I forgot this
19:31 <pmichaud> questions, or shall I proceed to another example?
19:32 <PerlJam> Why does the ROOT transform do node = node['program'] ?
19:32 <pmichaud> because the actual node that it starts with is the <apply> node
19:32 <pmichaud> so, I just shift down to the <program> node
19:33 <PerlJam> so, that's a way to ignore nodes?
19:33 <pmichaud> that's probably an artifact of the recent refactors
19:33 <pmichaud> better might be to write:   pnode = node['program'] and then use pnode through the remainder of the rule
19:33 <pmichaud> so it doesn't look like an lvalue
19:34 <pmichaud> but yes, I'm essentially skipping the <apply> node and jumping directly into the <program> subnode
19:34 <PerlJam> okay, keep going. It's slowly sinking in :)
19:34 <pmichaud> okay, let's look at a more complex transform:  ABC::Grammar::statement
19:35 <pmichaud> the grammar rule is in http://svn.perl.org/parrot/trunk/languages/abc/src/abc.pg
19:35 <pmichaud> at "rule statement", and we can see it's a sequence of alternations
19:35 <PerlJam> yep
19:35 <pmichaud> so, the transform simply checks for each alternative and dispatches to the correct subtransform
19:36 <pmichaud> the next one is if_statement
19:37 <pmichaud> the parse rule for if_statement is  rule if_statement { if \( <expression> \) <statement> [ else <statement> ]? }
19:37 <pmichaud> so, $<expression> is an expression, and $<statement> is an array of 1 or 2 statements
19:38 <pmichaud> (if we didn't want the array, we could've aliased <statement> in the rule...)
19:38 <pmichaud> for an if statement, we simply create a PAST::Op node that has
19:38 <pmichaud>     'pasttype' => 'if'
19:39 <pmichaud>     child 0 => expression to be evaluated
19:39 <pmichaud>     child 1 => what to do if child 0 was true
19:39 <pmichaud>     child 2 => what to do if child 0 was false
19:39 <pmichaud> either child1 or child2 may be missing/null, in which case the node will do nothing (actually, it returns the result of child 0)
19:40 <PerlJam> yeah, the code makes sense. (in PASTGrammar.tg)
19:40 <pmichaud> I'll brush past while_statement, because it's cool -- it's the same concept as if_statement, except child 0 is the expression and child
                 1 is the thing to be done each time child 0 evaluates to true
19:41 <pmichaud> we also have pasttype values of 'unless' and 'until', and I'll soon be adding 'repeat_while' and 'repeat_until'
19:42 <pmichaud> (an interesting side note is that short-circuiting infix:&& is just 'pasttype'=>'if', and short-circuting infix:|| is just
                 'pasttype'=>'unless'
19:43 <pmichaud> okay so far?  next we delve into expressions
19:44 <PerlJam> yeah, I'm already looking there :)
19:45 <PerlJam> The ctype stuff is documented where again?
19:45 <pmichaud> it's going to change, but it's documented in compilers/past-pm/PAST/Node.pir
19:45 <pmichaud> ctype will become 'consttype', and vtype will become 'valuetype'
19:46 <PerlJam> okie.
19:46 <PerlJam> (I've forgotten what the + means)
19:46 <pmichaud> + means numeric, ~ means string
19:46 <pmichaud> (think perl6)
19:46 <PerlJam> so, i+ is?
19:46 <PerlJam> integer or float?
19:46 <pmichaud> i+ means the value can be used as-is in any int or numeric context
19:47 <pmichaud> n+ would mean the value can be used as-is in any num or numeric context
19:47 <PerlJam> Hm.  "const type" makes more sense to me as "context type"  now  :)
19:47 <pmichaud> that makes sense
19:48 <pmichaud> at any rate, it's just a compiler hint.  If no ctype is given, then past-pm will put the constant into a PMC (according to the vtype)
                 before doing anything with it
19:49 <pmichaud> per discussions on p2p with allison, we're likely to move ctype completely out of the individual PAST::Val nodes and into a table of
                 type => consttype
19:49 <pmichaud> so then past-pm would figure out the ctype based on the vtype
19:50 <pmichaud> (i.e., vtype == '.Integer' means it can be a constant in any int or numeric context)
19:50 <pmichaud> I'm still figuring out how to get language-specific tables into past-pm
19:52 <pmichaud> sounds like you've got the hang of things from here -- any other areas you want me to comment on?
19:52 <PerlJam> not at the moment.
19:53 <PerlJam> now all I need is a nice smallish language to try this stuff out on so that it can solidify in my head a little better
19:53 <pmichaud> the perl6 stuff obviously has a few more intricacies, but thus far it's just abc with more stuff in it
19:53 <PerlJam> I was thinking about trying my hand at javascript.
19:54 <pmichaud> that would be very cool.  I'm curious to see what ruby/Cardinal could do with this
19:55  * pmichaud goes to see if he has a capture log of this somewhere