I've done some work on the read_token function.
The implementation on the site has some problems.
* The string conversion to chstr is unnecessary. Rune comparisons can be done simple with `ch == '('`.
* The 128 byte buffer will cause a panic when you enter an atom that exceeds this size. The idiomatic way to do this, is to use a `bytes.Buffer` instance.
* There is also an extra error check in the rune reading loop. The original ignored possible EOF's. Not sure how relevant it was in this particular use case, but just ignoring errors outright is never a good idea.
Fab, thanks for that. Not sure how I picked up the idea that a string( chstr ) was needed there.
As far as error checks go - yeah, there are many places I spotted on the way through where errors might happen that neither the original nor my version are checking for. I'll have another shot at it and see what I can clean up at some point in the week.
You might try looping your tokenizer in its own goroutine and passing tokens along a channel. That separates emitting a token from the loop/return structure of next_token.
Looks like I'm not the only one working on this. A couple of days ago I forked a Lispy language in Go called Kakapo. Yesterday I added goroutines, channels and macros.
8 comments
[ 0.75 ms ] story [ 31.4 ms ] thread* The string conversion to chstr is unnecessary. Rune comparisons can be done simple with `ch == '('`.
* The 128 byte buffer will cause a panic when you enter an atom that exceeds this size. The idiomatic way to do this, is to use a `bytes.Buffer` instance.
* There is also an extra error check in the rune reading loop. The original ignored possible EOF's. Not sure how relevant it was in this particular use case, but just ignoring errors outright is never a good idea.
https://gist.github.com/2551271
As far as error checks go - yeah, there are many places I spotted on the way through where errors might happen that neither the original nor my version are checking for. I'll have another shot at it and see what I can clean up at some point in the week.
Edit: check out this example http://blog.golang.org/2011/09/two-go-talks-lexical-scanning...
Having "grown up" in OO-land, the stateFn pattern was a bit odd to me. However in practical terms this pattern is pretty neat.
I've written a very basic lisp in Python and I'm trying to decide if I like Go by learning some.
https://github.com/ijt/kakapo