Unnecessarily indirect. A JSON file would have provided the same advantages here, except it would not have required a fully separate language runtime, just a JSON library. Text files as data is definitely a great idea, though.
But using JSON would have required parsing text in order to load the file. That's fine in a lot of domains, but games often have to load megs and megs of data in a very short time.
To speed things up, lots of games have this sort of external program that reads in a human-readable text format and outputs cooked binary files (with appropriate endianness and so forth) for the game runtime to consume. The fact that this utility is an external program means that there's not a reason not to use a nice language instead of a C library.
For a toy example like this, the advantage isn't obvious, but when you're consuming dozens or hundreds of megs of compiled text, it's indispensable.
How was the text not parsed in the author's example? He did not write the parsing code, but the text always has to be parsed somewhere if you're going to do anything with the data in the text.
The parent post was suggesting to replace external calls to Common Lisp with external calls to a library for JSON parsing. In both instances, the text is parsed, and in both instances, it's possible to transform the human-readable data to a convenient-for-C binary format. The parent suggested that calling a JSON library would be faster than Common Lisp - I cannot evaluate that claim, but it may be true.
The distinction I'm trying to make is between parsing the text once (using a compiler-like utility which is separate from the game runtime) then shipping the "compiled" binary format, versus shipping the text file and parsing it every time it's loaded by the game.
I took the parent post to be suggesting linking JSON into the game and parsing on each load -- IMO, the separate-data-compiler approach is superior for large projects, as it speeds up loading by a couple orders of magnitude. That's what I was trying to say.
(On the other hand, linking the entire Common Lisp runtime into your game just to load text files is a terrible idea. Obviously JSON is better for runtime parsing, but I don't think that's what the author is doing.)
So, if you then ask what language the separate data compiler should be written in, it seems like Lisp is a nice approach since it has a nice native object format with a built-in parser. Presumably you could also write it in JavaScript or some other language with excellent JSON support (Ruby? I'm getting outside my expertise here) and get a similar benefit vis-a-vis not having to write a parser.
Edit: I'm betraying my C heritage by saying "linking the Common Lisp runtime." Does that make any sense at all? Assume I mean some viable method of shipping Lisp with the game and making external calls to it somehow. ;)
I understand what you're saying, but I think you're unnecessarily conflating two orthogonal concepts: how the human-readable data is parsed, and whether the shipped game uses that process or loads a binary format. Jerf's post made no mention of changing the second concept, only the first.
Presumably you could also write it in JavaScript or some other language with excellent JSON support
No surprise since JSON (aka JavaScript Object Notation) was designed for interpretation by JavaScript and subsets can also be directly interpreted by languages like Python.
"But using JSON would have required parsing text in order to load the file."
As others pointed out, this is a silly objection. You still don't need to bring Lisp into your dependency chain. Many JSON libraries will let you serialize straight into a struct you define in C, and if you want to slam the resulting memory to disk and bring it back later, nothing stops you. You get the JSON interpreter for fast turnaround during development and a fast freeze/thaw when you're done and want speed, and the freeze/thaw is adding basically no code.
It won't be hardly any easier in Lisp than it will in C; what you win with an expressive language on a toy problem you'll lose with the impedance mismatch between Lisp and C. You can either write it once in C or twice in C and Lisp.
10 comments
[ 5.2 ms ] story [ 279 ms ] threadTo speed things up, lots of games have this sort of external program that reads in a human-readable text format and outputs cooked binary files (with appropriate endianness and so forth) for the game runtime to consume. The fact that this utility is an external program means that there's not a reason not to use a nice language instead of a C library.
For a toy example like this, the advantage isn't obvious, but when you're consuming dozens or hundreds of megs of compiled text, it's indispensable.
The parent post was suggesting to replace external calls to Common Lisp with external calls to a library for JSON parsing. In both instances, the text is parsed, and in both instances, it's possible to transform the human-readable data to a convenient-for-C binary format. The parent suggested that calling a JSON library would be faster than Common Lisp - I cannot evaluate that claim, but it may be true.
I took the parent post to be suggesting linking JSON into the game and parsing on each load -- IMO, the separate-data-compiler approach is superior for large projects, as it speeds up loading by a couple orders of magnitude. That's what I was trying to say.
(On the other hand, linking the entire Common Lisp runtime into your game just to load text files is a terrible idea. Obviously JSON is better for runtime parsing, but I don't think that's what the author is doing.)
So, if you then ask what language the separate data compiler should be written in, it seems like Lisp is a nice approach since it has a nice native object format with a built-in parser. Presumably you could also write it in JavaScript or some other language with excellent JSON support (Ruby? I'm getting outside my expertise here) and get a similar benefit vis-a-vis not having to write a parser.
Edit: I'm betraying my C heritage by saying "linking the Common Lisp runtime." Does that make any sense at all? Assume I mean some viable method of shipping Lisp with the game and making external calls to it somehow. ;)
No surprise since JSON (aka JavaScript Object Notation) was designed for interpretation by JavaScript and subsets can also be directly interpreted by languages like Python.
As others pointed out, this is a silly objection. You still don't need to bring Lisp into your dependency chain. Many JSON libraries will let you serialize straight into a struct you define in C, and if you want to slam the resulting memory to disk and bring it back later, nothing stops you. You get the JSON interpreter for fast turnaround during development and a fast freeze/thaw when you're done and want speed, and the freeze/thaw is adding basically no code.
It won't be hardly any easier in Lisp than it will in C; what you win with an expressive language on a toy problem you'll lose with the impedance mismatch between Lisp and C. You can either write it once in C or twice in C and Lisp.