You don't need both modules and files. Decades of Smalltalk development experience backs this up.
Files should be an implementation detail. It should be just one back-end option out of many for persisting code.
There needs to be some way of conceptualizing a set of code. The programmer needs to perform operations on sets of code. (Merging, diffs, patches, etc...) Programmers need to store code and share it with other programmers. Files are good at doing all of these things, but none of these have anything to do with some special inherent quality only files possess.
Smalltalkers have been operating on the level of granularity of individual methods for quite awhile. It brings a lot of nice flexibility.
I would suggest that Namespaces have enough overlapping functionality with other "set of code" type entities, that they can just subsume the role of those other entities. The Java community has shown that it's workable for every project to have its own namespace. Namespaces also resolve a lot of name collision problems that keep cropping up. (Isn't that dandy?)
EDIT: The entire set of libraries for a variety of programming languages is readily available. It would be a cool project to re-granulize the commonly used library corpus for a particular programming language at the level of individual functions. There would be a lot of small-granularity dependency information that wouldn't be available but some of it could be inferred. Statically typed languages could have more of it inferred than dynamically typed languages, but there could still be a lot for the latter.
> You don't need both modules and files. Decades of Smalltalk development experience backs this up.
While decades of elisp development shows us that without modules, nobody will want to use your language to write programs spanning more than one file. =\",
Files allow you to give code to your friends without extraordinary means, to use generalized programs like git and grep, and to post a copy up on the web.
Files also mean that your IDE doesn't have to try to be an OS. (Still can if it wants to, hello Emacs.)
That's just plain wrong. They are something you can send over communication protocols. They are a very useful tool for sharing, transmitting, and storing information. I'm fine with files as all of those things. When files subsume conceptual territory -- this is when there's inertia.
It would be like people insisting that novels can only exist in the form of bound rectangular slabs of dead tree. Once you free concepts from implementation details, new flexibility is enabled.
Files allow you to give code to your friends without extraordinary means, to use generalized programs like git and grep, and to post a copy up on the web.
Yes, but to fulfill these functions, they do not have to stake out conceptual territory, like becoming the granularity at which your version control system must operate. Again, decades of Smalltalk development shows that you can use files to give code to friends, use programs like git (Monticello is git for Smalltalk) and grep, and post copies on the web. Smalltalkers do all of the above without files subsuming more conceptual function than just being files -- a particular means of storing and sharing information.
So what plain-text source encodings are is a serialization format. Now that's interesting, because running with that thought, what if there were multiple equivalent serializations for the same program? What if one were more convenient than another for certain transformations? Would that let us build better tools?
At some level, and in some domains, this already exists: the multiple serializations of RDF (N3, Turtle, RDF/XML, etc) plus OWL is very loosely equivalent to Prolog. There are very popular domain-specific non-textual programming environments too – the three obvious ones are Excel, LabVIEW and Max/MSP.
Isn't a lot of this the whole Lisp thing again? (The human-readable serialization format for Lisp is, as I understand it, damn close to being an ASCII serialization of the AST for Lisp, hence...)
Well, yes and no. Lisp is an ASCII serialization of the Lisp AST. This isn't true for most other languages. There is no reason why source for most languages couldn't be serialized as a combination of the AST and the stream of lexical tokens. This would have a few interesting advantages. For example, everyone could have the source formatting of their preference.
EDIT: Another example -- someone could implement a language homomorphic with Python without significant whitespace and curly brackets.
Even for lisp, you would have to define an extended syntax for it and modify the parser to additionally read and store whitespaces, comments and newlines. This way you will be able to serialize your source to plaintext without loosing the original format.
Non text sourcecode also allows for way smarter tooling, tree diffs on the AST, graphical repls and improved rendering of sourcecode.
Even for lisp, you would have to define an extended syntax for it and modify the parser to additionally read and store whitespaces, comments and newlines.
Note I said AST + stream of lexical tokens. If you store the whitespace and comments as tokens, such information can be preserved, including formatting hints.
Minor pedantry: Most variants of Lisp are not precisely a serialization of the AST. The read engine and macro engines have to pass over the code before it is truly an AST. For Common Lisp I believe technically either can be invoked at any time, but I am not a Common Lisp language lawyer.
I took the "flat" approach in the Loom server code (see https://loom.cc/source). Formerly I used all the object-oriented module techniques, but finally I scrapped all that for direct calls to functions in a flat name space. I found it more flexible and easier to understand, and it didn't force me to think about artificial boundaries. This makes the Perl code similar to my C code. I don't even use many static functions in C, preferring everything to have a unique name.
I also wrote a functional language interpreter in C. The language is called Fexl (see http://fexl.com/code/). The C code obviously uses a uniquely named flat function space. However, the Fexl language itself has far more flexibility, since you can easily create functions within functions.
In that case you're not at all worried about the names test1 and test2 conflicting with anything else. It's very lightweight, just like the Fibonacci example shown on this thread.
If you really want to "export" functions declared inside a scope, you can do it like this:
Armstrong doesn't seem to be saying anything about one of the more damning problems he points out - encapsulation. All functions in a module are available to each other, but hidden from code not in that module.
His solution of simply storing all code in a key-value DB only makes this worse in exchange for the dubious benefit of removing the ? "where do I put this new function?". Since he makes a big deal of encapsulation, his solution seems questionable.
EDIT - Please don't upvote this. I'm wrong in an incredibly silly way, and my wrongness doesn't need to be rewarded. If you feel generous, upvote ericflo instead for pointing out just how badly I missed the point.
Common Lisp can be compiled one function at a time, and functions belong to packages, which are pretty much key-value databases you can modify, where keys are strings (name of the function) and the values are the functions themselves. So if there is something wrong with some function, I just update the value in the "database" and all callers will call the newly defined function from now on. Nothing magical really. This workflow is supported by the CL IDEs such as SLIME.
Yea, Joe is slowly coming around to the idea of image based development instead of file based development. He's a few decades too late to be original, but it's still a great idea.
Of course, throwing out files has consequences on the tools you use, big time. Ask any Smalltalk'er.
Of course, throwing out files has consequences on the tools you use, big time. Ask any Smalltalk'er.
You can ask me. Who said anything about throwing out files? I use file-based tools all the time. I've grepped files and diffed them. I've exported a Class as a file, done some operation on it with another tool, then filed it back into the image. Whatever tool is best for the job. There's in-image tools, but I'm not limited to them.
Not being shackled to files has tremendous positive consequences. For example, I am free to code in the debugger almost 100% of the time. Even irb and iPython can look a bit restricted in comparison. If I do something esoteric to low-level code and crash the image, there's a transactional log of my code changes I can recover from, almost with impunity.
The choice of text editors for main development is rather restricted, though. (But with a pattern of very short methods as the norm, and the ability to customize the browsers at the level of individual methods, this isn't that big a deal.)
From a collaboration point of view, the thought of programming with single-function modules sounds like a nightmare. Now you don't have to find and install matching versions of five libraries, you have to find suitable versions of a hundred functions.
Perhaps nice people step in to stop the suffering and provide packages of function versions which are mutually compatible and which are usually used together -- but that's modules again.
Perhaps nice people step in to stop the suffering and provide packages of function versions which are mutually compatible and which are usually used together -- but that's modules again
Not exactly. You've sort of moved the large globs of stuff from the back-end to the front-end, where the grouping is more useful.
I'm reminded of difficulties, like the kind faced when purchasing 1.5" electrical tape. That stuff's mostly used only by pro electricians, so you won't find it retail, and it's even hard to find a particular brand from a contractor's supply outfit, and when you do, you have to buy it in big lots, like a whole box of 10 rolls. What if you just want to fix a drum and you need a particular make of tape made by Scotch and you only want one roll? Out of luck.
So say you need a particular function. You end up importing this entire module, which has dependencies that also are defined in large-granularity terms (other modules) that have their own dependencies. So to use one function, you get saddled with a whole heap of dependency overhead. You're really paying the price for "a whole box" where what you really need is just one particular thing.
But if everything was stored in easily accessible public repositories at the granularity of individual functions, this wouldn't be the case. You'd be able to pull the particular version of the function you need, and it would just pull the particular versions of the functions it depends on, and so on.
Things would be a whole lot more memory efficient. Another way to think of it: modules are a lot less modular than than they really should be.
So say you need a particular function. You end up importing this entire module, which has dependencies that also are defined in large-granularity terms (other modules) that have their own dependencies. So to use one function, you get saddled with a whole heap of dependency overhead.
Just playing the devil's advocate here: in this use case, why not just write the function yourself?
Because, if there was a way to quickly find and import the right function (with small-granularity dependencies), it would be much faster in a great many cases.
This is how PHP works, and I think it's smart. But to avoid the pitfalls of PHP, the new global functions need to be named in a way that's logical and unlikely to step on any toes.
So you'll start prefixing all names with a string so they don't clash. Then you'll notice all your related code uses these strings, what a waste, so you'll create a "module" in order to be able to drop the string in a defined context. Thus namespaces/modules are reinvented. Function drives form.
I understand his frustration, but a single global namespace is not the solution. This would be a regression to the place that PHP has been trying to escape from.
If modules aren't available, then people will invent pseudo namespace qualifiers (e.g., misc_foo). A global namespace will become like the .com TLD--a few early landgrabbers with the cool names like foo and bar, then a bevy of latecomers with fooo and baaar67.
The module/namespace concept isn't a 100% solution, but it's sufficient. People understand hierarchy. It's simple and effective. And it also solves the global-visibility/encapsulation issue that he sidestepped.
I don't think its really a single global 'namespace'.
It seems that instead of using actual names for the key in the k/v database, you would use something closer to an ip-address. give the function a unique id, and then make it have a really good description. To find the function, you use the description, and then reference the unique id.
To reference it in your code, you would bind the unique id to the descriptive name.
What I want to know is how you prevent massive duplication of functionality in this k/v database (with slightly different argument conventions or implementation details, for example).
You'd have the duplicate functionality problem anyhow, as function tables essentially work as k/v databases already. The keys are the method signatures (e.g. name, parameters, return type) and the value is the actual implementation.
Unless your language syntax specifies that argument conventions (e.g. names or ordering of parameters) are part of the method signature, you're always going to have to deal with the problem of foo(int a, char b) mapping differently from foo(int i, char c) or foo(char b, int a). You could do this, but it's not really widespread as far as I know. I know of no language that deduplicates functions based on implementation details.
Furthermore it's undoable in general. Equivalence of lambda terms being undecideable and so on.
Point being, at some point functions become sufficiently different that it's hard to tell. What if you write one md5 function in Erlang, another in whatever that Ruby syntax for Erlang is called. I'm sure the Erlang they both generate, or bytecode, or what-have-you (I don't know how BEAM or HIPE work on the inside) are very different, to the point that checking that they are truly equivalent is going to be near-impossible, even for a very strictly defined program like MD5.
I would rather have misc.foo (or in python terms, from misc import foo), than this_is_a_unique_function_name_that_does_foo, or '0000100302foo' with a ton of metadata to explain what it does and why it's better than every other similar function.
I'm trying to imagine visually parsing a program where the function names are merely unique identifiers that don't necessarily relate to their function, and it's not going well.
<quote> I'm trying to imagine visually parsing a program where the function names are merely unique identifiers that don't necessarily relate to their function... </quote>
> Also, It lets symbols inside the module call their friends by quux instead of foo_bar_baz_quux.
This is the main point in support of modules: they make you use names just like you are used to in real life. In a given context, shorter names suffice. When more contexts are involved, then you need qualifiers.
The global namespace in PHP is awesome. I don't have to learn a framework to solve a problem, there's a function for it.
Namespace qualifiers in names are a fine solution to the collision problem. That's effectively what's there already except every darn function has to be in a module.
Names don't have to be treated like property, you can invoke policy on them and change bad names. His dreams about rich metadata probably include version tags and hashes that would prevent this from silently breaking anything or in a way that couldn't be repaired by a tool.
Modules don't solve the problem you think they do, they just make the name of all functions longer and harder to figure out. They're not hierarchical.
He addressed issues of visibility/encapsulation and gave a great example of where modules fail to solve the problem. Though I don't quite follow his suggested fix, it looks like using lexical scope to encapsulate fib/3 but my Erlang is actually a bit rusty.
> Names don't have to be treated like property, you can invoke policy on them and change bad names.
A lot of PHP projects involve working with existing components or apps. Some of these use the global namespace with very obvious names, eg. phpBB's User, CodeIgniter's Session. Try and combine two components that use the same obvious names, and everything explodes in a giant mess of E_ERROR.
(It's the same deal with globally-scoped constants:
Erlang's module system isn't just a namespace mechanism; it really lends itself well to stuff like gen_server where you need to provide a set of callbacks. I'm surprised he didn't even touch on this in his post, as it definitely seems like one of Erlang's strengths to me....
This is kinda off the point, but I'm surprised at the spelling and punctuation mistakes in that mail--generally high-profile hackers have excellent written English.
i.e. "but their isn't", "Do we need module's at all?", "do suggest alternative syntax's here."
Also, regarding his writing, he did at least preface his post with This is a brain-dump-stream-of-consciousness-thing. Despite being a stickler for grammar rules myself, if I don't proofread my writing, there are bound to be plenty of grammar errors and typos that pop up, especially if my brain is really flying with ideas and my fingers are struggling to keep up. His post had that feel to it as well. A bunch of ideas, semi-related, bouncing ideas around haphazardly.
His native language is English. He's just sloppy. He's been in Sweden for decades and speaks Swedish with a typical English accent.
Source 1: My opinion. I've met him many times.
Source 2: His thesis states that he started at the Ericsson CSLab in 1985. CSLab was in Sweden.
Source 3: This (http://www.cse.chalmers.se/~rjmh/Armstrong/bits.ps) set of slides states that he has a B.Sc. from UCL (London) from 1972 and started at CSLAb in 1986 (not 1985, seems the dates are a bit fluid).
> spelling and punctuation mistakes
> excellent written English
Erlang's syntax, naming, and abbreviation conventions are straight schizophrenic.
Keywords, directives, method/module names, variables, and arguments are randomly spelled out, others shortened, if you're lucky with underscores or CamelCase, there are some familiar C style conventions but not widespread, directory structure seems to be highly project dependent, etc. - the list goes on.
It's a very "cluttered" language and the lack of a strong proficiency in written English as you mention clearly shows (to me) in the language/framework.
Keyword/directive spelling is British and any resemblance of C style conventions is a coincidence, because nobody cares about them in the Erlang world. Yes, Erlang has a weird syntax. Get over it.
I do agree to the point that standard libraries could have a more unified style. But the language is simple and concise.
This issue of where to place functions is something I have been curious about, too.
In the Lisp community, what are the standards for where to store your generic methods as opposed to defined classes they work with? Rephrased, if I define classes Foo and Bar, and I write generic method foobar (accepts as params instances of Foo and Bar), where do I put foobar?
In the C++ world, where should I put my friend functions that suffer a similar lack of obvious home?
I have often seen solutions where some package/class is chosen arbitrarily as the "proper" home for these cross-class communicators, but I have long felt like this is a compromise rather than good organization. And yes, I recognize that there are at least two kinds of organizations: In what file is my code? In what namespace is my code? I am concerned with the namespace aspect.
Generics/friends/multimethods are one of the solutions identified on the slides you posted (and protocols are interesting), but I did not see mention of appropriate namespaces for these.
I also thought about the topic of modules for a long time (about 2 years) for my language Babel-17 (www.babel-17.com). I also wrangled with the issue of encapsulation. I now know exactly how modules should work, what they are good for and why we need them ;-) And why you cannot have proper modules in Erlang without radical change. I am currently writing up these ideas in the spec for the new version of Babel-17 (v0.3) and implementing them. An implementation of Babel-17 that has modules will be available in about a week.
You already have this with CommonJS packages and modules as used by Node, RingoJS, Akshell and others.
The latest additions to require() mean that if you do something like require('foo') and there's a directory 'foo' in your require path that contains a package.json file (i.e. if you have package foo installed via the likes of npm), then the main module mentioned in that package.json file (which contains the meta data Joe speaks of) is immediately available to you. The module isn't a function, but rather an object, but then JavaScript isn't a purely functional language.
Now, admittedly, currently you need to pre install the packages, but there's no reason why require couldn't be modified to install them for you if they're unavailable or, if your code ran on a platform like Akshell, the packages wouldn't already be available to you.
Actually in node at least modules can export functions directly, just assign a function to `module.exports` and when you require() that module you'll get a function back. Plus you can add attributes to that function since functions are also objects.
Functions calling (linking) each other, and being built out of available pieces, ideally at a global scale -- this is just like the structure of the web.
* Functions should have URLs
* Functions should be augmented with metadata, like their language/platform
* There should be more and more fully defined IMT/MIME data types for lots of data
(Although I am more casual about this idea, and am only thinking half-seriously.)
We naturally recoil from the global namespace idea because we (rightly) anticipate huge issues with organization and duplication, so we want a hierarchical structure (files and modules) to keep our functions organized.
How about leaving our storage hierarchical for organizational purposes, but streamlining access? Instead of always using tedious import and require statements, simply call/use your functions, objects, gems, plugins, etc. directly and let the compiler and/or runtime infer from your usage which you are referring to in the case of ambiguity. Only if the ambiguity cannot be resolved in this manner would the programmer need to be explicit. Intelligent metadata and indexing could also add a lot of power to this sort of system.
Currently even our best languages require a large amount of cruft and legwork that is only really necessary in those 5% of cases where ambiguity can't be automatically inferred away. Seems like optimizing for the edge case if I've ever seen it.
Emacs is using single namespace, and people are just contributing .el scripts here and there (put in the right place, they get automatically loaded - (I'm just an user of these .el scripts - mainly for lisp/lua development))
Also "C" - single namespace, and though very verbose sometimes, google it, and you'll find result (saved me many times looking for Win32 API, GTK, Cairo, lua api, etc.)
But what about data? Static data, vars, etc. Also some languages/systems have initialization/deinitialization of the module (register/unregister, etc.)
But in general I like the idea, and thought about it, now I'm even thinking more.
He talks about putting them in database, well each database would have to have a name - maybe that's the name of the package (and you can rename), and you can merge. And if the DB is say SQL - you can even operate on merging databases way better than the methods of "ar, lib (msvc), ranlib, etc." or whatever the language/runtime provides
No one has mentioned that modules/packages hold together code that shares concepts.
Surely getting rid of Java packages and keeping the classes would be far less extreme than what Joe is suggesting for Erlang -- yet then we would have a zillion "meanings" of things like Image, Server, PDFFile, etc.
What about the Erlang functions that aren't "file2md5" and "downcase_char"? Is he way over-generalizing, or do Erlang programs typically just munge data in obvious ways?
You can package code without making the package a formal part of the namespace. Having a global namespace doesn't preclude packaging code, even if they are traditionally mixed as one thing. Many Smalltalk's have a single global namespace, it works pretty well.
I think this just exacerbates the problem with erlangs flat namespace, right now it is impossible to have 2 versions of a library inside the same vm because they both exist inside the same namespace, it is insane the amount of people that have had very obscure errors because they happened to call a module "http.erl"
I would appreciate a solution that is a first class solution 'in code' as opposed to some special case with module loaders, but I would like to see people talking about the problems they are solving before pontificating about solutions
It seems that this might maker code easier to write (the first time around) at the expense of making it more difficult to read. That's totally counter to the prevailing wisdom.
I'd also be curious to know what metadata the functions could be tagged with to make them sufficiently easy to find. The module and project that a function or type belongs to provides a lot of contextual information about it; coming up with metadata which captures that information without simply duplicating it (and therefore reinventing modules under another guise) sounds fairly non-trivial.
My first thought after reading the beginnings of his bulleted list: PHP? Haha. Still reading. It feels almost surreal to be reading the brain dump // stream of consciousness of someone who's created something so great. He talks just like you or I. Very fascinating.
There's nothing special about getting something like that done. There's nothing special about people who spend time on that kind of stuff either, save for the fact that they spend time on it. It's not difficult to be at the top, what's difficult is getting there and having enough resolve and persistence to do it. Only change is painful.
I like this idea. The problem I see is when you have different, but equally valid implementations.
In this case, it would make sense to back the name up with the developer/company name. But with no further decoration, as discussed. That way, it's clear which function implementation you're referring to. You can still have the mainstream (popular? chosen by the project's team?) implementations be referenced by function name alone. Any other implementations would need the dev/company name. That way you can have different parts of the project still able to reference different implementations of the same function.
The alternative would be to decorate the function with the implementation detail. And I think that's going against the simplicity he's after. Worse still is you'd still need to know you'll be supporting multiple implementations up front.
102 comments
[ 4.7 ms ] story [ 208 ms ] threadFiles should be an implementation detail. It should be just one back-end option out of many for persisting code.
There needs to be some way of conceptualizing a set of code. The programmer needs to perform operations on sets of code. (Merging, diffs, patches, etc...) Programmers need to store code and share it with other programmers. Files are good at doing all of these things, but none of these have anything to do with some special inherent quality only files possess.
Smalltalkers have been operating on the level of granularity of individual methods for quite awhile. It brings a lot of nice flexibility.
I would suggest that Namespaces have enough overlapping functionality with other "set of code" type entities, that they can just subsume the role of those other entities. The Java community has shown that it's workable for every project to have its own namespace. Namespaces also resolve a lot of name collision problems that keep cropping up. (Isn't that dandy?)
EDIT: The entire set of libraries for a variety of programming languages is readily available. It would be a cool project to re-granulize the commonly used library corpus for a particular programming language at the level of individual functions. There would be a lot of small-granularity dependency information that wouldn't be available but some of it could be inferred. Statically typed languages could have more of it inferred than dynamically typed languages, but there could still be a lot for the latter.
While decades of elisp development shows us that without modules, nobody will want to use your language to write programs spanning more than one file. =\",
Files allow you to give code to your friends without extraordinary means, to use generalized programs like git and grep, and to post a copy up on the web.
Files also mean that your IDE doesn't have to try to be an OS. (Still can if it wants to, hello Emacs.)
That's just plain wrong. They are something you can send over communication protocols. They are a very useful tool for sharing, transmitting, and storing information. I'm fine with files as all of those things. When files subsume conceptual territory -- this is when there's inertia.
It would be like people insisting that novels can only exist in the form of bound rectangular slabs of dead tree. Once you free concepts from implementation details, new flexibility is enabled.
Files allow you to give code to your friends without extraordinary means, to use generalized programs like git and grep, and to post a copy up on the web.
Yes, but to fulfill these functions, they do not have to stake out conceptual territory, like becoming the granularity at which your version control system must operate. Again, decades of Smalltalk development shows that you can use files to give code to friends, use programs like git (Monticello is git for Smalltalk) and grep, and post copies on the web. Smalltalkers do all of the above without files subsuming more conceptual function than just being files -- a particular means of storing and sharing information.
At some level, and in some domains, this already exists: the multiple serializations of RDF (N3, Turtle, RDF/XML, etc) plus OWL is very loosely equivalent to Prolog. There are very popular domain-specific non-textual programming environments too – the three obvious ones are Excel, LabVIEW and Max/MSP.
There's got to be something in that for someone.
Thank you! Well put! Actually, source code is just a human-readable serialization format. There's a lot to be done with that idea as well.
EDIT: Another example -- someone could implement a language homomorphic with Python without significant whitespace and curly brackets.
Note I said AST + stream of lexical tokens. If you store the whitespace and comments as tokens, such information can be preserved, including formatting hints.
I also wrote a functional language interpreter in C. The language is called Fexl (see http://fexl.com/code/). The C code obviously uses a uniquely named flat function space. However, the Fexl language itself has far more flexibility, since you can easily create functions within functions.
For example you can easily do things like this:
In that case you're not at all worried about the names test1 and test2 conflicting with anything else. It's very lightweight, just like the Fibonacci example shown on this thread.If you really want to "export" functions declared inside a scope, you can do it like this:
Then to grab the functions you say: You can change the names too, like this: There's no extra magic in the language, you're just applying the module to a handler which grabs the exported functions.His solution of simply storing all code in a key-value DB only makes this worse in exchange for the dubious benefit of removing the ? "where do I put this new function?". Since he makes a big deal of encapsulation, his solution seems questionable.
EDIT - Please don't upvote this. I'm wrong in an incredibly silly way, and my wrongness doesn't need to be rewarded. If you feel generous, upvote ericflo instead for pointing out just how badly I missed the point.
This is pretty much how I view my programming in Common Lisp.
This is also how a lot of Smalltalkers operate.
Of course, throwing out files has consequences on the tools you use, big time. Ask any Smalltalk'er.
You can ask me. Who said anything about throwing out files? I use file-based tools all the time. I've grepped files and diffed them. I've exported a Class as a file, done some operation on it with another tool, then filed it back into the image. Whatever tool is best for the job. There's in-image tools, but I'm not limited to them.
Not being shackled to files has tremendous positive consequences. For example, I am free to code in the debugger almost 100% of the time. Even irb and iPython can look a bit restricted in comparison. If I do something esoteric to low-level code and crash the image, there's a transactional log of my code changes I can recover from, almost with impunity.
The choice of text editors for main development is rather restricted, though. (But with a pattern of very short methods as the norm, and the ability to customize the browsers at the level of individual methods, this isn't that big a deal.)
Perhaps nice people step in to stop the suffering and provide packages of function versions which are mutually compatible and which are usually used together -- but that's modules again.
Not exactly. You've sort of moved the large globs of stuff from the back-end to the front-end, where the grouping is more useful.
I'm reminded of difficulties, like the kind faced when purchasing 1.5" electrical tape. That stuff's mostly used only by pro electricians, so you won't find it retail, and it's even hard to find a particular brand from a contractor's supply outfit, and when you do, you have to buy it in big lots, like a whole box of 10 rolls. What if you just want to fix a drum and you need a particular make of tape made by Scotch and you only want one roll? Out of luck.
So say you need a particular function. You end up importing this entire module, which has dependencies that also are defined in large-granularity terms (other modules) that have their own dependencies. So to use one function, you get saddled with a whole heap of dependency overhead. You're really paying the price for "a whole box" where what you really need is just one particular thing.
But if everything was stored in easily accessible public repositories at the granularity of individual functions, this wouldn't be the case. You'd be able to pull the particular version of the function you need, and it would just pull the particular versions of the functions it depends on, and so on.
Things would be a whole lot more memory efficient. Another way to think of it: modules are a lot less modular than than they really should be.
Just playing the devil's advocate here: in this use case, why not just write the function yourself?
If modules aren't available, then people will invent pseudo namespace qualifiers (e.g., misc_foo). A global namespace will become like the .com TLD--a few early landgrabbers with the cool names like foo and bar, then a bevy of latecomers with fooo and baaar67.
The module/namespace concept isn't a 100% solution, but it's sufficient. People understand hierarchy. It's simple and effective. And it also solves the global-visibility/encapsulation issue that he sidestepped.
It seems that instead of using actual names for the key in the k/v database, you would use something closer to an ip-address. give the function a unique id, and then make it have a really good description. To find the function, you use the description, and then reference the unique id.
To reference it in your code, you would bind the unique id to the descriptive name.
What I want to know is how you prevent massive duplication of functionality in this k/v database (with slightly different argument conventions or implementation details, for example).
Unless your language syntax specifies that argument conventions (e.g. names or ordering of parameters) are part of the method signature, you're always going to have to deal with the problem of foo(int a, char b) mapping differently from foo(int i, char c) or foo(char b, int a). You could do this, but it's not really widespread as far as I know. I know of no language that deduplicates functions based on implementation details.
Point being, at some point functions become sufficiently different that it's hard to tell. What if you write one md5 function in Erlang, another in whatever that Ruby syntax for Erlang is called. I'm sure the Erlang they both generate, or bytecode, or what-have-you (I don't know how BEAM or HIPE work on the inside) are very different, to the point that checking that they are truly equivalent is going to be near-impossible, even for a very strictly defined program like MD5.
My point being - a point also raised by Joe - is that just "adding dots" doesn't solve anything.
I'm trying to imagine visually parsing a program where the function names are merely unique identifiers that don't necessarily relate to their function, and it's not going well.
from global_database import foo291 as foo
and you'd have lots of meta information associated to foo291 so you could easily find it by doing:
db-search blah
Just to be pedantic: you're doing this already.
int add(int a, int b){ return a - b; }
It's visually much easier to parse foo.bar_baz.quux than foo_bar_baz_quux.
Also, It lets symbols inside the module call their friends by quux instead of foo_bar_baz_quux.
This is the main point in support of modules: they make you use names just like you are used to in real life. In a given context, shorter names suffice. When more contexts are involved, then you need qualifiers.
Namespace qualifiers in names are a fine solution to the collision problem. That's effectively what's there already except every darn function has to be in a module.
Names don't have to be treated like property, you can invoke policy on them and change bad names. His dreams about rich metadata probably include version tags and hashes that would prevent this from silently breaking anything or in a way that couldn't be repaired by a tool.
Modules don't solve the problem you think they do, they just make the name of all functions longer and harder to figure out. They're not hierarchical.
He addressed issues of visibility/encapsulation and gave a great example of where modules fail to solve the problem. Though I don't quite follow his suggested fix, it looks like using lexical scope to encapsulate fib/3 but my Erlang is actually a bit rusty.
A lot of PHP projects involve working with existing components or apps. Some of these use the global namespace with very obvious names, eg. phpBB's User, CodeIgniter's Session. Try and combine two components that use the same obvious names, and everything explodes in a giant mess of E_ERROR.
(It's the same deal with globally-scoped constants:
Makes life very interesting.)i.e. "but their isn't", "Do we need module's at all?", "do suggest alternative syntax's here."
Also, regarding his writing, he did at least preface his post with This is a brain-dump-stream-of-consciousness-thing. Despite being a stickler for grammar rules myself, if I don't proofread my writing, there are bound to be plenty of grammar errors and typos that pop up, especially if my brain is really flying with ideas and my fingers are struggling to keep up. His post had that feel to it as well. A bunch of ideas, semi-related, bouncing ideas around haphazardly.
He showed us his early design documents of erlang, including original compile times and performance measurements. Awesome stuff.
He does sound British, but there's a certain foreign-ness about the way he speaks (probably picked up from Ericsson).
Source 1: My opinion. I've met him many times.
Source 2: His thesis states that he started at the Ericsson CSLab in 1985. CSLab was in Sweden.
Source 3: This (http://www.cse.chalmers.se/~rjmh/Armstrong/bits.ps) set of slides states that he has a B.Sc. from UCL (London) from 1972 and started at CSLAb in 1986 (not 1985, seems the dates are a bit fluid).
Erlang's syntax, naming, and abbreviation conventions are straight schizophrenic.
Keywords, directives, method/module names, variables, and arguments are randomly spelled out, others shortened, if you're lucky with underscores or CamelCase, there are some familiar C style conventions but not widespread, directory structure seems to be highly project dependent, etc. - the list goes on.
It's a very "cluttered" language and the lack of a strong proficiency in written English as you mention clearly shows (to me) in the language/framework.
I do agree to the point that standard libraries could have a more unified style. But the language is simple and concise.
In the Lisp community, what are the standards for where to store your generic methods as opposed to defined classes they work with? Rephrased, if I define classes Foo and Bar, and I write generic method foobar (accepts as params instances of Foo and Bar), where do I put foobar?
In the C++ world, where should I put my friend functions that suffer a similar lack of obvious home?
I have often seen solutions where some package/class is chosen arbitrarily as the "proper" home for these cross-class communicators, but I have long felt like this is a compromise rather than good organization. And yes, I recognize that there are at least two kinds of organizations: In what file is my code? In what namespace is my code? I am concerned with the namespace aspect.
This sounds similar to the expression problem.
http://www.infoq.com/presentations/Clojure-Expression-Proble...
The latest additions to require() mean that if you do something like require('foo') and there's a directory 'foo' in your require path that contains a package.json file (i.e. if you have package foo installed via the likes of npm), then the main module mentioned in that package.json file (which contains the meta data Joe speaks of) is immediately available to you. The module isn't a function, but rather an object, but then JavaScript isn't a purely functional language.
Now, admittedly, currently you need to pre install the packages, but there's no reason why require couldn't be modified to install them for you if they're unavailable or, if your code ran on a platform like Akshell, the packages wouldn't already be available to you.
Functions calling (linking) each other, and being built out of available pieces, ideally at a global scale -- this is just like the structure of the web.
* Functions should have URLs
* Functions should be augmented with metadata, like their language/platform
* There should be more and more fully defined IMT/MIME data types for lots of data
(Although I am more casual about this idea, and am only thinking half-seriously.)
We naturally recoil from the global namespace idea because we (rightly) anticipate huge issues with organization and duplication, so we want a hierarchical structure (files and modules) to keep our functions organized.
How about leaving our storage hierarchical for organizational purposes, but streamlining access? Instead of always using tedious import and require statements, simply call/use your functions, objects, gems, plugins, etc. directly and let the compiler and/or runtime infer from your usage which you are referring to in the case of ambiguity. Only if the ambiguity cannot be resolved in this manner would the programmer need to be explicit. Intelligent metadata and indexing could also add a lot of power to this sort of system.
Currently even our best languages require a large amount of cruft and legwork that is only really necessary in those 5% of cases where ambiguity can't be automatically inferred away. Seems like optimizing for the edge case if I've ever seen it.
Also "C" - single namespace, and though very verbose sometimes, google it, and you'll find result (saved me many times looking for Win32 API, GTK, Cairo, lua api, etc.)
But what about data? Static data, vars, etc. Also some languages/systems have initialization/deinitialization of the module (register/unregister, etc.)
But in general I like the idea, and thought about it, now I'm even thinking more.
He talks about putting them in database, well each database would have to have a name - maybe that's the name of the package (and you can rename), and you can merge. And if the DB is say SQL - you can even operate on merging databases way better than the methods of "ar, lib (msvc), ranlib, etc." or whatever the language/runtime provides
Surely getting rid of Java packages and keeping the classes would be far less extreme than what Joe is suggesting for Erlang -- yet then we would have a zillion "meanings" of things like Image, Server, PDFFile, etc.
What about the Erlang functions that aren't "file2md5" and "downcase_char"? Is he way over-generalizing, or do Erlang programs typically just munge data in obvious ways?
I would appreciate a solution that is a first class solution 'in code' as opposed to some special case with module loaders, but I would like to see people talking about the problems they are solving before pontificating about solutions
I'd also be curious to know what metadata the functions could be tagged with to make them sufficiently easy to find. The module and project that a function or type belongs to provides a lot of contextual information about it; coming up with metadata which captures that information without simply duplicating it (and therefore reinventing modules under another guise) sounds fairly non-trivial.
In this case, it would make sense to back the name up with the developer/company name. But with no further decoration, as discussed. That way, it's clear which function implementation you're referring to. You can still have the mainstream (popular? chosen by the project's team?) implementations be referenced by function name alone. Any other implementations would need the dev/company name. That way you can have different parts of the project still able to reference different implementations of the same function.
The alternative would be to decorate the function with the implementation detail. And I think that's going against the simplicity he's after. Worse still is you'd still need to know you'll be supporting multiple implementations up front.