The REPL in (Lisp) driven development is generally attached to your editor, not like irb or ipython or whatever: so the thing you're building is either the resulting function, or something that mostly looks like a test already anyway and is easy to copy into a test module.
I suspect the problem is less that it discourages than that it does not encourage.
Which gets back to the posted article. REPL-driven development permitted him to feel confident without doing his usual process, it didn't discourage decoupling, but it didn't encourage decoupling either. TDD, to be done well, almost requires decoupling your code to make it more testable.
The same may (I'm not sure I agree, but I've never used Jupyter notebooks) be true of using notebook-styled development approaches and modularity. If the workflow permits relatively easy (and mostly safe) development without encouraging modularity, then it's incumbent on the developer to bring that discipline into the workflow.
And unfortunately, most of the new crop of programmers don't have that discipline. I tell them to write tests and they eyeroll when they think I'm not looking.
Jeremy Howard developed the fastai library using Jupyter Notebooks and his nbdev tool[1]. I'm not saying that to disagree with your claim - I don't have enough experience with Jupyter Notebooks yet to have much of a strong opinion on whether or not it discourages modularity. Just that folks who want to have the best of both worlds might be interested in how it can be done. My first thought is that it looks like Donald Knuth's literate programming ideas put to practice, and it has me excited about programming again.
Just to be fair, I don't think many practiononers of REPL-driven design would consider it a replacement for tests. It's an alternative workflow to TDD, yes, but the idea is that you still write comprehensive tests, they're just usually at a higher more system-y level than the ones you get from TDD.
It does come off very strangely dichotomic. As though REPL driven means no tests and bad practices, whereas TDD means tests and good practices?
As an idea, maybe we could do... both? Explore at the REPL, and then commit to tests and clean up once you've solidified your approach (i.e. how most Clojurists likely operate).
I think TDD purists seem to see no point in unit tests that are not written before the code. That is, if the unit tests were not used to drive the design and implementation of the code under test, they are useless. I find a lot of value in the test driven approach, but this purist view does not make sense to me. I sometimes, or often, write the code first (being able to do so in a repl in languages that support that is even better), when I find it easier to think through the problem in terms of implementation rather than expected output. Sometimes I will then comment out the code and begin writing tests and re-writing the code to pass them. I think that's what he should have done here; he could have benefited from playing around in the repl, but reduced his fear of refactoring later on by having the tests.
Yeah this is mostly how I operate when writing clojure.. development mostlty happens at the REPL and the bits that work get copied into a file somewhere and then once I have an idea how the whole system is going to work, I write an automated test for it instead of testing it manually. Tests replace manual testing in my workflow.
I would consider it a replacement for throwaway tests.
If I don't have a REPL, I often write a lot of little throwaway tests to deal with intermediate bits of the code. They end up being redundant with the real tests. They're often really bad tests, in that they're tightly coupled to implementation details. I should delete them. And I often forget to. Which creates extra maintenance burden for myself and my teammates.
If I do have a REPL, I do all of that extra intermediate verification junk in the REPL. It's often more effective, it's usually less effort, and it always reduces the volume of unnecessary brittle tests in the codebase.
This is spot on. You need tests so that, during the next REPL session, you don't make changes that broke your old ones. I do also agree that people should try out TDD, especially if tests are fast to run. But a REPL is powerful tool that should not be avoided. There's a reason it's being bundled with the latest JDK releases and is an essential part of LISP and other modern languages.
Reading the blogpost again, I don't see him suggesting that you should replace unit tests with "REPL-driven design", he is merely suggesting replacing the TDD workflow with a RDD workflow, which you can still end up with unit tests, it's just that you don't write them first.
His argument feels like a straw man. You can and should fill in a few tests after you confirm something in the REPL. Maybe Clojure folks don't do that, but Ruby and Elixir devs often do. I would imagine he's familiar with that approach.
I concur. Use REPL driven development to arrive at a solution, but use unit tests (or similar) to make sure there are no regressions in the future and that it continues actually being a solution.
I literally wrote code in .clj files and then my editor can slurp that up and barf it into a repl. There was never a point where I was writing something that "only existed in the repl" such that I couldn't just put it in a unit test if it needed formalized.
The usual workflow was something like
(comment
;; experiments go here
(defn add [a b]
(+ a b))
(= 2 (add 1 1))
(= 3 (add 1 2))
(= 99 (add 9 90))
)
Once you have what you want, you move the code out of the comment block. Whether you move it into a library or a test file is just dependent on the type of thing you're moving. Functions moved into library code. The samples that proved the happy paths (or unhappy paths) would go into test files using `clojure.test/is` (and typically a direct copy/paste).
The REPL in Clojure (and other Lisps) is far more tightly integrated with your editor environment. When I wrote python, I would do what you're describing: test something in (i)python, write some code, rinse, repeat. With Clojure, I Write said experiments in the same place my code already is (and will end up). Making that feedback loop 10x tighter makes a world of difference in UX: I don't think it helps insight to equate "typing things in irb" with "mashing C-x C-e until the expr does what you want".
(I have no idea if an equivalent workflow exists in Elxir, but I've never seen it been done that way: just like people use ipython, they use iex in my experience.)
A pretty common pattern I've seen in both Ruby and Elixir is to begin building up the state you want to work with in a test and then drop into a repl using pry (either adding a breakpoint in the code or in the test). I think this allows you to "Write said experiments in the same place [your] code already is".
Maybe I'm just misunderstanding what people in Clojure are doing. Please feel to point me in the right direction.
When you say "use a REPL", do you mean something like CIDER/SLIME, or do you mean something like ipython/irb? I do REPL (that is: CIDER) driven development all the time, and most of my experiments turn into tests quite naturally.
This is a good point. In parenthetically (it is also about not using statements) inclined langs it is easy to select from start to end of an expression or subexpressions. I never feel that ease when working with a Python REPL, while it is a joy to work in Geiser usually.
The distinction is a bit artificial. I agree that Python's REPL is underpowered when compared with SLIME. But it's trivial to write up a bit of Python code in an editor, and just 'reload(modname)' or 'exec(file(...).read())' from the REPL. Emacs' python-mode (like many other editors) also supports "evaluate this expression / line / block" to make this process simpler. YMMV, but my Common Lisp development really doesn't look very different from my Python work.
Even if we stipulate that's true[], saying you could do it in non-Lisps doesn't detract from the observation that virtually nobody actually does it, while both groups still call it "REPL".
[] I really don't think it is: reload(mod) comes with way more caveats than C-c C-k in CIDER.
How could you possibly know that "virtually nobody actually does it"? I have no reason to believe that I'm a special snowflake. 'reload' and 'exec' exist for these very purposes, and have been around since Python immemorial (well, at least since I started using Python 20 years ago). As for the 'caveats': if you're using 'reload' in your development process -- yes, you should take a few moments to read the documentation first, as you should with anything else. It isn't magic, and its behaviour is well explained.
You're making far too much of the distinction between different kinds of REPL. As Peter Norvig put it [1], as long as you have a "development process that allows for rapid development with continuous improvement", minor distinctions between languages [such as quibbles over how 'reload' works] really don't matter that much.
Squeeze all the power out of the tools that you use, and get things done -- ultimately that's all that matters. Religious wars like these were tiresome in the era of the Smug Lisp Weenie, and they haven't improved with age.
I made a distinction between CIDER/SLIME (which I think is what most Lisp users think of when they hear REPL-driven development) and ipython/irb/iex..., which I think is what most Python/Ruby/Elixir/... programmers call "REPL" in REPL-driven development. This distinction is important, because much of the criticism in these threads has been around how you translate REPL-developed code into saved-in-files code. If I'm using, say, Jupyter Notebook, I'd say: yes, that's exactly an issue I've had and can immediately identify -- but with my CIDER "REPL-driven" flow, that's not an issue at all. You don't need to declare superiority of one to observe that they are different. I am pointing out one highly contextualized advantage, yes, but I did not anoint CIDER the One True Way. There are advantages to Jupyter Notebooks compared to REPL-driven development too. I think calling my commentary a "religious war" is unnecessarily inflammatory.
You ask how I could know that: I have used Python for a similar amount of time as you, and have been a Fellow of the PSF for about a decade. During that time I've attended dozens of PyCons across the world. My wife is the Executive Director of the PSF, so I've been the second person in and the penultimate one out for most of those PyCon USes, including the post-conference sprints. Previously, I was the Freenode group contact for and one of the most active contributors to #python. I've worked with a lot more Python programmers than most, and typically meeting them where _they are_: either pairing next to them or over IRC. People from all over the world and different experience levels and backgrounds. People who lived and breathed Emacs and are definitely aware of how SLIME/inferior processes work. I can't recall a single instance where someone said "eval this in a REPL" and the other party did that by calling reload() or exec from an editor. The vast majority of times someone mentioned reload(), it's been someone who built a long-lived process like an IRC bot, with hot code reloading via reload(), being confused that reload() "didn't work" because they kept seeing the old behavior. YMMV. This is all anecdata, sure, but I observe that IPython 5 casually broke inferior-shell compatibility and somehow that made it to release. It's possible that there are huge swathes of people who do write their own "trivial" reload()/exec integration, but that feels like the stronger claim to me than "there may be confusion here between what users of these languages typically mean when they say REPL".
I want to make sure I understand your argument correctly. Starting with the strongest form of the argument: are you saying you believe most Python/... programmers think "REPL" means "something involving reload" and not, say, ipython? I assume we can agree that seems silly, but perhaps not: and I think that's closest to directly refuting my claim. Unless I'm misreading your comment: I'm saying "different groups use this term to mean different things", and your comment is "come on they are practically the same" (ok, fine, we can disagree!) and "you can do reload()!" (sure, but I didn't say it's impossible, I said that's not what people generally mean). So, perhaps a weaker version of your argument: you're saying you think some nontrivial amount of Python users use a reload/exec-based editor integration? Enough that "REPL" is not suddenly a confusing term? So, something like what, 20%, 10% of users?
Fair point about my religious-war comment, and I apologize. I was probably deserving of the Smug Lisp Weenie label at one point in my career, and for whatever irrational reason your comment sparked those memories. (For what it's worth, while I don't use Clojure any more (and stopped before the CIDER days) I'm fairly familiar with the language -- I was even a core contributor for a brief shining moment. (I think my contribution was the :as clause in (use) statements. The old (use) clauses used to be horrendously long. No doubt that my code has been replaced 10 times over by now.)) At any rate, it was foolish of me to try suppressing a religious war by inciting one.
As for my argument, I think it was in two points. First was to challenge what evidence you had to support your claims. From your reply, I'm confident from that you're speaking from knowledge of many developers' experiences. Perhaps I am more of a special snowflake than I realized.
As for the second point, I'm basically restating Norvig's argument, as I think it's a good one. I'm not terribly concerned about distinctions between different REPL-ish tools. I do take your point that the distinctions can muddy conversation, e.g. when trying to relate experiences of (say) Jupyter users vs. `python-mode` users -- or worse, Lisp vs. Python.
But to paraphrase (co-opt?) Norvig, I think these are second-order differences. Important enough in a few contexts, but distracting in most. My thesis is simple: everyone should experience interactive programming, and should ideally incorporate it into their development process, because it makes development faster and more effective. Whether that means Jupyter, CIDER, SLIME, ghci, etc. is a local decision, based on where you're at and what your community is; I don't think "REPL" in particular is a special shibboleth of the Enlightened. The difference between the Java/C++/Rust set and the Python/Ruby/CL/Clojure set is far greater than any minor differences we might have within the interactive camp, and "internal" disagreements in front of the unindoctrinated tend to distract from the greater good, IMO.
Any way that we can help new developers to experience interactive programming is a positive step. Personally I think that's the first-order problem, and I think our collective, repeated failure to recognize that is a source of much pointless and religious argument. Perhaps that's why this conversation reminded me of those wasted days, and why I responded with more aggression than I should have.
Apology accepted: I completely understand why a reasonable person could infer those things, especially since I'm hardly a neutral party given that I founded a Clojure nonprofit--"partisan" would be an entirely fair assumption :-)
I also happily stipulate that non-Lisps not working that way is largely praxis and not some fundamental language deficiency, and definitely strongly agree that interactive programming is a powerful tool we should get into more hands.
This feels like a straw man. Or maybe a false dichotomy. I think it's entirely possible to ensure code is testable before writing the test. It's also not unusual to tinker in a repl before encoding the functionality as a test. Pretty let down by this post.
Clean Code was one of the first books I read professionally, and I'm glad I did. Since then, I've formed the opinion that "Uncle Bob" sometimes (as in this case) takes helpful ideas a bit too far. But I'm still open to correction.
Does anyone know of an open source codebase he's written or been a major contributor to? I know he writes a lot about clean code, but I've never seen production code from him.
Two completely different things. A npm package is a module of code. A function is... A function. Having a lot of small functions is helpful as it builds up a vocabulary in the codebase, so instead of having to understand every single line each time you come across it, you can extract the meaning into a name.
What the npm ecosystem is doing is a completely different thing.
Yes, those are completely different because I do not release small functions that I compose to bigger functions as separate libraries to be pulled in by the hundreds.
>so instead of having to understand every single line each time you come across it, you can extract the meaning into a name.
Until you really have to understand what is going on, like when you need to fix a bug or refactor.
Then suddenly you to deal with the overhead of holding the branches of 20 levels of indirection in your working memory.
Also, this assumes the people you're working with are great at naming things. Which, from my experience, is almost no one.
In general, we have higher-level programming languages so we can do just this, hide the complexity under indirection. It's basically the purpose of the languages we tend to use today, otherwise we would all be still using assembly.
Your job as a programmer is being able to walk this ladder of indirection up and down until you find what you seek and fix/add/remove whatever.
You'll always be dealing with way more than 20 levels of indirection when programming, even with a "hello world" program. If adding two more levels of indirection makes your mental model break down, you might consider upgrading your environment/workflow a bit to support this.
Regarding naming I agree with you. Good names are really, really, really hard to come up with. Entire books have been written on how to name things. Doesn't excuse us from trying though, the only way we can get better is by doing.
That claim doesn't even pass a simple common sense test: why would Node.js developers be so much more influenced by Bob Martin (of object oriented and Java fame) than developers in other languages?
And it's not even entirely true. You can see that the top ten depended upon packages [1] in npm are the opposite of "literally do one thing". They are either full blown libraries like React or collections of related functions. I'm not claiming npm does not have a problem with too many one liner functions. But the most popular packages do avoid this practice, which is encouraging, and sets the bar for what you should do if you want to be at the top.
What other motivations could there be for being the author of many very small and popular npm packages when it comes to interviews and internet fame? There are probably a lot more reasonable answers than "influenced by Bob Martin".
> I'm not claiming npm does not have a problem with too many one liner functions. But the most popular packages do avoid this practice, which is encouraging, and sets the bar for what you should do if you want to be at the top.
Just put my claim back in context, with an emphasis on the first sentence and you've got your answer.
While I do think that Bob goes overboard with his suggestions in Clean Code, I found it as a necessary counterbalance to encourage readable code in my team, where the methods tends to be gigantic blobs. I'm sure there are better books, but it was a good one.
For instance, one suggestion I did like from Clean Code is to separate exception handling from business logic. I don't always practice it, but it really helped in making my work-related code more readable.
I rarely use REPL driven development, however in my limited experience using it, it only makes integration testing faster, not unit testing.
I used this methodology with a Slack bot, because doing integration testing with Slack is annoying when I'm not directly testing the Slack communication portion.
Edit: It seems like there are a lot of comments surrounding him "abandoning" TDD. Not only did he not do that, he literally says in the conclusion of the article that he would not recommend doing it.
What languages have you tried REPL driven development with? As far as I know, Golang doesn't have a REPL that is connected with your editor so you can evaluate inline code and Golang is not made for REPL driven development either, so you have bunch of implicit state and you can't overwrite function definitions at runtime either.
I think you need to have a language (or structure your particular program in a way) that is really made with live-coding in mind for it to actually give you any benefits, like Smalltalk or Clojure and similar. Otherwise it's just adding another step to reach the real program you're running.
Edit: using your example as an example here, hope you don't mind. A program made with REPL driven development would have way less code in the main function, in order for you to test the code outside with your REPL. The `StoreGame` function would accept a data-store + the argument you have now, so at development-time, you can pass it arbitrary stores. And similar changes.
You probably wouldn't be able to, without restructuring how you're handling your state that is being used there. A REPL driven made program usually is made that the runtime carries the state and exposes it, so you could just select that part and run it, and see the output of it.
I'm guessing delve is more like a traditional debugger that steps through each line, rather than an REPL, correct?
Correct, but it does allow you to modify values in memory, so you could do the branching and state changes as you go. I admit I could have made that flow more "editable" from the debugger/REPL, however it did serve its purpose in terms of prototyping ideas before I integrated them into the real thing. I also do use it to debug weird state things of issues reported to me, which I do mess with the values at runtime with Delve to simulate.
I may be misunderstanding the "true" REPL development here, but I think the points laid out by the article seems to match my experience to a degree.
That's interesting, didn't know that! Thanks for sharing. I don't think there is anything that is more "true" than something else, it's just varying degree of personal productivity and in the end we all work very differently, so one size does not fit all. I'm glad you're sharing your thoughts with me and seems to be similar to the REPL environment I'm running myself.
Indeed, I'm going to think about some of the suggestions you mentioned the next time I'm in there playing with it. Perhaps I'll hit a wall with delve that other language tools (like Clojure) handle better.
I looked into this subject a bit more deeply in regards to Go, and you are right. Go doesn't have a REPL built in, and most of the implemented REPLs only work off of stdin. So, the Clojure style of REPL development is not as straight-forward in Go. However, it does seem like one of the userland REPLs could be adapted to work with an editor (like vscode) if the input source wasn't locked to stdin.
It seems like a ripe target for an opensource library to accomplish this. I may explore this later on when I have more time to devote.
I kind of practice REPL driven development in Go actually. I usually write 1-2 functions at a time, and simply test them by repeatedly running the program, with tests happening in an init() function.
When everything's well again, I move the bits to a test case.
I guess my point would be that with a REPL driven development in Clojure, there is no moving, as the code is already where it should be, you just executed it from there directly.
And you don't need to extract anything into functions, lisp-syntax kind of already gives you all the separation you need, so you can just execute code in place if you need it to.
It's not REPL-driven design, it's REPL-driven development. You don't design by driving a REPL, that would be stupid. It seems Bob has lots of misconception and is blaming his poor design on the REPL. Haven't read such a hollow article in a while.
You still write tests in any non-trivial Clojure codebase, and you still probably do REPL-based development for some parts of that project.
And if you're doing both of those things above, then you almost certainly run some or all of your tests WHILE doing REPL-based development from your always-on REPL. In fact, that's one of the best ways since you don't suffer the JVM startup cost.
This is sort of like how I develop using Jupyter notebooks. Write code in the notebook until I get a function / component correct. Copy it into a module. Restart the kernel and import the module. Rinse and repeat.
Except unlike a REPL, all your code is still there so that you eventually find yourself in an unreproducible state that on the surface looks reproducible. Referencing variables that don't exist in the notebook (only to be caught when run by someone else), cells that need to be executed out of order, etc. The problems go on and on. People sometimes go days or weeks without restarting their notebook and the amount of "shadow" state is remarkable.
I think notebooks are useful (though only for exploratory analysis), but there has to be a better way.
Part of the problem is discipline and avoiding anti-patterns. It's unfortunate that Jupyter encourages this kind of inconsistency to creep in, leaving it up to the analyst to maintain top-to-bottom "runnability".
There's no reason for this, and it bothers me enough that I'm doing something about it.
A (series of) REPL commands is just a test, where acceptance and rejection is manual: we look at the output, and keep changing the code until we get what we want.
At which point, we should be able to say to the REPL: yes, this result is correct, this is a test, and it's now called "Thing does what I want when I give it X and Y".
The runtime then saves this as a test, and will run it automatically when you ask it to. If there's a regression, it will boot back up into the interactive REPL so you can inspect it and fix it.
I've been writing a REPL. Turns out that's a lot of work, but I'm getting to the point where I can add this feature, which is the last one I really want to add before I go public with this thing.
The trouble is that for the repl “test” to be reproducible, you don’t just need to store the relevant commands and result, but the full state/history of the repl. Since the point of a repl is to explore and experiment, it seems like your tests would end up with a lot of noise in them... but perhaps there are ways to get around this? I agree the concept of quickly converting repl sessions to tests is very compelling.
I store the complete repl history, complete with results and session metadata, in a database.
Pruning a session's history down to only what's needed is a manual process, true, and laying the groundwork to provide that feature has been the main yak shave.
Before I get to my real comment, I guess a fair criticism of it would be “Well, where in the [development process] hierarchy do you actually want to live?”, which I guess is a longer discussion, but as a developer, you can’t become battle-hardened without going to battle. So we can nod to Carl Sagan[0] and say “I made this from scratch”, and know there’s value in that. At least occasionally running your REPLs by hand and taking the actual trip, instead of just wanting the destination. It may not be tenable as a full-time production development strategy, but that doesn’t mean it’s worthless.
A common approach would be that the code you're testing / playing with is defined in a file somewhere, and reloading that file resets the REPL state sufficiently to iterate on your tests. e.g. if it does rely on external state, the reload will re-initialize that state. Morally similar to the setup instructions in a unit test.
This sounds extremely useful and touches on one of the big frustrations I have with "REPL driven development".
Say what you will about TDD, but it does (often? sometimes? depends widely on how the tests are being written) produce useful, informative and functional artifacts. If you need to know how to create some entity or how to run some process, chances are there are tests that you can refer to that will show you how things are intended to work. This shouldn't take the place of comprehensive documentation, but it's a welcome counterpart.
With REPL driven development, there's nothing of the sort. If the developer has not provided documentation, you'll probably have to try to reverse engineer a given workflow based on the state of the system, which can be very problematic.
In the specific case I have in mind, there was a business process that involved multiple pieces of the application being used to accomplish a task (creating database entries, etc.). Prior to my inheriting the project, this had been done in an ad hoc manner using the REPL and was not documented. When I had to try to run this process myself, I had nothing to go on.
This is absolutely the result of poor practices and, documentation aside, if the developers had been using tests to drive out this behavior, I'd have had a sort of playbook that I could reference.
That sounds horrible! But this is an operational failure, not a development one. (Well, the one clear development gap is that nobody thought to write a script to replace all those REPL commands. There's no reason that couldn't have been done, that's not a REPL issue at all.)
It's not uncommon for REPL users to write tests for their code. And nobody worth their salt would think that the REPL is a license not to document and operationalize their work properly! Your root cause seems to be some mix of developer laziness or inexperience, and poor business practice -- not really a tooling issue.
What language is your REPL for? Can you inject a repl into a piece of production code, like.
import IPython
IPython.embed()
I have used similar techniques to inject unit tests into the same pass as the integration tests. Basically at this point in the program, pause and run a bunch of unit tests with the internal state setup by the integration test. Requires less mocking, can differentiate between an integration and unit test failure.
It's in LuaJIT, which lets me use first-class environments to separate the client-server relationship while running everything in a single Lua_state.
The (long) roadmap involves implementing the Jupyter protocol, adding a client library which can be injected into production code, and eventually using that common basis to extend it to the other languages supported by Jupyter, and any others which choose to do the groundwork.
Bringing in all of ZeroMQ is a lot of work I've been putting off, but I need it all over the place.
I actually want this for both code and test development. For me the REPL is the first place that sees an implementation- multiple implementations, weighing various design factors- which then gets copy pasted into a source file and then REPL state reloaded (the Clojure "reloaded" workflow), and then the capability offered by the updated implementation exercised with test-like code in the REPL- which then as you describe should magically get into the test suite source for reuse.
Good luck. What language are you doing this work in?
For what it’s worth, here’s a very simplistic/comparatively incomplete attempt at something like that from Cognitect (the Clojure/Datomic folks) a while back:
When I work with Ruby, I often set things up with a script that contacts a method that I can call to trigger reloading of everything and a "replay all known good steps" while storing some state in globals. This is fairly easy in Ruby as you can redefine classes at will. It takes a bit more effort if you want to replicate the state more perfectly, e.g. by deleting methods that have disappeared etc., but that's rarely needed.
I then use "pry", and fire things up while I keep my code in an editor. I can test things in the pry repl, and when I find something that makes sense I add it as an additional method to my code. I usually cut and paste and keep the editor separate, but "pry" does allow triggering your preferred editor directly from the repl and also supports its own code reloading -- I prefer my own mechanism because it gives me more direct control, though.
Regularly I'll exit and run from a fresh state just to make sure I've not accidentally messed up anything with the reloads, but really that's mostly paranoia.
It's not a replacement for tests, but it tends to produce relatively easily testable code because it encourages code that allows you to quickly and painlessly execute steps manually, and so you quickly will e.g. encapsulate mocking steps and setup in suitable helpers etc. because it gets tedious to write them manually.
I like that a whole lot better than TDD, because TDD tests to drive (over)thinking things first vs. experimentation, I feel, because you invest too much in the test cases upfront.
The test cases are important, but it is very easy to end up with a design that is optimized for the tests rather than for your real world use cases. It doesn't have to be that way, and some people manage to do TDD without getting caught up in that, but for me at least, I've found it's much easier to be disciplined about making the code easy to execute in general when I work with it in a REPL on a regular basis, and build tests around that.
The big advantage of TDD is: it guarantees you will write tests. Can't skip tests if you write the tests first.
The big disadvantage, which is why I don't use it, is that you pre-commit to the "correct" function signature and response to some number of parameters. Optimized for the test, as you say.
When I'm working with a repl, it's easy to mutate and experiment with those things until I'm satisfied with them.
But then you still have to translate that into tests, or you get regressions. I'm lazy; I want to skip that part.
That's exactly the sort of test you get with the Selenium browser plugin: click "Record", do some things, verify the end state, click "Stop". You can then replay what is effectively an acceptance test. The problem (and it's a doozy) is that these tests are about as brittle as it gets: change basically anything in the markup and the test will fail. A good unit test will only ever touch a tiny area of the system, and therefore has a correspondingly tiny chance of failing whenever the system changes. And when it does fail because of changes to the rest of the system, it's usually for a good reason, like a changed argument list, not a bad reason like moving a button.
In short, these sorts of tests are super easy to create but become a massive friction at every level of the application since they depend on so much.
Bob Martin is myopic in his view of software, in a way that I think is harmful to teams and the industry.
It is my experience that good software comes from teams working together well, teams that are in some sort of harmony with each other AND the system they are building/using. They understand it and each other. Good software is not derived from a singular technique.
I do like test-first approaches when I can, I like SOLID. I probably agree with Bob on a lot, actually. I think they are good techniques to employ.
However, it has not been uncommon in my career to find people (and myself) getting dogmatic about these things and arguing over them. There are half baked attempts to change everything to an Entity. There are flaky test suites with a lot of mocks because someone is hot on TDD, but does not have the bandwidth to maintain the suite for the whole team.
This blogicle is another example. As pointed out by other posters, REPL driven development can fold VERY well into a test driven workflow. However, that's not enough, he's actively saying it's not as good and you need to do The One True technique.
I think this is true in general. There are very few "always right" ideas in software development--virtually every choice is a function of a broader economic context. For example, if you work for a startup and you need to determine market fit quickly, it's more important to move fast than to have very high degrees of quality (although below a certain threshold, low quality impedes velocity) or performance. Economic context affects virtually everything, and the difference between good and great software engineers is the ability to understand economic context (and changes in economic context) and the technology/process ramifications.
You can’t test everything. To believe we can is just hubris; even if we get every function to have coverage, we won’t have coverage of the full range of inputs unless all that’s being done is some very simple programming. The combinatorial explosion is fast.
Testing definitely has value, but when testing you always have to make the assumption that you can’t test everything, and then the trick is deciding what to test versus what not to. I think people forget this and focus on having every function tested, every interface having coverage, etc.
Quickcheck, clojure.spec and similar efforts are getting damn close to it, and using clojure.spec raised my confidence in my code a lot, more than unit testing ever did. Since generative/property based testing is gaining more ground, we'll get better tools as well.
Not to say there is not still a place for manually written unit tests.
While I am a big fan of property based testing, it’s hardly able to “test everything”. It’s great for testing pure functions, but as soon as things have side effects, property based testing becomes very cumbersome. Unfortunately, this is precisely where a large amount of bugs originate from.
And I say all this as someone who has pursued this very idea of property testing side effects for years and in the end had to admit failure. It just is too much work get right (eg uuids being generated not unique, to name just one example), and your test runs become incredibly slow, which is a killer in many REPL-driven workflows.
I have a (probably horrible) joke reply to the question "what is a monad?" I say "that's where you put the real programming." I have also never done any serious with Haskell, so don't make the mistake of taking me seriously on this :)
however true the statement may be, it's not therefore an abandonment of pure functional programming. You don't have to program in `IO` or `MonadIO m => m`.
Expressing your state in terms of pure functions of input events, as can be done with an FRP like Reflex, means you can test all your logic purely. You can be confident that you'll only request to launch the nukes when the nukes need to be launched.
Then you have to make yourself confident that, when the system asks the nukes to launch, it sends the right signal. But that's mockable by running the system attached to device that reads the signal and reports its correctness.
And making sure the nukes take off, aim correctly, and land on their target is not my department (says Werner von Braun).
If you can't capture the side effects to determine if they're correct, you can't test it period. Property based testing, unit testing, or integration testing, doesn't matter.
People believe that even if you don’t say it, however. While TDD I’m sure started with good intentions, the reason it hasn’t seen as wide adoption as it could have, in my opinion, is that it is frequently stated as requiring that everything be developed from tests.
I find it much more convincing if someone were to say, “test what’s easy” instead, that’s all.
But Bob didn't take a moment to see if or how he could integrate his existing knowledge into this new way of doing things, or ask if there might be something he's not seeing. I've taken those concerns to people in the clojure community, and they've been accommodating.
That there are drawbacks to the REPL feedback loop is true. But they aren't the drawbacks Bob claims to have found here, because he ceased investigating at the point he felt his priors were confirmed.
A set of principles for program design, specifically OO (and some flavors of OO at that), but some of the ideas are general enough to apply to most languages and paradigms.
I have seen Bob a couple of times in person. He is a good speaker, his rhetoric is strong, but he has some pretty large blind spots. He out of hand writes off formal methods and type systems, which is ridiculous given his stated goals around software quality.
I remember watching a talk of Bob's where he pumped up how good his approach to coding was, and when asked to provide examples, he could/would/did not provide any. Seriously, if you can't demonstrate what you're talking about, I'm just gonna take you less seriously.
People like this need to be put into managing a team of visual basic devs at an obscure local DMV IT department, and force them to make projects after projects until they realize that if they want something done in those situations, they have to get dumber.
Having been on the receiving end of code reviews where the reviewer uses clean code dogma against me like a weapon of mass destruction, I am much more a fan of letting the small stuff that can't be easily caught via automation slide. There's no one true technique, even if you are coding for yourself. There just isn't. That's like saying there's one true way to paint, and code is more art than science.
Well, there are helpful code / design review comments that can't be automated. I totally agree that if it's a tiny issue, just let it go. There's only so much feedback people can hear and actually respond to. But there are also helpful things to say.
> Bob Martin is myopic in his view of software, in a way that I think is harmful to teams and the industry.
His lectures and books really are harmful. Beginners read them and get the impression that you can design good software by following a bunch of simplistic rules of thumb. In reality, you need to understand what trade-offs you're making when committing to certain design decisions and, even more importantly, you need to be able to discuss this with other people.
For example, it's not "small functions are good", it's "when you make a function smaller, here are the benefits and here are costs". Primitive, sloganistic thinking is the bane of software engineering profession right now.
I don't think he's harmful. I think you take the good with the bad with him. He's way too convicted to trust blindly, but he also talks about some great topics that aren't widely discussed. So, if you consider him the leader of a religion, you'll be led astray. But if you just read critically, you'll see that he definitely has a lot of insight to take away. Just discard the stuff that you don't agree with.
I think Bob is a very insightful person and has great intuition and probably more experience than any of us. Sure he has certain bias towards certain things; who doesn't really?
Saying that his posts are harmful is a bit of a stretch.
Even smart, highly qualified people (which Bob is) are prone to cognitive bias. One such bias is, the more you have espoused something, the more fervently you believe in it, and the harder it is for you to acknowledge any shortcomings of it.
So I agree with the parent ... his writings are in many cases harmful. They're a mix of good and bad, but often net harmful.
I think Bob is a very insightful person and has great intuition and probably more experience than any of us.
Why on earth would you think that?
Saying that his posts are harmful is a bit of a stretch.
Mentor your first few people who keep defending rookie mistakes with "But Uncle Bob said..." and you might change your mind about that.
Bob Martin says some sensible things, but he also says some things that are at best debatable and at worst outright contradicted by the evidence base, and he says all of them with great conviction and presumed authority so that his target audience can't tell the difference. That alone makes him dangerous as a source of advice.
I have been developing for 54 years. I have been through all manner of design philosophies, from Structured Programming, to Object Oriented Programming, to Agile, to TDD along with hack-it-till-it-works.
If you don't do the engineering, which he does not teach, you are going to struggle.
Bob Martin belongs to the OO-evangelism camp that emerged from the OO fad of mid-1990s.
Another OO-evanglist who belongs to this camp is Ron Jeffries.
For a textbook example of how much of a snake-oil OO-evangelism is, checkout the sudoku solver fiasco involving Ron Jeffries and Peter Norvig (you'll find plenty of search results regarding this).
I’m no fan of Bob Martin, but the Ron Jeffries fiasco was a failure of TDD, not of OOP. The meandering inability to focus on writing a solution, instead becoming lost in modelling the problem, is a hallmark of the tunnel vision that TDD engenders. He’d have screwed it up in Prolog.
Last year I came up with a new term for a type of developer: The Medium Developer. Medium in that they have a medium amount of experience (3-7 years, not that all in that range fall in this category), and Medium in that they have a medium amount of talent (not bad, not great), and Medium in that the religiously consume and spout Medium articles. They are the most zealous type of developer I've encountered. They believe TDD is the One True Way, or they think CI/CD with iterations in the minutes where you constantly commit to master prevents all bugs, or that SOLID is the word of the developer gods.
They are awful to work with and be around. All of those concept I mentioned have benefits in them, but not as a religion. Medium programmers lack nuance and deep experience and are dismissive of those that do, they see it as weakness. When their favorite technique doesn't take them to the holy Nirvana of software, they lash out at other, less zealous developers and blame their lack of discipline.
Completely shitty people - I just hope they grow out of it. But they are seductive in their intensity and (bad) managers love to hear how they have a simple solution to all their woes - rather than software development being inherently difficult (because you only have to do a specific hard things once, but that's another discussion!)
This is a PEBKAC, not a problem with REPL-driven development. He had all the test bodies written as input in the REPL, he had all the test results printed as output in the REPL. Why didn't he turn these into unit tests as he programmed?
That's a simple lack of foresight on the programmer's side that is then blamed on the tools that were used.
Using a REPL has been critical for me to develop any meaningful software (more than tests and more than types). I always wanted to be able to take a portion of my REPL history and transform it into a test, instead of basically rewriting it (lazy, I know).
This was the programming model I learned on the lisp machine and Common Lisp: define a few datastructures and some operations on them, explore them with some live data, and expand. All iteratively at the REPL.
A few years later a friend referred to this dismissively as “Programming by successive approximation.”
There is a lot of truth to this, but the distinction isn’t as sharp as it sounds: you still have to think ahead, and as you explore and expand you’ve typically always got something working, some test cases and invariants, and typically you have the benefit of working on live data.
Even though I do most of my work in compiled C++, my work model isn’t that different.
> So I’ve learned my lesson. REPL driven development feels easier and faster than TDD; but it is not. Next time, it’s back to TDD for me.
In the long run it's not faster. However, the feature he was using REPL driven development for did ship faster. That's a valid tradeoff to make, as long as you're doing it consciously.
Is this just the equivalent of saying "Hey, REPLs are nice when you're trying to figure something out", but spouted off as some deep programming wisdom?
Like, folks... REPLs are really nice when you want to figure out how a function or module or whatever works. Then yeah... you can take that knowledge you gained and apply it to your work.
Like.. how is this a deep profound revelation for folks?
REPL isn't new. It's an old term, coined I don't know when but certainly a long time ago as Lisps have used it for decades. And yes, your shell or terminal is a REPL.
It needs a name so that we can have this discussion and compare and contrast different methodologies.
For example
* Compile run debug. You write some code potentially with tests. Run it and either poke it manually or run tests to see what it did. If it didn't do what was expected you either edit it, write some more tests, add print statements, add break points or whatever and try again.
You are forced to create all state in one go and pay one complete edit build debug cycle for each thing you want to modify in isolation. This could be 30 seconds or hours.
* Minimal interactive development. You mostly write code as above but gain the ability to poke individual functions or pieces of functionality in order to aid your understanding when things don't work correctly. You write tests eventually or not.
* Repl driven development. You mostly write and interact with a live environment either by actually typing the code in a repl or writing it into a source code file and hitting a shortcut to evaluate as you make changes redefining it piece by piece in terms of the rest of the code and the state you have built up by prior evaluations. This persistent state and the ability to redefine parts of it are what separates it from writing some python and occasionally opening up ipython to poke the code.
Why not compose with the repl, then write tests at the end?
The common fear hear is that “you be motivated to write the test, or the test will be less good”
For motivation I never found it to be a big issue, and for the quality of the test, writing it after often produces better results. You get more time to think about _what and how_ you want to test
Bob Martin, who is quite opinionated, has been brutally wrong about some things. For example, there was a TDD attempt to solve Sudoku that failed. Uncle Bob was following this, and said of the failure "At least he had the courage to post his failure".
More interesting would be a correct solution, which was posted by Peter Norvig. The approach he uses is drastically different that anything TDD is likely to get for you.
TDD isn't going to find solutions to hard problems beyond bowling.
There is a stark contrast between agile development and actual software engineering. Agile works where the customer and the developer don't really know what is being developed.
And as much as he has lectured about software development, I don't get the feeling that he has actually developed a substantial quantity of serious software.
In one of his clean coder books, he argues that Java is not object oriented. Unclear if that is a widely-held opinion. If you look for serious high-quality software, they are not agile. Such as the software for the space shuttle, or qmail, or other formally-verified software.
I have a friend who worked with him and it was sort of Uncle Bob's way or the highway.
143 comments
[ 2.7 ms ] story [ 191 ms ] threadWhich gets back to the posted article. REPL-driven development permitted him to feel confident without doing his usual process, it didn't discourage decoupling, but it didn't encourage decoupling either. TDD, to be done well, almost requires decoupling your code to make it more testable.
The same may (I'm not sure I agree, but I've never used Jupyter notebooks) be true of using notebook-styled development approaches and modularity. If the workflow permits relatively easy (and mostly safe) development without encouraging modularity, then it's incumbent on the developer to bring that discipline into the workflow.
[1] https://github.com/fastai/nbdev
As an idea, maybe we could do... both? Explore at the REPL, and then commit to tests and clean up once you've solidified your approach (i.e. how most Clojurists likely operate).
If I don't have a REPL, I often write a lot of little throwaway tests to deal with intermediate bits of the code. They end up being redundant with the real tests. They're often really bad tests, in that they're tightly coupled to implementation details. I should delete them. And I often forget to. Which creates extra maintenance burden for myself and my teammates.
If I do have a REPL, I do all of that extra intermediate verification junk in the REPL. It's often more effective, it's usually less effort, and it always reduces the volume of unnecessary brittle tests in the codebase.
The usual workflow was something like
Once you have what you want, you move the code out of the comment block. Whether you move it into a library or a test file is just dependent on the type of thing you're moving. Functions moved into library code. The samples that proved the happy paths (or unhappy paths) would go into test files using `clojure.test/is` (and typically a direct copy/paste).(I have no idea if an equivalent workflow exists in Elxir, but I've never seen it been done that way: just like people use ipython, they use iex in my experience.)
Maybe I'm just misunderstanding what people in Clojure are doing. Please feel to point me in the right direction.
a) Knowing both how to effectively write tests & how to effectively use a REPL.
and:
b) Knowing only how to bang around in a REPL enough to use it as a crutch to avoid writing any tests.
REPL is definitely not REPL.
[] I really don't think it is: reload(mod) comes with way more caveats than C-c C-k in CIDER.
You're making far too much of the distinction between different kinds of REPL. As Peter Norvig put it [1], as long as you have a "development process that allows for rapid development with continuous improvement", minor distinctions between languages [such as quibbles over how 'reload' works] really don't matter that much.
Squeeze all the power out of the tools that you use, and get things done -- ultimately that's all that matters. Religious wars like these were tiresome in the era of the Smug Lisp Weenie, and they haven't improved with age.
[1] https://news.ycombinator.com/item?id=1803815
You ask how I could know that: I have used Python for a similar amount of time as you, and have been a Fellow of the PSF for about a decade. During that time I've attended dozens of PyCons across the world. My wife is the Executive Director of the PSF, so I've been the second person in and the penultimate one out for most of those PyCon USes, including the post-conference sprints. Previously, I was the Freenode group contact for and one of the most active contributors to #python. I've worked with a lot more Python programmers than most, and typically meeting them where _they are_: either pairing next to them or over IRC. People from all over the world and different experience levels and backgrounds. People who lived and breathed Emacs and are definitely aware of how SLIME/inferior processes work. I can't recall a single instance where someone said "eval this in a REPL" and the other party did that by calling reload() or exec from an editor. The vast majority of times someone mentioned reload(), it's been someone who built a long-lived process like an IRC bot, with hot code reloading via reload(), being confused that reload() "didn't work" because they kept seeing the old behavior. YMMV. This is all anecdata, sure, but I observe that IPython 5 casually broke inferior-shell compatibility and somehow that made it to release. It's possible that there are huge swathes of people who do write their own "trivial" reload()/exec integration, but that feels like the stronger claim to me than "there may be confusion here between what users of these languages typically mean when they say REPL".
I want to make sure I understand your argument correctly. Starting with the strongest form of the argument: are you saying you believe most Python/... programmers think "REPL" means "something involving reload" and not, say, ipython? I assume we can agree that seems silly, but perhaps not: and I think that's closest to directly refuting my claim. Unless I'm misreading your comment: I'm saying "different groups use this term to mean different things", and your comment is "come on they are practically the same" (ok, fine, we can disagree!) and "you can do reload()!" (sure, but I didn't say it's impossible, I said that's not what people generally mean). So, perhaps a weaker version of your argument: you're saying you think some nontrivial amount of Python users use a reload/exec-based editor integration? Enough that "REPL" is not suddenly a confusing term? So, something like what, 20%, 10% of users?
As for my argument, I think it was in two points. First was to challenge what evidence you had to support your claims. From your reply, I'm confident from that you're speaking from knowledge of many developers' experiences. Perhaps I am more of a special snowflake than I realized.
As for the second point, I'm basically restating Norvig's argument, as I think it's a good one. I'm not terribly concerned about distinctions between different REPL-ish tools. I do take your point that the distinctions can muddy conversation, e.g. when trying to relate experiences of (say) Jupyter users vs. `python-mode` users -- or worse, Lisp vs. Python.
But to paraphrase (co-opt?) Norvig, I think these are second-order differences. Important enough in a few contexts, but distracting in most. My thesis is simple: everyone should experience interactive programming, and should ideally incorporate it into their development process, because it makes development faster and more effective. Whether that means Jupyter, CIDER, SLIME, ghci, etc. is a local decision, based on where you're at and what your community is; I don't think "REPL" in particular is a special shibboleth of the Enlightened. The difference between the Java/C++/Rust set and the Python/Ruby/CL/Clojure set is far greater than any minor differences we might have within the interactive camp, and "internal" disagreements in front of the unindoctrinated tend to distract from the greater good, IMO.
Any way that we can help new developers to experience interactive programming is a positive step. Personally I think that's the first-order problem, and I think our collective, repeated failure to recognize that is a source of much pointless and religious argument. Perhaps that's why this conversation reminded me of those wasted days, and why I responded with more aggression than I should have.
I also happily stipulate that non-Lisps not working that way is largely praxis and not some fundamental language deficiency, and definitely strongly agree that interactive programming is a powerful tool we should get into more hands.
Does anyone know of an open source codebase he's written or been a major contributor to? I know he writes a lot about clean code, but I've never seen production code from him.
Pretty sure he hasn’t written prod code in over a decade.
The general point that code should be clean is a good point to make. Writing a whole book about it seems a bit much.
Yup which is why we have millions of npm packages that do literally one thing.
What the npm ecosystem is doing is a completely different thing.
There’s no difference between module and function when you see that.
Until you really have to understand what is going on, like when you need to fix a bug or refactor. Then suddenly you to deal with the overhead of holding the branches of 20 levels of indirection in your working memory.
Also, this assumes the people you're working with are great at naming things. Which, from my experience, is almost no one.
Your job as a programmer is being able to walk this ladder of indirection up and down until you find what you seek and fix/add/remove whatever.
You'll always be dealing with way more than 20 levels of indirection when programming, even with a "hello world" program. If adding two more levels of indirection makes your mental model break down, you might consider upgrading your environment/workflow a bit to support this.
Regarding naming I agree with you. Good names are really, really, really hard to come up with. Entire books have been written on how to name things. Doesn't excuse us from trying though, the only way we can get better is by doing.
And it's not even entirely true. You can see that the top ten depended upon packages [1] in npm are the opposite of "literally do one thing". They are either full blown libraries like React or collections of related functions. I'm not claiming npm does not have a problem with too many one liner functions. But the most popular packages do avoid this practice, which is encouraging, and sets the bar for what you should do if you want to be at the top.
What other motivations could there be for being the author of many very small and popular npm packages when it comes to interviews and internet fame? There are probably a lot more reasonable answers than "influenced by Bob Martin".
[1] https://www.npmjs.com/browse/depended
https://www.npmjs.com/package/is-promise
7 million downloads a week is not "popular" to you?
npm is riddled with packages like these.
Just put my claim back in context, with an emphasis on the first sentence and you've got your answer.
While I do think that Bob goes overboard with his suggestions in Clean Code, I found it as a necessary counterbalance to encourage readable code in my team, where the methods tends to be gigantic blobs. I'm sure there are better books, but it was a good one.
For instance, one suggestion I did like from Clean Code is to separate exception handling from business logic. I don't always practice it, but it really helped in making my work-related code more readable.
I used this methodology with a Slack bot, because doing integration testing with Slack is annoying when I'm not directly testing the Slack communication portion.
For example: https://github.com/cjsaylor/chessbot/blob/master/cmd/repl/ma...
Edit: It seems like there are a lot of comments surrounding him "abandoning" TDD. Not only did he not do that, he literally says in the conclusion of the article that he would not recommend doing it.
I think you need to have a language (or structure your particular program in a way) that is really made with live-coding in mind for it to actually give you any benefits, like Smalltalk or Clojure and similar. Otherwise it's just adding another step to reach the real program you're running.
Edit: using your example as an example here, hope you don't mind. A program made with REPL driven development would have way less code in the main function, in order for you to test the code outside with your REPL. The `StoreGame` function would accept a data-store + the argument you have now, so at development-time, you can pass it arbitrary stores. And similar changes.
You can still get proper editor support with Golang (even though it isn't designed for it). I've done so here with VSCode working with a "remote" delve session: https://github.com/cjsaylor/chessbot/blob/11e1059aa77fed84d2...
My point being, if you wanted to execute the following code: https://github.com/cjsaylor/chessbot/blob/11e1059aa77fed84d2...
You probably wouldn't be able to, without restructuring how you're handling your state that is being used there. A REPL driven made program usually is made that the runtime carries the state and exposes it, so you could just select that part and run it, and see the output of it.
I'm guessing delve is more like a traditional debugger that steps through each line, rather than an REPL, correct?
I may be misunderstanding the "true" REPL development here, but I think the points laid out by the article seems to match my experience to a degree.
It seems like a ripe target for an opensource library to accomplish this. I may explore this later on when I have more time to devote.
When everything's well again, I move the bits to a test case.
And you don't need to extract anything into functions, lisp-syntax kind of already gives you all the separation you need, so you can just execute code in place if you need it to.
But TDD is not design either, in my opinion.
And if you're doing both of those things above, then you almost certainly run some or all of your tests WHILE doing REPL-based development from your always-on REPL. In fact, that's one of the best ways since you don't suffer the JVM startup cost.
I think notebooks are useful (though only for exploratory analysis), but there has to be a better way.
A (series of) REPL commands is just a test, where acceptance and rejection is manual: we look at the output, and keep changing the code until we get what we want.
At which point, we should be able to say to the REPL: yes, this result is correct, this is a test, and it's now called "Thing does what I want when I give it X and Y".
The runtime then saves this as a test, and will run it automatically when you ask it to. If there's a regression, it will boot back up into the interactive REPL so you can inspect it and fix it.
I've been writing a REPL. Turns out that's a lot of work, but I'm getting to the point where I can add this feature, which is the last one I really want to add before I go public with this thing.
I store the complete repl history, complete with results and session metadata, in a database.
Pruning a session's history down to only what's needed is a manual process, true, and laying the groundwork to provide that feature has been the main yak shave.
Before I get to my real comment, I guess a fair criticism of it would be “Well, where in the [development process] hierarchy do you actually want to live?”, which I guess is a longer discussion, but as a developer, you can’t become battle-hardened without going to battle. So we can nod to Carl Sagan[0] and say “I made this from scratch”, and know there’s value in that. At least occasionally running your REPLs by hand and taking the actual trip, instead of just wanting the destination. It may not be tenable as a full-time production development strategy, but that doesn’t mean it’s worthless.
[0] https://youtu.be/7s664NsLeFM
Say what you will about TDD, but it does (often? sometimes? depends widely on how the tests are being written) produce useful, informative and functional artifacts. If you need to know how to create some entity or how to run some process, chances are there are tests that you can refer to that will show you how things are intended to work. This shouldn't take the place of comprehensive documentation, but it's a welcome counterpart.
With REPL driven development, there's nothing of the sort. If the developer has not provided documentation, you'll probably have to try to reverse engineer a given workflow based on the state of the system, which can be very problematic.
Now imagine that you can instantly load a REPL session with complete tests, already executed, hanging out in the session history.
Now you can change anything in the test and see what it does. That's the idea anyway.
Wouldn't there be production code that you can read? You're already working in the codebase after all.
I agree that a unit test might have a cleaner, distilled setup procedure though.
This is absolutely the result of poor practices and, documentation aside, if the developers had been using tests to drive out this behavior, I'd have had a sort of playbook that I could reference.
It's not uncommon for REPL users to write tests for their code. And nobody worth their salt would think that the REPL is a license not to document and operationalize their work properly! Your root cause seems to be some mix of developer laziness or inexperience, and poor business practice -- not really a tooling issue.
The (long) roadmap involves implementing the Jupyter protocol, adding a client library which can be injected into production code, and eventually using that common basis to extend it to the other languages supported by Jupyter, and any others which choose to do the groundwork.
Bringing in all of ZeroMQ is a lot of work I've been putting off, but I need it all over the place.
https://elixir-lang.org/getting-started/mix-otp/docs-tests-a...
https://docs.python.org/3/library/doctest.html
I actually want this for both code and test development. For me the REPL is the first place that sees an implementation- multiple implementations, weighing various design factors- which then gets copy pasted into a source file and then REPL state reloaded (the Clojure "reloaded" workflow), and then the capability offered by the updated implementation exercised with test-like code in the REPL- which then as you describe should magically get into the test suite source for reuse.
Good luck. What language are you doing this work in?
https://github.com/cognitect-labs/transcriptor
When I work with Ruby, I often set things up with a script that contacts a method that I can call to trigger reloading of everything and a "replay all known good steps" while storing some state in globals. This is fairly easy in Ruby as you can redefine classes at will. It takes a bit more effort if you want to replicate the state more perfectly, e.g. by deleting methods that have disappeared etc., but that's rarely needed.
I then use "pry", and fire things up while I keep my code in an editor. I can test things in the pry repl, and when I find something that makes sense I add it as an additional method to my code. I usually cut and paste and keep the editor separate, but "pry" does allow triggering your preferred editor directly from the repl and also supports its own code reloading -- I prefer my own mechanism because it gives me more direct control, though.
Regularly I'll exit and run from a fresh state just to make sure I've not accidentally messed up anything with the reloads, but really that's mostly paranoia.
It's not a replacement for tests, but it tends to produce relatively easily testable code because it encourages code that allows you to quickly and painlessly execute steps manually, and so you quickly will e.g. encapsulate mocking steps and setup in suitable helpers etc. because it gets tedious to write them manually.
I like that a whole lot better than TDD, because TDD tests to drive (over)thinking things first vs. experimentation, I feel, because you invest too much in the test cases upfront.
The test cases are important, but it is very easy to end up with a design that is optimized for the tests rather than for your real world use cases. It doesn't have to be that way, and some people manage to do TDD without getting caught up in that, but for me at least, I've found it's much easier to be disciplined about making the code easy to execute in general when I work with it in a REPL on a regular basis, and build tests around that.
The big disadvantage, which is why I don't use it, is that you pre-commit to the "correct" function signature and response to some number of parameters. Optimized for the test, as you say.
When I'm working with a repl, it's easy to mutate and experiment with those things until I'm satisfied with them.
But then you still have to translate that into tests, or you get regressions. I'm lazy; I want to skip that part.
In short, these sorts of tests are super easy to create but become a massive friction at every level of the application since they depend on so much.
It is my experience that good software comes from teams working together well, teams that are in some sort of harmony with each other AND the system they are building/using. They understand it and each other. Good software is not derived from a singular technique.
I do like test-first approaches when I can, I like SOLID. I probably agree with Bob on a lot, actually. I think they are good techniques to employ.
However, it has not been uncommon in my career to find people (and myself) getting dogmatic about these things and arguing over them. There are half baked attempts to change everything to an Entity. There are flaky test suites with a lot of mocks because someone is hot on TDD, but does not have the bandwidth to maintain the suite for the whole team.
This blogicle is another example. As pointed out by other posters, REPL driven development can fold VERY well into a test driven workflow. However, that's not enough, he's actively saying it's not as good and you need to do The One True technique.
Testing definitely has value, but when testing you always have to make the assumption that you can’t test everything, and then the trick is deciding what to test versus what not to. I think people forget this and focus on having every function tested, every interface having coverage, etc.
Not to say there is not still a place for manually written unit tests.
And I say all this as someone who has pursued this very idea of property testing side effects for years and in the end had to admit failure. It just is too much work get right (eg uuids being generated not unique, to name just one example), and your test runs become incredibly slow, which is a killer in many REPL-driven workflows.
Expressing your state in terms of pure functions of input events, as can be done with an FRP like Reflex, means you can test all your logic purely. You can be confident that you'll only request to launch the nukes when the nukes need to be launched.
Then you have to make yourself confident that, when the system asks the nukes to launch, it sends the right signal. But that's mockable by running the system attached to device that reads the signal and reports its correctness.
And making sure the nukes take off, aim correctly, and land on their target is not my department (says Werner von Braun).
The problem with that (tautological) statement is that too many people read it as: "you can't test everything, so don't bother testing anything".
I find it much more convincing if someone were to say, “test what’s easy” instead, that’s all.
1. stumble over differences from their normal approach
2. catch on and use it everywhere (Stockholm syndrome? Testing limits?)
3. finally find places it doesn't work and can apply judiciously
Given that in our profession we're constantly learning and experimenting, Uncle Bob is still doing that to this day.
That there are drawbacks to the REPL feedback loop is true. But they aren't the drawbacks Bob claims to have found here, because he ceased investigating at the point he felt his priors were confirmed.
A set of principles for program design, specifically OO (and some flavors of OO at that), but some of the ideas are general enough to apply to most languages and paradigms.
Testing is his singular solution for everything.
I think it was this video, but I can't watch it all the way through right now -- https://youtu.be/ecIWPzGEbFc
His lectures and books really are harmful. Beginners read them and get the impression that you can design good software by following a bunch of simplistic rules of thumb. In reality, you need to understand what trade-offs you're making when committing to certain design decisions and, even more importantly, you need to be able to discuss this with other people.
For example, it's not "small functions are good", it's "when you make a function smaller, here are the benefits and here are costs". Primitive, sloganistic thinking is the bane of software engineering profession right now.
Saying that his posts are harmful is a bit of a stretch.
So I agree with the parent ... his writings are in many cases harmful. They're a mix of good and bad, but often net harmful.
Look at how much growth happened in the last 15 years.
Why on earth would you think that?
Saying that his posts are harmful is a bit of a stretch.
Mentor your first few people who keep defending rookie mistakes with "But Uncle Bob said..." and you might change your mind about that.
Bob Martin says some sensible things, but he also says some things that are at best debatable and at worst outright contradicted by the evidence base, and he says all of them with great conviction and presumed authority so that his target audience can't tell the difference. That alone makes him dangerous as a source of advice.
I have been developing for 54 years. I have been through all manner of design philosophies, from Structured Programming, to Object Oriented Programming, to Agile, to TDD along with hack-it-till-it-works.
If you don't do the engineering, which he does not teach, you are going to struggle.
I believe this is a harmful post.
Another OO-evanglist who belongs to this camp is Ron Jeffries.
For a textbook example of how much of a snake-oil OO-evangelism is, checkout the sudoku solver fiasco involving Ron Jeffries and Peter Norvig (you'll find plenty of search results regarding this).
They are awful to work with and be around. All of those concept I mentioned have benefits in them, but not as a religion. Medium programmers lack nuance and deep experience and are dismissive of those that do, they see it as weakness. When their favorite technique doesn't take them to the holy Nirvana of software, they lash out at other, less zealous developers and blame their lack of discipline.
Completely shitty people - I just hope they grow out of it. But they are seductive in their intensity and (bad) managers love to hear how they have a simple solution to all their woes - rather than software development being inherently difficult (because you only have to do a specific hard things once, but that's another discussion!)
That's a simple lack of foresight on the programmer's side that is then blamed on the tools that were used.
A few years later a friend referred to this dismissively as “Programming by successive approximation.”
There is a lot of truth to this, but the distinction isn’t as sharp as it sounds: you still have to think ahead, and as you explore and expand you’ve typically always got something working, some test cases and invariants, and typically you have the benefit of working on live data.
Even though I do most of my work in compiled C++, my work model isn’t that different.
Being able to poke and prod at a system is priceless for debugging and introspection.
In the long run it's not faster. However, the feature he was using REPL driven development for did ship faster. That's a valid tradeoff to make, as long as you're doing it consciously.
Like, folks... REPLs are really nice when you want to figure out how a function or module or whatever works. Then yeah... you can take that knowledge you gained and apply it to your work.
Like.. how is this a deep profound revelation for folks?
Isn't this just...using a computer?
I don't know; maybe I missed something.
For example
* Compile run debug. You write some code potentially with tests. Run it and either poke it manually or run tests to see what it did. If it didn't do what was expected you either edit it, write some more tests, add print statements, add break points or whatever and try again.
You are forced to create all state in one go and pay one complete edit build debug cycle for each thing you want to modify in isolation. This could be 30 seconds or hours.
* Minimal interactive development. You mostly write code as above but gain the ability to poke individual functions or pieces of functionality in order to aid your understanding when things don't work correctly. You write tests eventually or not.
* Repl driven development. You mostly write and interact with a live environment either by actually typing the code in a repl or writing it into a source code file and hitting a shortcut to evaluate as you make changes redefining it piece by piece in terms of the rest of the code and the state you have built up by prior evaluations. This persistent state and the ability to redefine parts of it are what separates it from writing some python and occasionally opening up ipython to poke the code.
Why not compose with the repl, then write tests at the end?
The common fear hear is that “you be motivated to write the test, or the test will be less good”
For motivation I never found it to be a big issue, and for the quality of the test, writing it after often produces better results. You get more time to think about _what and how_ you want to test
More interesting would be a correct solution, which was posted by Peter Norvig. The approach he uses is drastically different that anything TDD is likely to get for you.
TDD isn't going to find solutions to hard problems beyond bowling.
There is a stark contrast between agile development and actual software engineering. Agile works where the customer and the developer don't really know what is being developed.
And as much as he has lectured about software development, I don't get the feeling that he has actually developed a substantial quantity of serious software.
In one of his clean coder books, he argues that Java is not object oriented. Unclear if that is a widely-held opinion. If you look for serious high-quality software, they are not agile. Such as the software for the space shuttle, or qmail, or other formally-verified software.
I have a friend who worked with him and it was sort of Uncle Bob's way or the highway.