Most of my projects are born out of scribbles on paper, system diagrams with boxes and little arrows to try to describe the system I want to build or my thought process written down like in this post.
When I get it on paper and everything feels right I start creating the codebase but I have to get it down first, feels like I remember it more after I save my thoughts to external paper storage.
I have over the decades tried every combination I can think up or have read about for designing things and I always land back in the same place, A4 5mm square pad (cheap but not the cheapest) and a really good (but not expensive) set of ballpoints in black/blue/red/green (and mostly it's black anyway).
There is something about the complete free form nature of pen and paper that makes me think better, I bought a nice ipad/pencil a few years ago thinking that would replicate it but it didn't, it was closer but not close.
If it's something I want to keep I just take a photo but vast majority of notes are transient anyway.
I spent an unreasonable amount of time recently thinking about how to color- (and symbol-) code my notes. I wanted to create a simple generally applicable system that I can use for any project.
For thinking, nothing beats handwriting. It's just the unconstrained freedom of movement in every direction, shape or form, and the lack of friction and distractions.
But then you sometime also want to organize or share stuff, so that's where digital comes in. I have an automation that takes a photo of my notes with my phone, uploads it to Obsidian, and OCRs it into markdown and tags it, so I basically end up with a "symbolic" 1-1 equivalent of analog handwriting + digital text
My system still evolves, but what I currently use is:
Color:
- black = just "normal" text
- red = critical/"must"
- yellow = important/"should"
- green = highlighted/"could"
Symbols:
- bulletpoint
[ ] task
-> reminder
I also do brackets and underlines which translate to # heading1 and ## heading2 in markdown
I write by hand with a fountain pen with black ink + use 3 fineliners and small colored sticky flags.
Some observations:
- yellow is not great on white paper due to bad contrast; it also rarely comes as ballpoint, but fineliner options are available
- blue and green are visually very similar, to the point of kinda being redundant
- for sharing with other people, the system must be as generally understandable as possible. Here the traffic norms such as ISO 7010 (https://en.wikipedia.org/wiki/ISO_7010) could be considered
- also there's the aspect of accessibility, with a relevant portion of population being colorblind
- The simplest most effective system seems to be: one main color (black or blue) and one highlight color (red I think works best), plus a couple of symbols such as bullet point, task, reminder that translate well into markdown
I use the "uni-ball - Jetstream SXN-210" (in 1mm), they are superb, don't smear, write smooth every time and are cheap enough if I lend one to someone and don't get it back I don't regret it, ergonomically they are comfortable as well, not a lot else to say about them beyond that - just a really well designed pen.
Agreed, I used to role my eyes when my professors asked me to do things like pseudocode (which I still don't do too much) but the act of writing an outline for the general structure of a program always helped me get over the initial hump of starting a new project.
When I had started my PhD, I thought the writing part of a paper was just the last thing I needed to do to summarize my results, like a boring epilogue to the more exciting science. What surprised me was how much of the scientific process relied on my ability to convey my ideas to someone else. And to be able to do that you need clarity of mind, you need to have worked through multiple drafts to be able to get to your point as efficiently and clearly as possible. So many times I only discovered what I was actually researching during the writing part.
The problem with AI writing is that it makes people believe that writing is just writing and not a tool for thinking very deeply about the topic you are writing about. I am not completely opposed to AI assisted writing, but if I see AI writing, it’s telling me that the author tried to take a shortcut in their own thinking and it’s a good heuristic that the ideas will not be thought through enough to warrant my complete attention.
This happened to me recently. I just asked what the prompt was. I found it really tough to read AI words, not knowing the weight to put in each word. A almost gave it more credit than was due and responded in detail, before seeing the prompt and responding barely at all - equivalent to the thought put in.
>it’s telling me that the author tried to take a shortcut in their own thinking and it’s a good heuristic that the ideas will not be thought through enough to warrant my complete attention.
Even worse: Facing AI writing, there's a strong suspicion that any concerns or counterpoints you bring won't be actually considered, but rather copypasted into an llm session.
I feel that way about coding. Sometimes you have an idea and a plan to implement it. But once you start coding you realize that some things were not fully right or way more complex that it should
If you let AI code for you, you will lose that
This happens a lot, which is why agile-style methods kept popping up to replace waterfall techniques, even before anyone tagged them with the ‘agile’ label.
Ironically, that led to a Agile™ consultant-industrial complex that tried to make it a cookie-cutter process again, but we still haven’t escaped the need to break out the IDE and actually put idea into something concrete, and I doubt we ever will.
I think that is a problem between keyboard and chair situation. I use codex in addition to my own thinking through use cases and talking to stakeholders & domain experts to augment my understanding and find gaps in my plan. In other words, I work how I worked before but with a bunch of agents helping me research and diagram out plans faster before I start coding something.
I agree, it sounds like you have a problem between your keyboard and chair, if you're generating any volume of output and not getting it in detail.
The act of vetting it and finding the failures will increase understanding, but at a cost in speed over doing the research from scratch. (Generating subtly wrong text is often a good way to test understanding, and the output of AI systems is typically a mangled clone of some other well understood system. A very good exercise is building your internal pattern recognition so you can tell what's been mangled, mashed, and served up)
AI is very good at making the mash sound plausible, however, so if you don't look extremely closely, you end up repeating instead of dissecting it. And if you're hoping to learn from the AI how to design a system, you're in trouble.
you can dump your thoughts and constraints you can think of, and have it work through untangling, or use it as a summarizing notepad, or yo prototype what you are thinking about in different variations.
its no replacement for pen and paper, but no typed or drawn on tablet picture comes close to a real notebook to scribble on
its entirely a thinking tool, as much as a skip-thinking tool
The trick, I think - at least this has helped me - is not to use the AI to write for me, nor to use it to rewrite my drafts, but is to do the writing myself and to ask it for editorial feedback. This forces me to think through what I’ve written more deeply, and then rewrite it according to that deeper understanding.
I think if I used AI for the actual writing then, (a) there’d be a lot missing, and (b) I’d rob myself of much of the value, in better understanding, of writing myself.
We’re not all the same so YMMV, but this is what I’ve found from using AI in various ways to see if and how it can help with writing.
With respect to seeking feedback, my angle is that: even if you do agree with the feedback then what they do. Are you going to take it as is? Are you going to come up with it and phrase it in your own way?
What next? Next, what if there's a bunch more feedback that you agree with and you incorporate it. Then, where do we go from there?
Can you even go back to (b) and make your own edits at this point? I'm not sure and I worry about a slippery slope where you stray too far from your own original form -- so I landed on not seeking feedback either.
note: this all is definitely under progress and as I'm looking at it it's very noisy with all of the formatting... just finally getting my site put together
One of the things I ask AI to help edit my writing is a consistency pass, making sure my ideas hang together well. The real test is to see if other humans understand it.
I tell mine that line editing and formatting are the very last polish.
I am starting to believe that if something is worth sharing, it is worth writing (atleast once) by a human completely. An AI clean/pass-up after that might be okay but atleast once, an author should try to create the whole piece.
I used to have a real complicated dilemma on which content exactly to write (& then share), I recently started using handy's TTS and storing it in my scratchpad app within my native language or English and then I paste it into Chatgpt and use its memories to connect with other previous discussions I had.
It helps in saving thoughts which would've otherwise been lost, even provide some new perspectives and get them in a way that I can access them later in more depth.
I think that if I were to now create blogs of these contents. I used to think that it would perhaps be okay to share an blog like thing created by AI if say I spent 10-20 minutes thinking of the rough draft of it, but after reading your comment, I am reflecting on the fact that, atleast once, people should write things themselves no matter which AI thing they use before creating blogs.
As the point of blogs shouldn't be to share unlimited content but rather that it is the specific content where a human effort was placed. Maybe if one still wants, there could be some other blog post which could be written with all the AI blogs content I have created perhaps but still, it should be seperated away from human content.
And there is this fact that as you mention, writing itself forces you to think through ideas and thoughts which are highly relevant. So previously when I used to think that re-writing them would be a waste of time and that AI writes them good enough. I think that writing them still might make a lot of sense as it would be a net benefit if not for anybody else, then for me as it can help me better understand the material myself (even more so than reading/speaking it)
The most typing/writing of myself now occurs in Hackernews comments for me and I do observe that I learn a lot writing them personally.
Same with reading actually. Reading a book makes you think for a long time about the subject and form your own judgments. I like audiobooks and online content too, but they don’t seem to stimulate this nearly as much.
For audiobooks, I seem to not have as much of a tendency to pause the audio and stop and think, and it’s harder to reread a passage, so it’s a more passive experience.
For online content, there’s not as much of an intention to fully engage with the material. It’s easy to skim or switch to something else.
Beyond all this, there's also tailoring your message to your audience.
In sales, you might call this finding "words that work" or "the pain points". AI will not know how to do this because it's getting a vibe check of the room.
You cannot merely be a sensory part for the AI unless you want to sound as cringe and out-of-touch as the majority of the training data (stale social media). Garbage in, garbage out.
I spent a lot of time thinking about it. AI converges on average which is very bland. The labs are trying to get out of it with RLFH which sadly just produces a different “edgy” average (Calude-speak).
But things get better if you instruct AI to speak in certain style. It’s still average but for the chosen style and thus less tone-deaf for a given use case.
So my thinking is - how much mileage can we get by introducing specific style guides? We can create a library of styles, and to get out of average for a specific guide individual companies and even people will maintain their own private patches from upstream.
Another option is to maintain history of your own writing and thus preserve your own voice. I feel like “averaging” of style is somehow un-ergonomic, in that in numbs the mind, and thus even diversity for the sake of diversity is valuable.
Words are most effective when they buck the trend in just the right way.
I'll use the conference room example for clarity, even though a lot of people still work remote these days.
Reading the room could be as subtle yet significant as reacting to the speaker that was on immediately before you. It could be reacting to the temperature of the room or the catered lunch everyone just had. It can be noticing or participating in a particular trend of the audience (hawaiian shirt fridays, etc.). It can also be a million other things and they'll never let you get away with choosing at random. You have to just "get it" or you're going to be ignored at best.
You're at once trying to both signal in-group status while also asserting your own identity and setting the tone for everything you'll be saying after. Remote work has its own idiosyncrasies, but regardless, you need a pulse for this kind of work. And, all this is just the introduction. What about emphasis choices in the rest of your writing/speaking?
Trying to pin that down is simply not logical and it's not just a matter of "adding" a handful of tricks unless you want to alienate everyone. Sometimes it's also the things you don't say or do that make the difference. You also have to accept that you can't please everyone. You could actually piss off everyone but the most important person in the room and still win. That might actually be the best way or the only way to win sometimes (as was the case for the American sales pitch of AI itself!).
The real world is messy and adding AI to the mix just makes it worse.
> Yet the only difference between a FA and a TM is that
> the TM, unlike the FA, has paper and pencil.
> Think about it.
The analogy doesn't works if you really think about it. A TM with a finite tape is equivalent to an FA. The fundamental difference between a TM and an FA is that a TM has infinite space.
(Yes, I know I missed the point and am not very fun at parties.)
No, this is a very good point - you need a supply of pen and paper, such that it feels effectively infinite. Buying a nice notebook or a nice pen is a trap because both feel finite; buying a box of decent pens and a carton of decent notepads takes you out of the scarcity mindset and unlocks the Turing machine like advantages mentioned.
I was thinking of the same. FSA can multiply within bounds. For FSA, the storage is the state itself and bounded.
I agree with the sentiment but not a fan of analogies like this. We do have memory even if we don't write. Each organ adds to the memory imprint making it stronger. Visual, smell, taste, audio etc Each action adds more imprints. And this makes more sense to me than an analogy.
Another analogy which never made sense to me was improving by just 1% every day (1.01³⁶⁵).
Analogies do help explain phenomena I cannot even relate to at all, like telling me how life on Mars would be.
Ofcourse I'm not invited to parties as you can imagine, taking things literally.
A big part of understanding something is getting used to it, and analogies help with getting over a mental hang up. They may also help with deciding which areas of the new space to explore first.
But we should be examining the limits of applicability, as we do for most cognitive tools, so we know when to switch from one tool to another. It also helps to combine such tools in a complementary manner.
It's an attractive thought, this comparative look at FA and TM. However it is misleading to an extent.
A Turing Machine's capabilities exceed the FA's because it has a 'store of knowledge'. It is both recording and reading, repeatedly. There is erasure, too. So if we're aspiring to TM-hood, the guidance seems to be write, and re-read (your own thoughts), and re-evaluate, and re-write. This process is subject to the halting problem. /g
Writing as a thinking tool has become sort of a meme these days.
And, yes, I also generally believe it’s a good thing to use it as a tool. I try to do use it as often as I can.
But… I think it’s also important to note that there are - like with almost anything - pitfalls and downsides. I rarely see them discussed and thus want to bring them up:
For highly analytical people, writing may become counterproductive. Especially, when it is used to continuously elaborate thoughts rather than to challenge assumptions, test hypotheses or reach decisions.
It can lead to over-analyzing or pseudo-rationalizing.
There's always this peculiarity regarding the fact that writing by hand, is not the same as writing by kbd. So I'm always confused which of these two actually authors, mean, though I agree - any sort of note taking is better than none.
I endlessly enjoyed the comparison with a Turning Machine. When I study someone's complex work, I do notes to keep several things in my sight scope: a piece of paper, PowerPoint slide or some fancy markdown text editor with syntax highlight. Seems like that this focusing is somewhat the natural way of building neuronal connections: if a logical piece is missing we have to use imagination to complete it and only then it nicely fits into our brain's "hologram" of the world.
There is a wonderful book "Exploring Writing" by John Langan. I found it in our library when I was in my PhD writing phase. I highly recommend it to anyone who has problems building text and who wants to learn how to master its structuring.
And for freewriting, sometimes when it gets really hard to move further: https://3am.freewriter.app/
If you set 5 min target duration, you will be awarded with a full version of the song!
Writing code (Programming) is a thinking tool too.
When you are writing code manually, actually you are thinking about what you are dong, what you will build actually, what are your or users' real requirements, etc.
This will sound extremely stupid, but I have a skill that I use after letting an agent write code for a new feature. This skill creates a copy of the base branch and shows me each of the diffs that matter (outside of imports and other non-important boilerplate) and then I type it on an editor.
I don't use it too often because of the time it takes, but for critical work it is really good.
I have tried to come up with other equivalent ideas, like asking questions about a specific aspect of a feature, or showing the diff with some gaps to be filled, and even though they help I am still on the look out of methods to "imprint" the understanding of features into our brain.
Does anyone know any good papers or studies that could be useful in this research?
That’s so interesting. I remember trying to copy proofs in school out by hand but it did not work for me because - I think - I didn’t engage with the material as much as I should’ve. How do you ensure that you’re not simply blindly typing the code out and actually understand what’s going on?
Well, I only do it in order to try to understand it but there is no way to make sure you are understanding. But most of the time, if I am typing it's almost automatic that I pay attention to it. Not sure if it would work for others tho.
A common refrain for teachers and writers is that they learned so much producing their work. You think you know it, and then you go to teach it and realize how much you didn't know.
I'm a big fan of marbled composition books. They're cheap and durable enough. You can write in them without a table. I have 5-6 of them going at once for various tasks. And somehow (for me) the ideas flow more freely longhand than they do at the keyboard. When full, you can cut the spine off with a box cutter and scan it. G4 compression loses the color, but makes really small PDFs.
It's easy to produce better written content than an LLM. And that's good for the reader and the writer.
When i started writing i felt the pressure to always go from draft to a polished piece. i posted a couple things i worked semi hard on and received 0 views/reads because i don't really have much of an online presence at all.
From there i kinda just moved on and write purely for myself, and use it to up my own thoughts. Very rarely, something will receive a second pass days later, and thats just more to go back on my past thoughts and try to tighten them if i feel that is needed.
I tend to split my day into two parts, my work hours and my "unplug" hours, and the writing was mostly landing in those unplug hours as a way to sorta summarize my day. Last week or so I decided im moving that writing into the very first thing I do each day, and it has really been absurdly effective at getting new ideas flowing quickly. I used to depend on my later day brain that was somewhat "depleted", for lack of a better way to describe how i work, and from there that would kinda birth what i work on the next morning, with maybe some iteration while working.
Now instead I just do both, I open the day with ~500-1500 words, just anything that comes out, then i potentially get stuck, and ill maybe dump that to a model as just a modern rubber duck, not anything like "write for me, or finish this thought", just a kinda nudge to really toss back at me what im fully considering and then maybe i pivot back and chase down the most interesting of the topics for a few hundred more words, or get straight to work on what i thought was the most vital of all those ideas to chase while im fresh in the morning.
But, to the claim of how to think in the age of LLMs, I think it’s worth contrasting the other approach, which is a dialog with the LLM; two things are needed IMO to keep things honest:
First, you need to keep asking the LLM to explain things, until you understand what it did. Lots of angles here; have the LLM make a design/product doc, write one-off architecture explainers, write throwaway data-viz tools to explain the data distribution. Then ask a fresh copy to adversarially review the core concepts. Try asking a different frontier model too.
Second, you need to push for “proof”, in other words, ask for demos and try it yourself. Imagine yourself a CEO or VC; how do you know if the output of an entire org is vaporware? One thing to beware is launching into a field where you don’t have expertise and can’t actually validate the output. This takes time of course, but for example if you write a stress test script and get CPU/QPS numbers, these complex validators are IME quite reliable when you apply the same rigor from 1) above to their construction.
I'd been a software engineer for 25 years before language models came into the scene. There were so many parts of the engineering world I had no idea about: Large cloud data processing, lower level device / GPUs, UI frameworks and design.
I kept my to my niche.
Now I can build anything, but as I'm building it, I try to understand at least how it works. So I'm constantly asking questions. In the process, I think of my prior engineering skill as somewhat childlike. As I thought I was a decent engineer, I now see that I was decent at some things, but far from the kind of breadth that the wide engineering world has.
I think the takeaway is that you should always be curious and build things. Whether it's writing or discussing with an language model, writing software by hand, or creating software with a model and understanding it, I think you have the same effect.
> Now I can build anything, but as I'm building it, I try to understand at least how it works. So I'm constantly asking questions.
This is the way!
AI is the best technology ever invented for avoiding learning things. It’s also the best technology ever invented for learning things.
I think it’s very important to be on your guard for accidentally glazing over your understanding, but if understanding is your goal, there are so many more tools for this now.
Before AI, even when working collaboratively and bumping into concepts I did not fully grasp, there was so much more friction to get answers. Sure I could Google and try my luck, or I could ask a colleague. But the LLMs have infinite patience, can build diagrams, and can answer weird questions I would be afraid to ask.
Being forced to take notes and write essays in school made me hate handwriting specifically and dislike writing in general. Now that I have a choice, I never use stationery or take notes for any purpose, and everything useful that I learned or did in my entire life has been done without writing being a part of the process.
This is true for designing, too. I see a lot of people saying Figma is dead because you can prototype directly; for novel UX problems the act of designing — of putting ideas together, assembling a component, even getting good data into a design — is where a lot of my thinking and insights develop. The brain work plus the craft work, the tight loop between thinking and creating is so critical for me and I haven’t found nearly the same feeling when prompting.
In writing and designing I think we often have a thought-feeling first: something inchoate that we can’t quite articulate yet. Getting directly to the medium itself — text or frames and layouts — is how we give shape to that feeling and turn it into a concept, insight or design.
I actually appreciate working with LLMs on design quite a lot, although I'm not a designer by trade so my situation may differ. I enjoy getting an image-generated mockup (that is usually top class), then letting an LLM implement it, then seeing where it doesn't feel quite right. Then trying to understand why it doesn't feel right. When I do the CSS myself, I iterate one component at a time and usually not get the 'find the 100 differences' feel. Animations are even more fun, you feel they suck, but you have to slow them down to understand what's really making them suck. Not claiming I can do good designs, but it helped me identify faster why some designs were bad.
Yeah I can see when you’re on a certain part of a learning curve how that kind of iteration can be really useful. “That seems good, let’s see it. Hm now I feel how this part is clunky”. I can imagine looping that way for certain kinds of things I don’t have a ton of experience with — animations will be a good one for me actually.
I see what you are saying and for very specific problems/explorations I do still reach for Figma. But the days of crafting static screens and multi-artboard flows are gone. It's better to prototype the entire flow/interaction directly than trying to reason about it 'statically'. LLMs have also opened up an entire array of new possibilities for e.g. combining meaningful 3D and taking a 'leap of faith' into visualizations/interactions that would simply be too costly to properly explore before
> But the days of crafting static screens and multi-artboard flows are gone. It's better to prototype the entire flow/interaction directly than trying to reason about it 'statically'.
I haven’t found this to be true but it could well come down to the type of design work I do. I’m usually working on new features for complex workflows within an already complex app. When I’ve tried to yolo designs with LLMs they never come up with elegant visual or UX solutions. They’ve been occasionally useful as a way to get something on the screen that I can work against — ie “well that’s clearly wrong in these 20 ways” — but at that point I usually then reach for figma so I can just do the damn thing myself.
Agree about the more complex design options it opens up, I just haven’t had call to do much in those spaces yet.
Writing is not helpful just because it has Turing-tape memory or because it reduces noise in your signal to others.
It's mainly helpful in changing an exercise for youself into something useful for others, that requires you actually adopt their perspectives and migrate them to yours (or realize the error of your ways). You take on the capability not just to convey ideas but also to change the world.
Interestingly what has worked very well for me is that when I start a new project, I just have to start drawing a few boxes and connect them as I expect them to be communicating and with just an hour or so of doing this, I find myself ready to write code as if there was a block that got completely untangled with putting it on paper.
I have been finding I spend more time writing when working with AI. While I do let it generate docs, it does not always do a great job and I end up reorganizing, reframing, or changing vocabulary. Those passes end up require rigorous thinking.
There has been some occasions where I cannot use AI as a thinking partner. It doesn’t have the imagination, intuition, or judgement. I have been relying more on my team in those cases.
This article reminds me of another method: grabbing paper and pencil and start doodling around to work out some ideas.
78 comments
[ 2.4 ms ] story [ 34.9 ms ] threadWhen I get it on paper and everything feels right I start creating the codebase but I have to get it down first, feels like I remember it more after I save my thoughts to external paper storage.
There is something about the complete free form nature of pen and paper that makes me think better, I bought a nice ipad/pencil a few years ago thinking that would replicate it but it didn't, it was closer but not close.
If it's something I want to keep I just take a photo but vast majority of notes are transient anyway.
I made a little system where blue diagrams are services I need to code and host like a nodejs server
green is stuff I need to host only like nginx, but no coding
red is external infra I can't control at all, like Anthropic's API.
And then I make drawings of how I connect them and in the end I just generate the code, so I go from pen and paper to claude code.
For thinking, nothing beats handwriting. It's just the unconstrained freedom of movement in every direction, shape or form, and the lack of friction and distractions.
But then you sometime also want to organize or share stuff, so that's where digital comes in. I have an automation that takes a photo of my notes with my phone, uploads it to Obsidian, and OCRs it into markdown and tags it, so I basically end up with a "symbolic" 1-1 equivalent of analog handwriting + digital text
My system still evolves, but what I currently use is:
Color: - black = just "normal" text - red = critical/"must" - yellow = important/"should" - green = highlighted/"could"
Symbols: - bulletpoint [ ] task -> reminder
I also do brackets and underlines which translate to # heading1 and ## heading2 in markdown
I write by hand with a fountain pen with black ink + use 3 fineliners and small colored sticky flags.
Some observations: - yellow is not great on white paper due to bad contrast; it also rarely comes as ballpoint, but fineliner options are available - blue and green are visually very similar, to the point of kinda being redundant - for sharing with other people, the system must be as generally understandable as possible. Here the traffic norms such as ISO 7010 (https://en.wikipedia.org/wiki/ISO_7010) could be considered - also there's the aspect of accessibility, with a relevant portion of population being colorblind - The simplest most effective system seems to be: one main color (black or blue) and one highlight color (red I think works best), plus a couple of symbols such as bullet point, task, reminder that translate well into markdown
I really think we could go back to pen and paper fully for software design.
It sounds like you have some stationery recommendations!
I'm always in the market for really-good-but-not-expensive pens.
The problem with AI writing is that it makes people believe that writing is just writing and not a tool for thinking very deeply about the topic you are writing about. I am not completely opposed to AI assisted writing, but if I see AI writing, it’s telling me that the author tried to take a shortcut in their own thinking and it’s a good heuristic that the ideas will not be thought through enough to warrant my complete attention.
Even worse: Facing AI writing, there's a strong suspicion that any concerns or counterpoints you bring won't be actually considered, but rather copypasted into an llm session.
Ironically, that led to a Agile™ consultant-industrial complex that tried to make it a cookie-cutter process again, but we still haven’t escaped the need to break out the IDE and actually put idea into something concrete, and I doubt we ever will.
The writing is downstream of that.
The act of vetting it and finding the failures will increase understanding, but at a cost in speed over doing the research from scratch. (Generating subtly wrong text is often a good way to test understanding, and the output of AI systems is typically a mangled clone of some other well understood system. A very good exercise is building your internal pattern recognition so you can tell what's been mangled, mashed, and served up)
AI is very good at making the mash sound plausible, however, so if you don't look extremely closely, you end up repeating instead of dissecting it. And if you're hoping to learn from the AI how to design a system, you're in trouble.
you can dump your thoughts and constraints you can think of, and have it work through untangling, or use it as a summarizing notepad, or yo prototype what you are thinking about in different variations.
its no replacement for pen and paper, but no typed or drawn on tablet picture comes close to a real notebook to scribble on its entirely a thinking tool, as much as a skip-thinking tool
I think if I used AI for the actual writing then, (a) there’d be a lot missing, and (b) I’d rob myself of much of the value, in better understanding, of writing myself.
We’re not all the same so YMMV, but this is what I’ve found from using AI in various ways to see if and how it can help with writing.
The feedback bit is surely more debatable, but I would argue the initial writing and allowing edits are not.
https://antoaenono.com/write/fragments/writing#:~:text=never...
With respect to seeking feedback, my angle is that: even if you do agree with the feedback then what they do. Are you going to take it as is? Are you going to come up with it and phrase it in your own way?
What next? Next, what if there's a bunch more feedback that you agree with and you incorporate it. Then, where do we go from there?
Can you even go back to (b) and make your own edits at this point? I'm not sure and I worry about a slippery slope where you stray too far from your own original form -- so I landed on not seeking feedback either.
note: this all is definitely under progress and as I'm looking at it it's very noisy with all of the formatting... just finally getting my site put together
also not sure about my fish progress bar hah
I tell mine that line editing and formatting are the very last polish.
I used to have a real complicated dilemma on which content exactly to write (& then share), I recently started using handy's TTS and storing it in my scratchpad app within my native language or English and then I paste it into Chatgpt and use its memories to connect with other previous discussions I had.
It helps in saving thoughts which would've otherwise been lost, even provide some new perspectives and get them in a way that I can access them later in more depth.
I think that if I were to now create blogs of these contents. I used to think that it would perhaps be okay to share an blog like thing created by AI if say I spent 10-20 minutes thinking of the rough draft of it, but after reading your comment, I am reflecting on the fact that, atleast once, people should write things themselves no matter which AI thing they use before creating blogs.
As the point of blogs shouldn't be to share unlimited content but rather that it is the specific content where a human effort was placed. Maybe if one still wants, there could be some other blog post which could be written with all the AI blogs content I have created perhaps but still, it should be seperated away from human content.
And there is this fact that as you mention, writing itself forces you to think through ideas and thoughts which are highly relevant. So previously when I used to think that re-writing them would be a waste of time and that AI writes them good enough. I think that writing them still might make a lot of sense as it would be a net benefit if not for anybody else, then for me as it can help me better understand the material myself (even more so than reading/speaking it)
The most typing/writing of myself now occurs in Hackernews comments for me and I do observe that I learn a lot writing them personally.
For audiobooks, I seem to not have as much of a tendency to pause the audio and stop and think, and it’s harder to reread a passage, so it’s a more passive experience.
For online content, there’s not as much of an intention to fully engage with the material. It’s easy to skim or switch to something else.
Thanks for writing that down. It explains in simple terms the difference
Not all writing needs to be published.
In sales, you might call this finding "words that work" or "the pain points". AI will not know how to do this because it's getting a vibe check of the room.
You cannot merely be a sensory part for the AI unless you want to sound as cringe and out-of-touch as the majority of the training data (stale social media). Garbage in, garbage out.
But things get better if you instruct AI to speak in certain style. It’s still average but for the chosen style and thus less tone-deaf for a given use case.
So my thinking is - how much mileage can we get by introducing specific style guides? We can create a library of styles, and to get out of average for a specific guide individual companies and even people will maintain their own private patches from upstream.
Another option is to maintain history of your own writing and thus preserve your own voice. I feel like “averaging” of style is somehow un-ergonomic, in that in numbs the mind, and thus even diversity for the sake of diversity is valuable.
I'll use the conference room example for clarity, even though a lot of people still work remote these days.
Reading the room could be as subtle yet significant as reacting to the speaker that was on immediately before you. It could be reacting to the temperature of the room or the catered lunch everyone just had. It can be noticing or participating in a particular trend of the audience (hawaiian shirt fridays, etc.). It can also be a million other things and they'll never let you get away with choosing at random. You have to just "get it" or you're going to be ignored at best.
You're at once trying to both signal in-group status while also asserting your own identity and setting the tone for everything you'll be saying after. Remote work has its own idiosyncrasies, but regardless, you need a pulse for this kind of work. And, all this is just the introduction. What about emphasis choices in the rest of your writing/speaking?
Trying to pin that down is simply not logical and it's not just a matter of "adding" a handful of tricks unless you want to alienate everyone. Sometimes it's also the things you don't say or do that make the difference. You also have to accept that you can't please everyone. You could actually piss off everyone but the most important person in the room and still win. That might actually be the best way or the only way to win sometimes (as was the case for the American sales pitch of AI itself!).
The real world is messy and adding AI to the mix just makes it worse.
> the TM, unlike the FA, has paper and pencil.
> Think about it.
The analogy doesn't works if you really think about it. A TM with a finite tape is equivalent to an FA. The fundamental difference between a TM and an FA is that a TM has infinite space.
(Yes, I know I missed the point and am not very fun at parties.)
Another analogy which never made sense to me was improving by just 1% every day (1.01³⁶⁵).
Analogies do help explain phenomena I cannot even relate to at all, like telling me how life on Mars would be.
Ofcourse I'm not invited to parties as you can imagine, taking things literally.
A big part of understanding something is getting used to it, and analogies help with getting over a mental hang up. They may also help with deciding which areas of the new space to explore first.
But we should be examining the limits of applicability, as we do for most cognitive tools, so we know when to switch from one tool to another. It also helps to combine such tools in a complementary manner.
A Turing Machine's capabilities exceed the FA's because it has a 'store of knowledge'. It is both recording and reading, repeatedly. There is erasure, too. So if we're aspiring to TM-hood, the guidance seems to be write, and re-read (your own thoughts), and re-evaluate, and re-write. This process is subject to the halting problem. /g
And, yes, I also generally believe it’s a good thing to use it as a tool. I try to do use it as often as I can.
But… I think it’s also important to note that there are - like with almost anything - pitfalls and downsides. I rarely see them discussed and thus want to bring them up:
For highly analytical people, writing may become counterproductive. Especially, when it is used to continuously elaborate thoughts rather than to challenge assumptions, test hypotheses or reach decisions.
It can lead to over-analyzing or pseudo-rationalizing.
Ask your favorite LLM for scientific references like this one: https://pubmed.ncbi.nlm.nih.gov/40137068/
There is a wonderful book "Exploring Writing" by John Langan. I found it in our library when I was in my PhD writing phase. I highly recommend it to anyone who has problems building text and who wants to learn how to master its structuring.
When you are writing code manually, actually you are thinking about what you are dong, what you will build actually, what are your or users' real requirements, etc.
I don't use it too often because of the time it takes, but for critical work it is really good.
I have tried to come up with other equivalent ideas, like asking questions about a specific aspect of a feature, or showing the diff with some gaps to be filled, and even though they help I am still on the look out of methods to "imprint" the understanding of features into our brain.
Does anyone know any good papers or studies that could be useful in this research?
Im even saying this with some level of sincerity
I'm a big fan of marbled composition books. They're cheap and durable enough. You can write in them without a table. I have 5-6 of them going at once for various tasks. And somehow (for me) the ideas flow more freely longhand than they do at the keyboard. When full, you can cut the spine off with a box cutter and scan it. G4 compression loses the color, but makes really small PDFs.
It's easy to produce better written content than an LLM. And that's good for the reader and the writer.
From there i kinda just moved on and write purely for myself, and use it to up my own thoughts. Very rarely, something will receive a second pass days later, and thats just more to go back on my past thoughts and try to tighten them if i feel that is needed.
I tend to split my day into two parts, my work hours and my "unplug" hours, and the writing was mostly landing in those unplug hours as a way to sorta summarize my day. Last week or so I decided im moving that writing into the very first thing I do each day, and it has really been absurdly effective at getting new ideas flowing quickly. I used to depend on my later day brain that was somewhat "depleted", for lack of a better way to describe how i work, and from there that would kinda birth what i work on the next morning, with maybe some iteration while working.
Now instead I just do both, I open the day with ~500-1500 words, just anything that comes out, then i potentially get stuck, and ill maybe dump that to a model as just a modern rubber duck, not anything like "write for me, or finish this thought", just a kinda nudge to really toss back at me what im fully considering and then maybe i pivot back and chase down the most interesting of the topics for a few hundred more words, or get straight to work on what i thought was the most vital of all those ideas to chase while im fresh in the morning.
But, to the claim of how to think in the age of LLMs, I think it’s worth contrasting the other approach, which is a dialog with the LLM; two things are needed IMO to keep things honest:
First, you need to keep asking the LLM to explain things, until you understand what it did. Lots of angles here; have the LLM make a design/product doc, write one-off architecture explainers, write throwaway data-viz tools to explain the data distribution. Then ask a fresh copy to adversarially review the core concepts. Try asking a different frontier model too.
Second, you need to push for “proof”, in other words, ask for demos and try it yourself. Imagine yourself a CEO or VC; how do you know if the output of an entire org is vaporware? One thing to beware is launching into a field where you don’t have expertise and can’t actually validate the output. This takes time of course, but for example if you write a stress test script and get CPU/QPS numbers, these complex validators are IME quite reliable when you apply the same rigor from 1) above to their construction.
I kept my to my niche.
Now I can build anything, but as I'm building it, I try to understand at least how it works. So I'm constantly asking questions. In the process, I think of my prior engineering skill as somewhat childlike. As I thought I was a decent engineer, I now see that I was decent at some things, but far from the kind of breadth that the wide engineering world has.
I think the takeaway is that you should always be curious and build things. Whether it's writing or discussing with an language model, writing software by hand, or creating software with a model and understanding it, I think you have the same effect.
This is the way!
AI is the best technology ever invented for avoiding learning things. It’s also the best technology ever invented for learning things.
I think it’s very important to be on your guard for accidentally glazing over your understanding, but if understanding is your goal, there are so many more tools for this now.
With writing I think it's the same thing.
In writing and designing I think we often have a thought-feeling first: something inchoate that we can’t quite articulate yet. Getting directly to the medium itself — text or frames and layouts — is how we give shape to that feeling and turn it into a concept, insight or design.
I haven’t found this to be true but it could well come down to the type of design work I do. I’m usually working on new features for complex workflows within an already complex app. When I’ve tried to yolo designs with LLMs they never come up with elegant visual or UX solutions. They’ve been occasionally useful as a way to get something on the screen that I can work against — ie “well that’s clearly wrong in these 20 ways” — but at that point I usually then reach for figma so I can just do the damn thing myself.
Agree about the more complex design options it opens up, I just haven’t had call to do much in those spaces yet.
It's mainly helpful in changing an exercise for youself into something useful for others, that requires you actually adopt their perspectives and migrate them to yours (or realize the error of your ways). You take on the capability not just to convey ideas but also to change the world.
There has been some occasions where I cannot use AI as a thinking partner. It doesn’t have the imagination, intuition, or judgement. I have been relying more on my team in those cases.
This article reminds me of another method: grabbing paper and pencil and start doodling around to work out some ideas.