I don't get it. I'm only starting my career but most of the tenured/senior developers I know spend too much time in meetings to write that much boilerplate
The problem is that you only get out of GPT what you ask of it. For example, it’s likely that more senior developers would write abstractions because they can simplify unit testing. Others might have dived into writing TypeScript types, e.g. meta-programming or generics.
The output ChatGPT actually gave is interesting but not inspiring or relevant. They didn’t even uselessly add Winston for logging. It’s like when you ask GPT for images of “more, even more, work harder, etc.” - you sorta get more of the same, not something new or innovative. You have to ask for what you want, it’s not that creative and doesn’t really understand yet.
Don’t worry, they’ll hit you on the code reviews and ask you to refactor your working to-the-point code into some sort of factory-like nonsense soon enough.
That joke was originally written in java. This means that chatGPT was able to actually extract the idea of this joke, and combine it with the typescript language syntax, to produce this output.
Even being a freelance programmer, this reminds me a bit of the evolution of my own programming style until i hit burnout several years ago. Now i try to be that junior programmer again. Even if its good to know all these programming patterns its also helpful to know how to write compact and dense style. Pico8 and Dwitter helped me with that.
I’m a Principal Cloud Engineer and I practice minimum viable software. Less to grok, less to maintain, less debugging effort during an incident. In the context of a larger system it may make sense to implement a logger for wrapping messages with consistent contextual values because DRY.
I’ll give ChatGPT a pass because the prompt was so contrived. I would be curious how it would respond if prompted with a more open-ended problem.
out of curiosity, why do you use the word "grok" instead of "understand" here? I have what I recognize is probably an irrational annoyance with the word grok and i want to better understand what people mean by it to potentially dislike it less
I suppose I meant it as to say thoroughly understand a piece of code, why it exists, the context it operates in, and the implications of modifying it. I’ve learned the hard way how making a decision about the need for code or the way it’s implemented can lead to unintended consequences, some of which may be difficult to detect. So I’m using grok to describe a completeness of understanding, which I believe aligns with its dictionary definition.
I believe the definition is more to intuitively understand something. Like if you had never seen a door before, you could grok that the handle on the door was used to pull it open. Someone doesn’t have to explain it to you since it’s the only pulling option available
„Seems like it learned the boilerplate from Java and transfered it to TypeScript. “ - Just like real developers in a large corp… Really, I think ChatGPT is joking here without knowing it. There must be lots of jokes like this in the training material.
Notice the “I asked it to refactor…” … “now how would a…”
This is a sequence of prompts in a single context which is “pre biasing” the LLM to respond the way you want it to.
Many such “emergent behaviours” are from the asker, not the LLM.
When you look in the mirror, you only see yourself.
> In the final exercise of this experiment, I challenged ChatGPT to step back, assess the code's objectives, and propose a better solution. The attempt was unsuccessful at first, as it continued to tweak the existing program and preserve the "existing architecture." Only when I instructed it to envision starting from scratch did it offer a new solution, as follows:
I forgot what people were calling it but this is almost exactly the same idea as that trend where you ask ChatGPT to generate pictures, continuously asking for "more" of some aspect of the picture (make this dish more spicy, etc), only hidden by the fact that instead of simply asking it to make the code more complex, they're naming specific (but in the end, arbitrary) steps of what ChatGPT would still interpret as escalation.
That would be awesome if ChatGPT picked up on the prompts as cues to generate a joke. Sounds fine for generation of entertainment content, but less so for anything critical.
So to round-out the joke, ask it how a 'galaxy brain' would write it, and we should end up at console.log(message) again.
It was as misguided then as it is now.
What this post bemoans didn't actually occur, really.
What did happen was that the big framework gave you increasingly more entrypoints into how the application was constructed.
But there always was a way to just go with defaults if you didn't want to adjust anything (at least in Java-Land, which this one specifically calls out for this behavior)
Wherever a framework should give the developers the option to adjust things to such a degree is another question that wasn't raised by this post. It just claims that you needed the increasingly silly factories to construct your application, which isn't true, as they were always optional.
I was kinda thinking maybe the LLM was rooting through its “comical software engineering discussions about enterprise abstraction pathology” networks when it wrote these responses.
It understands progressive complexity in code. Keyword: understand.
Because there's no underlying training data that a stochastic parrot would need to generate this output the gap by this lack of data is only bridged by something that can be described a high level word: "understanding".
chatGPT understood the request and delivered answers to the request. I'm not saying those answers are correct. That doesn't even matter. What matters is chatGPT gave a biased answer to that request that indicates understanding of several concepts including ranking in human social structures and complexity of code.
One could say that the answers are incorrect. chatGPT hallucinated those answers because staff engineers don't write code like that. These people don't get it. The act of the hallucination indicates "understanding" regardless.
How I write a "Hello, World!" program as an expert haskell programmer
{-# LANGUAGE ImportQualifiedPost #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes #-}
{-# LANGUAGE TemplateHaskell #-}
module Main where
import Foreign.C.Types
import Language.C.Inline qualified as C
C.include "<stdio.h>"
C.include "<stdlib.h>"
main :: IO ()
main = [C.block| void {
system("python -c 'print \"Hello, World!\"'");
} |]
I think it'd be interesting to see the actual prompts that went with this.
I followed it through to the accompanying blog post[1] and I found the examples for "we close at 6pm on Friday" interesting because none of them work. They fail the stated problem (they don't test for Friday) and the unstated problem (when do we re-open? the examples re-open at midnight). And of course, without the prompts I can only guess whether the fault was in the answer or the question.
If the complex version worked and the simple version was naïve - or vice versa - it'd make for a much more interesting conclusion.
41 comments
[ 3.0 ms ] story [ 53.9 ms ] threadThe problem is that you only get out of GPT what you ask of it. For example, it’s likely that more senior developers would write abstractions because they can simplify unit testing. Others might have dived into writing TypeScript types, e.g. meta-programming or generics.
The output ChatGPT actually gave is interesting but not inspiring or relevant. They didn’t even uselessly add Winston for logging. It’s like when you ask GPT for images of “more, even more, work harder, etc.” - you sorta get more of the same, not something new or innovative. You have to ask for what you want, it’s not that creative and doesn’t really understand yet.
Then you start to realize that everything needs to sit behind some abstraction.
Well from a very pedantic and technical perspective stdout can be tested but overall IO cannot be unit tested.
It can be tested via e2e or integration tests meaning you need external factors to verify a side effect occurred.
I’ll give ChatGPT a pass because the prompt was so contrived. I would be curious how it would respond if prompted with a more open-ended problem.
Tried the same with Haskell and the Fibonacci series because there's a similar joke on the internet. It's similar but not as stereotypical.
This is a sequence of prompts in a single context which is “pre biasing” the LLM to respond the way you want it to.
Many such “emergent behaviours” are from the asker, not the LLM.
When you look in the mirror, you only see yourself.
> In the final exercise of this experiment, I challenged ChatGPT to step back, assess the code's objectives, and propose a better solution. The attempt was unsuccessful at first, as it continued to tweak the existing program and preserve the "existing architecture." Only when I instructed it to envision starting from scratch did it offer a new solution, as follows:
Surprise.
Naive usage.
So to round-out the joke, ask it how a 'galaxy brain' would write it, and we should end up at console.log(message) again.
https://gwern.net/doc/cs/2005-09-30-smith-whyihateframeworks...
What did happen was that the big framework gave you increasingly more entrypoints into how the application was constructed.
But there always was a way to just go with defaults if you didn't want to adjust anything (at least in Java-Land, which this one specifically calls out for this behavior)
Wherever a framework should give the developers the option to adjust things to such a degree is another question that wasn't raised by this post. It just claims that you needed the increasingly silly factories to construct your application, which isn't true, as they were always optional.
Because there's no underlying training data that a stochastic parrot would need to generate this output the gap by this lack of data is only bridged by something that can be described a high level word: "understanding".
chatGPT understood the request and delivered answers to the request. I'm not saying those answers are correct. That doesn't even matter. What matters is chatGPT gave a biased answer to that request that indicates understanding of several concepts including ranking in human social structures and complexity of code.
One could say that the answers are incorrect. chatGPT hallucinated those answers because staff engineers don't write code like that. These people don't get it. The act of the hallucination indicates "understanding" regardless.
I followed it through to the accompanying blog post[1] and I found the examples for "we close at 6pm on Friday" interesting because none of them work. They fail the stated problem (they don't test for Friday) and the unstated problem (when do we re-open? the examples re-open at midnight). And of course, without the prompts I can only guess whether the fault was in the answer or the question.
If the complex version worked and the simple version was naïve - or vice versa - it'd make for a much more interesting conclusion.
[1] https://koenvangilst.nl/blog/keeping-code-complexity-in-chec...