Ask HN: Can build CRUD apps, what now?
I started coding in 2010 when I was in High school using JScript and Microsoft PageBuilder with the Help Manual(chm file) that it came with (I had no internet connection). Later I picked up PHP, JS, Java and a whole lot of other technologies which allowed me to be an amazing CRUD app developer, My resume is filled with the latest trendy words like Docker, Scala, Kafka, Spark, Hive, react, K8s or any new framework that seems to make a buzz. All this was amazing but I recently realized that I am not so smart and I can't seem to build anything other than CRUD apps and it is quite depressing.
There is a world out there about compilers, run-times, drivers, emulators, VMs and OSes that I can't seem to grasp, it is just too complicated for me. My Computer Science degree and the newer degrees do not focus on these, they have become 4 year long coding boot camps that focus on getting people job ready with AWS, React, Ruby or whatever trendy. This is true for the majority of the not so prestigious universities out there.
I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general. I am pretty sure a lot of people are or were in the same boat as me. I don't really have the time(Have wife & kids) to take a compiler course or set out to build my own OS, what would be the easiest way to learn these lower level things?
Things I have tried so far,
1. Reading the Dragon book(I find this would be useful only once you have built a bad compiler) 2. Paying people older than 40-45 to teach me some C/C++ and some tricks of the trade (expensive but I gained a huge amount of knowledge in a short time)
any other tricks, or comments about your own journey would be helpful.
85 comments
[ 3.1 ms ] story [ 173 ms ] threadTen years after that I was thinking "great, I've spent 10 years validating email addresses on contact us forms" (the reality was quite different, of course, but I was feeling like I'd lost the thread). But then I started doing a little more with ecommerce and got good at mentoring people and so on.
Now, it's ten years after that. I am managing a bunch of programmers and still write a little code. And I'm wondering how I will stay relevant for the next twenty or so years but so far, so good.
And I keep learning more and more. Not about compilers and Docker (this does come up, I guess) but about projects and people, business, domain knowledge about the area we build software for, and so on.
On paper, yes, I'm nothing. Just some average dude. But I really like my job, my team, and my skills.
I'm not saying this will happen to you but what I am saying is you shouldn't feel inadequate because you can't build a C compiler in assembly or something. That's the kind of thing only someone truly, truly interested would do. It's not something you do because you should or because it will make you a "real programmer".
Find an industry or business that fascinates you and figure out what they need to see to hire you and go for that.
Depending on your field / industry, having good knowledge of accounting is magnitude times better than compiler knowledge.
Learning about optimization (both db and app), write good / clean code, debugging are useful skills wherever you go.
Unless you want to change course, it's perfectly fine.
Disagree. Most JS frameworks require toolchains these days, particularly in the form of Babel (parsing/compilation) and Webpack (linking). If you understand the way compilers and linkers work, then these web tools are more useful.
Sadly, a lot of web devs just think it's too complicated and so copypasta their configs without understanding, but lots of frustration.
I'm interested with it. Got any examples at what can they be used further if you know compilers and linkers? So far without understanding it I can use it for compiling react.
However I didn't said that those skills aren't useful. They just aren't requirement for CRUD / web devs. And I'd argue that learning business side is far more useful than technical side (after you've learnt up to certain level)
This is an excellent perspective, and a recommendation for OP I hadn't thought of. Learning how accounting or finance works, how sales works and then how to think strategically about what each faction involved in a business is interested in (including customers and various types of employees) can be very helpful. Appreciating and starting to understand design can also be helpful.
> I'm interested with it. Got any examples at what can they be used further if you know compilers and linkers? So far without understanding it I can use it for compiling react.
I'd take a look at the sources on GitHub at first. Maybe clone them, set breakpoints and start debugging.
In React, the compiler (based on Babel) is going to translate JSX into (virtual) DOM operations, plus handle some of the JS features that aren't supported in the vanilla browser implementations. Webpack is going to read in these translated sources as well as the React runtime (which has the logic handling virtual DOM operations) and any other assets you use (images, CSS, etc.) as represented in JavaScript. With everything in the same namespace, Webpack can then begin "linking" the "objects" and resolving references. It will then begin writing them all out as you specify in the webpack config.
This is highly analogous to e.g. `gcc` (C compiler) and `ld` (linker). The compiler takes sources and translates them to something the machine understands, the "object" files. But the references to anything outside of each object file are all broken. The linker takes all of the object files and resolves the references, including to runtime dependencies like "shared libraries"/DLLs, and then it organizes the final outputs in a way the programmer specifies (usually there's a default that just works, but for some types of outputs, you write "linker script").
Also like with webpack, there are all sorts of kind of lousy tools that aim to automate the creation of the build scripts, like Automake or CMake. These tools do save time in most cases, but they are leaky abstractions that are quite complex to understand vs. the relatively simple (though also difficult) compiler and linker, which are fairly easy to reason about with some experience. React-scripts is I guess the latest analogous tool I've seen that tries to hide away webpack, though I've also seen create-react-app and razzle.
The beauty is, if you understand how people were dealing with some of these issues in compiling C or FORTRAN or whatever 50 years ago, it's easier to see what the pitfalls and strengths of the current web toolchains, despite the complete dissimilarity on the surface. The patterns are quite similar.
EDIT: it might be useful to find a tutorial you like on regex and parsing. (Strict) regex are equivalent to "deterministic finite automata," which are conceptual, abstract virtual machines that transition between states based on inputs. Parsers are more related to a class of automata called "pushdown" automata (though there's a lot more variation, and parsers are not actually implemented as PDAs despite some loose equivalence). Understanding the hierarchy of automata (DFAs up to Turing machines and then the purely imaginary ones like "oracles"), as well as the distinction between deterministic and non-deterministic and how those can be made equivalent (for some trade-off in real world efficiency) is also useful. It might also be good to study and explore BNF-type grammars and parser generators. Some of these exist in JavaScript, like Ohm. Ohm is particularly useful because it has a visual browser-based editor that explains exactly how a parse is being performed given some test input and grammar. Once you kind of understand that, you can experiment with other kinds of parsers (Earley, LALR, recursive descent, etc.).
Writing a s...
Regarding VMs and OSes:
Since it seems that you have been working as a web developer, I think a soft transition for you would be to get into modern devops practices:
- start with vagrant to spin up a linux virtual machine and start playing with it. E.g., how to setup ssh keys, how to install packages in an automated an idempotent way (e.g., using Ansible), etc.
- while you are dealing with virtual machines, you'll learn also a bit about networking (e.g., what's a private network? what's NAT?)
- when you know enough you could start using Terraform to spin up virtual machines in the cloud. There's some magic the moment you run one command and you have a linux machine up and running in some cloud provider! Even better: you run another command and the machine is totally destroyed :)
In such a journey you'll be exposed to different and potential new concepts for you: vpcs, virtualization, a bit of security, firewall rules, ssh, tcp/ip, etc.
I also got a lot out of Bob Nystrom's "Crafting Interpreters" book
My personal path hasnt shown that to be true, but maybe I’m the odd duck.
We are at the stage of lacking a lot of software help. We have an assembler, but that's it.
We're expecting some initial chips back in a few months from using the Google/Skywater/Efabless shuttle program: https://efabless.com/open_shuttle_program
Currently we are simulating the CPU in verilog simulators, as well as fpga's. There's a lot of fun very low-level work to be done.
I'd love to help.
Do you need someone to write embedded tools? I am getting into FPGA for RISCV with the Arty board and was wondering if you need any help regarding this.
I really don’t think most modern CS programs are glorified bootcamps. Not sure where you’re getting that from.
Some of the worst offenders were:
* A graduate-level parallel computing class taught in sync with a 1000-person online class aimed at OMCS students. For the level of depth we went into, I may as well have just taken a Coursera course.
* A PhD seminar on compilers, led by a tenured professor who was reusing slides from 2002 and seemed not to know anything about modern compiler theory or practice. "Huh? What do you mean a 'lattice'?" he said. Thankfully his TA cared enough to develop some projects around LLVM, which is the only useful bit of knowledge I took away from the course.
* A natural language processing course with a severely outdated bibliography, where the projects required us to make changes to convoluted "starter code" riddled with bugs. Half of the final exam was manually computing the probability table (by hand, on paper, without a calculator) for a quite complicated probabilistic context-free grammar.
* A machine learning course where the lecture slides were written by the TA, and where the professor was often seeing the slides for the very first time five minutes before lecture.
And that's just the courses I was able to take.
* For my entire four years as an undergraduate at a large public university, there were no classes offered on the topics of programming language theory or computer graphics. They simply couldn't attract research faculty in these areas, and instead of hiring a lecturer, they thought it better not to offer the subjects at all.
* When I became a graduate student at another large public university, I was excited at the chance to finally take some PL + graphics courses. But nope, during my first year, all of the PL + graphics faculty left to join other universities / industry.
Everything newer you will learn on the job.
I think these days, a proper compilers course should also be a comparative study of programming languages. There is a lot to learn from comparing how C++ vs Python vs Haskell go from text -> machine code. There are both practical and theoretical aspects to this, and I would expect a graduate-level research-focused course to cover the more theoretical side.
This may be true for graduate programs, but I wholeheartedly believe that undergrad Comp Sci programs should be completed at a 4-year school mostly because the course work is less specialized, much easier to teach and generally attracts pretty decent pool of instructors at most schools.
Additionally, bootcamps are not online courses and shouldn't be conflated. A 6 to 9-months bootcamp goes through material at such a high pace that it often becomes counter-productive. I have a friend who paid $10K for a 6-months bootcamp, got burned out and learned very little in the end. The entire experience basically turned him away. I have another friend who started Comp Sci program at a community college and he's been absolutely happy with overall experience and pace. I have helped both of them and would generally recommend staying the hell away from bootcamps, especially with no prior programming experience.
Teaches you how computers, languages and operating systems work from basic principles
* Little Schemer, Seasoned Schemer
Teaches you about the power of functions, recursion, various CS topics and lisp's syntax
* K&R C Programming Language
Teaches you about lower-level programming
I came upon this book/course: https://mitpress.mit.edu/books/elements-computing-systems-se...
From a Hacker News post IIRC, and it helped more than I can express. At least for me it filled in all the little gaps in the computing stack and really made me "get it". I could relate to the old timers complaining about how everything is so bloated now.
The course is very hands on and you build a "full stack" from logic gates to an operating system that can run programs. It's truly amazing, and I think I learned more from this book than most of my college courses.
I believe it's also offered for free on coursera and has a website: https://www.nand2tetris.org/
But if you really want to know how a full computer works from the ground up, I think this book would be well worth the time.
If you've already determined that you can't do something because it's just too complicated, there's a strong chance you'll be proven right. In my experience, all of the things we software people deal with down to networking, hardware drivers, task scheduling, you name it, were created and explained by humans. Nature constrains us with time and space performance and quantum mechanics. If another human was smart enough to create this stuff ex nihilo, certainly with some work you can be smart enough to understand it.
> My Computer Science degree and the newer degrees do not focus on these, they have become 4 year long coding boot camps that focus on getting people job ready with AWS, React, Ruby or whatever trendy. This is true for the majority of the not so prestigious universities out there.
No one is going to give you a checklist for how to be creative or pursue your own goals. You have to do it. If you're capable of asking this on HN, I think you're capable of setting a few goals. Just write them down and put the deadline a long ways away.
> I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general. I am pretty sure a lot of people are or were in the same boat as me. I don't really have the time(Have wife & kids) to take a compiler course or set out to build my own OS, what would be the easiest way to learn these lower level things?
I'm in my thirties. When I was in school, there were a lot of kids who were just punching the requirements and doing what they were told to get by. Any time they got stuck, they'd have to ask the professor or TA in office hours. Others of us learned that we could solve a lot of problems ourselves if we cut them down to their essences and wrote small programs to see what was actually going on. That heuristic was very helpful.
Once I really did get stuck after doing this, and went to office hours. My professor (an old school Unix graybeard) was so thankful that I had put in the leg work of examining the problem with "toy programs" as he called them that he gave me an exceptionally detailed explanation for what was going on, and how I could show this to myself. He made the comment that a lack of curiosity was a problem among my peers, but that it can be grown and cultivated like other skills.
One other thing the older generations did better than we do is look at the history of our field. If you don't know who Alan Kay, Doug Engelbart, Nikolaus Wirth, Alan Perlis, Ivan Sutherland, Leslie Lamport, Margaret Hamilton or Ted Nelson are, go find some information about them (most have talks or demos on YouTube). Then find out the rest of the missing pantheon of computing. Our industry goes through fads in sort of a fashion culture, and in my experience it leads people to feel burned out our stuck exactly as you described. There are some enduring truths which our field does a poor job of passing onto posterity. Turns out more people had interesting ideas beyond Dijkstra and Turing.
> 1. Reading the Dragon book(I find this would be useful only once you have built a bad compiler) 2. Paying people older than 40-45 to teach me some C/C++ and some tricks of the trade (expensive but I gained a huge amount of knowledge in a short time)
My advice: pick a small, interesting problem and tackle it for some set amount of time each day, maybe after the kids are in bed or your wife is enjoying a bubble bath or whatever. It doesn't have to be long - maybe a half hour to an hour. Just do it consistently.
I've found that GitHub, YouTube and HN itself have loads of interesting project ideas and tutorials. I find those much better than just reading a book. In those cases, the book becomes supplementary material, and is usually most useful at exploring theoretical issues,...
In the specific area of programming languages, a good example might be to write a toy Forth interpreter.
(Note: There is nothing wrong with vocational schools for those who need to learn specific technologies. In fact, many people with CS degrees could benefit from courses in specific technologies. However, the purpose of a bachelor's degree in computer science is to teach students the fundamentals of computer science, and I believe undergraduates students are done a disservice when they are not taught the fundamentals.)
I highly recommend going through the book Structure and Interpretation of Computer Programs (https://mitpress.mit.edu/sites/default/files/sicp/full-text/...). This may be your first exposure to functional programming in Scheme. Even though it was originally designed as the textbook to MIT's introductory CS course (before switching to Python in the late 2000's), this book is useful for anyone interested in computer science, and it's easier to read for those who already know how to program. By the time you get through the end of this book, you will have written a Scheme interpreter, a logic programming interpreter that can handle unification (don't worry about what that means right now), and a Scheme compiler.
Once you finish that book, then it should be easier for your to delve deeper into the topics of your choosing. Want to learn more about computer architecture? I highly recommend getting the latest edition of Patterson and Hennessy's Computer Organization and Design (https://www.elsevier.com/books/computer-organization-and-des...). Want to learn about operating systems? I highly recommend Remzi and Andrea Arpaci-Dusseau's free Three Easy Pieces (https://pages.cs.wisc.edu/~remzi/OSTEP/). For compilers, you already have the dragon book, which should be easier to read once completing SICP.
To be honest, it sounds like youre disillusioned more with application development and not the particularities of the technologies you work with.
Guess what? I'm in the older generation, I have a CS degree from the late 1980s. I'm comfortable with hardware, compilers, run-times, drivers, operating systems, etc. but all that stuff you listed is like a mystery. I don't know what it is, why or when I would use it, etc other than at a very high level.
I mostly write CRUD apps.
That's where the volume is. There's a big demand for programmers to do web stuff. That demand may or may not continue. It's so stylized that it should have been automated by now. Have an exit strategy.
There are many things in computing that are hard and more interesting, but they tend to be low-volume. From self-driving cars to industrial control, there are many areas of specialization. But not many people do those things. You can get stuck in a niche where there are no jobs near you.
If you're stuck doing web apps, at least learn to do them in Go. The performance is pretty good and there are fewer layers.
Someone recommended Rust. I write a lot of Rust, but for non-web stuff. It's useful for things where you'd otherwise have to suffer through debugging C++. Don't write web apps in Rust; it's the wrong tool for the job.
A user-level understanding of machine learning is useful. At least learn how to start up TensorFlow.
How compilers work, and especially how they optimize, is fascinating, but there's not much demand for people to do that. Distributed database internals are similar. As is networking. These areas are very important, but most of the big problems have been solved and they don't need many people.
Operating systems are also interesting, but things seem to have settled down to either Linux or Windows and mods to them.
The blockchain/distributed finance/cryptocurrency world has a sizable following, but it's mostly the make-money-fast crowd, some of whom are crooks and some of whom are suckers. Be cautious.
This is highly variable. The more rote aspects of web development are indeed automatable: the parts that feel like repeating boilerplate because that's what they are. Creating duplicative controller actions, state updates, etc.
But then there are the parts that would be extremely difficult to automate. Creating rich, unique UI elements with lots of custom logic. Anticipating the user's needs before they realize they even have them. These take a lot of skill and experience, and are truly fun to work on – assuming they're your jam. Lucky for me, they are.
I thought that's what coding bootcamps were for?
Do you want to become a specialist or a generalist? You can dive into one or two of these topics and specialize the rest of your life into it. By the way: a specialist in any of these topics would probably not know any of the fancy buzzwords you've listed in your resume. As a matter of fact, I think you'd have more career opportunities instead.
Keep in mind that the older generation had a "few layers" to explore and could easily understand each slice until the bare metal. These days, it is much more broad and in-depth.
So you want to specialize in something, or become a generalist?
It's very easy to get stuck in the world of CRUD because it pays the bills, but as you noted there are a lot of technologies.
Shifting out of that world requires risk tolerance because there will be failure after failure. That wall you speak of is the nature of hard things and the time you can dedicate bashing your head against that wall.
My advice for climbing/breaking the wall:
Pick a small project that you find interesting: build an OS, game, programming language, VM, etc... and just do it. The key is to cross the hump of understanding and get something simple working, and the good news is that conceptual understanding isn't that far away. The tool to help you get unblock is writing. Write the comments of what you are trying to do, and break it down until it becomes code-like. Read books on the subject and bounce back to writing what you read.
However, going from conceptual understanding to professional environment is different because the concepts start to inter-relate and the complexity manifests.
Prior writings on this:
http://jeffrey.io/writings/thoughts-for-young-hackers-and-so...
http://jeffrey.io/writings/build-first-buy-second.html
What I don't understand from your post is what your end goal is? Why do you want to learn low-level stuff like building compilers or operating systems? Is it for work? Desire to improve yourself? What can help you better depends on the answer to this question.
I can offer a couple of pointers I hope they can help:
1. I was frustrated by my lack of understating of low-level concepts and how it prevented me from understanding the more complex pattern. The thing that clicked from me was building a small CPU and related hardware (timer, VGA frame-buffer, etc...), I don't remember which site/guide I follow, but today ben eater videos or nand2tetris seems good choices.
I also read a lot about game console programming (GameBoy color in particular), a lot of that low-level concepts were really a lot simpler to grasp with the visual demo to demonstrate. I'm also constantly surprised how much concepts like interrupt, how a mmu works, etc... are useful even when writing high-level code like I'm doing now.
2. Stop considering Databases, compilers, and operating systems as a single unit to learn. They are made of a lot of smaller and simpler components that together form a system. And while when are together seem very complex, often when divided into smaller part became a lot easier to learn even if the total "complexity" remains the same. Take compiler, for example, there is code generation, optimization, and syntax parsing. Instead of trying to build or understand a compiler start by learning the basics of parsing, for example by building a simple chatbot (one of the first things I wrote was an IRC bot), evolve it to understand more complex queries (for example SQL).
Related to this: One thing I could not understand was how big relational databases, like MySQL, worked, a lot of the literature on the topic seem to have a perspective that didn't help me (like the math behind it). The ah-ha moment to me was with sqlite and seeing that at the base was a simple key-value store, and everything else was built on top of this. One takeaway, for me, was that when you feel that you don't understand something, maybe you are asking the wrong question! in my case I keep searching "how relational databases work" instead of what I was really interested in "how relational databases are implemented".
3. I would not start with a thing like "the dragon book", from what I see is an extremely complete book about "everything you need to know about compiler" (hasn't read it, so correct me if I'm wrong). Book like that can seem intimidating and you risk of reading a lot of material without really learning and I feel more like an "information dump". Search for smaller book or course that are an "introduction" to compiler instead of the "everything you need to know", they may not teach you everything, but you will have a better understating of "core" concepts that allow you to tackle more complex topics a lot easier.
Side note: in the specific case of compiler search for a book/course that focuses on building an actual small language! The book my professor gave to me only had a disjointed example for each topic. Something like "crafting interpreter" with a precise goal is MUCH easier to understand while teaching the same concept.
Hope this can help, or at least a starting point for a discussion.
edit: formatting
The older generation are in this place because back then none of the technologies you listed existed and so they had to know the more low level things.
A book that might help some of your knowledge gaps (at least directionally) is The Imposter's Handbook[1] by Rob Conery.
If you're specifically aiming to get away from CRUD take a look at the various Domain Driven Design books by Eric Evans[2] and others. This is the opposite direction to your ask though (more high level than low level).
If you want to go deeper in Java, read Effective Java[3] and Java Concurrency in Practice (JCIP)[4]
Lastly, if you want to experience the reasons why some of that early comp-sci stuff happened the way it did, playing with expensive hardware is the least effective way to find understanding of constraints. Instead consider grabbing devices with significant limitations like an Arduino, ESP32, or similar (I'm partial to the Wio Terminal[5] and M5Stack[6] as interesting easy to get started devices, but others would suggest various Arduinos or Raspberry Pi devices). Realistically you're going to learn much more by coming up with a self-challenging project and completing it than just by learning it for the sake of learning.
[1]: https://bigmachine.io/products/the-imposters-handbook/
[2]: https://www.amazon.com/Domain-Driven-Design-Tackling-Complex...
[3]: https://www.amazon.com/Effective-Java-Joshua-Bloch/dp/013468...
[4]: https://www.amazon.com/Java-Concurrency-Practice-CONCURRENCY...
[5]: https://www.seeedstudio.com/Wio-Terminal-p-4509.html
[6]: https://m5stack.com/
For consuming non-beginner material, it’s okay to read a blog post and not fully understand everything. Things will click together over time.
I would look for jobs in software companies who make stuff you want to learn to make, but for some parts of their products they also need these Docker/Scala/Kafka/etc. from your resume. If you are good at these things, they gonna pay you well i.e. enough to support a family. I think over time you’ll inevitably learn their lower-level stuff you have to integrate with. You can then change position within the company should you want to.