How to keep up with AI/ML as a full stack dev?

I'm part of a team building developer tools for real-time AI use cases (voice and video). I feel like I have three overlapping perspectives and goals re this new stuff:

1. To figure out what we should build I need to have a good understanding of what's possible and useful right now.

2. I talk to our customers a lot. Helping them understand what's possible and useful today (and what that might look like six months or a year from now) is part of my job.

3. I think this is a step-function change in what computers are good at, and that's really exciting and intellectually interesting.

My AI information diet right now is a few podcasts, twitter, and email newsletters. A few links:

  - Latent space podcast and newsletter: https://www.latent.space/podcast
  - Ben's Bites newsletter: https://news.bensbites.com/
  - Ethan Mollick newsletter: https://www.oneusefulthing.org/
  - Zvi Mowshowitz newsletter: https://thezvi.substack.com/
  - Rohan Paul twitter: https://x.com/rohanpaul_ai

Hacker News is a good source for news.

As far as learning, you have to build something.

I suggest you just start with example code from the OpenAI or Anthropic website for using the chat completion API. They have Node.js code.

r/locallama on reddit is interesting.

On Youtube, see Matt Wolfe, Matthew Berman, David Shapiro. Not really developer-focused but will mention developments.

You can search for terms like 'AI Engineer' or "agentic" or "LangChain" on youtube also.

To get motivated, maybe play around with the replicate.com API. It has cut and paste examples and many interesting models.

More ideas: search for "Crew AI" on X/Twitter.

ML moves fast, but not as fast as you probably think. There's a difference between innovations in architectures and demonstrations of them in domains (both are useful, both are necessary research, but they are different).

Instead, keep up with what tools are relevant to you. If things are moving fast and aren't sticking, then in a way they aren't moving fast, are they? You're just chasing hype and you'll never keep up.

On the production side, I also see a common mistake of relying on benchmarks too heavily. I understand why this happens, but the truth is more nuanced than this. Just because something works well on a benchmark does not mean it will work well (or better than others) on your application. ResNet is still commonly used and still a great option for many applications. Not everything needs a 1B+ transformer. Consider your constraints: performance, compute, resource costs, inference time, and all that jazz. Right now if you have familiarity (no need for expertise) in FFNs (feed forward/linear), CNNs, ResNets, and Transformers, you're going to be fine. Though I'd encourage you to learn further about training procedures like GANs (commonly mistaken as an architecture), unsupervised pretraining (DINO), and tuning. It may be helpful to learn a high level of diffusion and LLMs, but it depends on your use cases. (And learn whatever you're interested in and you find passion in! Don't let need stop you, but if you don't find interest in this stuff, don't worry either. You won't be left behind)

If you aren't just integrating tools and need to tune models, then do spend time learning this and focusing on generalization. The major lessons learned here have not drastically changed for decades and it is likely to be that way. We do continue to learn and get better, but this doesn't happen in leaps and bounds. So it is okay if you periodically revisit instead of trying to keep up in real time. Because in real time, gamechangers are infrequent (of course everyone wants to advertise being a gamechanger, but we're not chasing every new programing language right?). Let the test of time reduce the noise for you.

  >  I most often can’t see any use case for AI/ML in our products

Look to solve problems and then right tool for the problem, don't look for problems to justify a tool (fine for educational purposes).

Run the following models:

- Speech-to-text - Text-to-text - Text-to-speech - Text-to-image - Image-to-text - Text-to-video - Video-to-text

Start by integrating third-party APIs, and later switch to open-source models.

Implement everything using your preferred backend language. After that, connect it to a frontend framework of your choice to create interactive interfaces.

You want use your own data? Put it in a database and connect it to your backend, and run these models on your database.

Once you’ve done this, you’ll have completed your full stack development training.

I'm admittedly a skeptic on all this so take what I am about to say with a grain of salt: You should trust that voice. We're in a hype cycle. It was VR before and crypto before that. Big tech is trying _very_ hard to convince you that you need this. They need you to need this tech because they are lighting billions on fire right now trying to make it smart enough to do anything useful. Short of them coming up with a truly miraculous breakthrough in the next 12 to 24 months (very unlikely but theres always a chance) investors are gonna get fed up and turn off the money fountain.

It's always a good idea to learn and grow your skillset. I am just not sure this is an investment that will pay off.

Regarding your projects, either just brute force into an existing one, or start a new project. For the former, the purpose isn't to make the product better (exactly) but for you to learn. For the later, OpenAI and Anthropic APIs are good enough to mess around and build a lot of different things. Don't let analysis paralysis stop you, start messing around and finding out.

Would your product benefit from recommender systems, natural language input/output, image detection, summarization, pattern matching and/or analyzing large datasets?

If so, then maybe ML can help you out.

I'm of the opinion that if you need ML, you'll eventually realize it because the solutions to your problem you find will be served by applications of ML.

That's to say, while doing research, due diligence, etc, you will inevitably stumble upon approaches that use ML successfully or unsuccessfully.

Examples:

- Build a useful bash script using ChatGPT prompts and blog about it

- Build a text summariser component for your personal blog using Xenova / Transformers.js

- Build an email reply bot generator that uses ChatGPT prompt with sentiment analysis (doesn't have to actually send email, it could just do an API call to ChatGPT and print the message to the screen).

Just a few small examples and maybe a course or two (e.g. Prompt Engineering for Developers) should look great.

However I question how many companies really care about it right now. Most interviews I've done lately didn't bring it up even once.

But that said, maybe in a few months or year or so it will become more essential for most engineers.

I am in a similar position to you… I have a job they the application of AI to that job isn’t readily apparent. My posture during all this is to use AI products as they become available that are appropriate and help me, but ultimately I’m waiting for the market to mature so I can see if and how I should move forward once the bubble pops and directions are more clear. I have little interest in running on the hamster wheel that is bleeding edge AI development, especially when I already have a job that doesn’t need it.

It's definitely worth trying them out as a user just to see what they're capable (and incapable of). There are also some pretty interesting use cases for them for tasks that would be ridiculously complicated to develop from scratch and "it just works" (ignoring prompt poisoning). Think parsing and summarizing. If you're an app developer, look into edge models and what they can do.

Otherwise dip your toes in other model types - image classification and object recognition are also still getting better. Mobile image processing is driven by ML models at this point. This is my research domain and ResNet and UNet are still ubiquitous architectures.

If you want to be sceptical, ignore AI and read ML instead, and understand these algorithms are just another tool you can reach for. They're not "intelligent".

Try writing single page web app or command line python app using the Claude 3.5 chat. Interact with it like you might in a pair programming session where you don’t have the keyboard. When you’ve got something interesting, have it rewrite it in another language. Complain about the bugs. Ask it what new features might are it better. Ask it to write tests. Ask it to write bash scripts to manage running it. Ask it how to deploy and monitor it. Run llama 3.1 on your laptop with ollama. Run phi3-mini on your phone.

The problem is that everyone says they aren’t going to get better, but no one has any data to back that up. If you listen carefully it's almost always based on a lack of imagination. Data is what matters, and we have been inventing new benchmarking problems because they're too good at the old ones. Ignore the hype, both for and against: none of that matters. Spend some time using them and decide for yourself. This time is different.

Offtopic, but today I encountered my first AI-might-be-running-the-business moment. I had a helpdesk ticket open with IT for an issue with my laptop. It got assigned to a real person. After a few days of back-and-forth, the issue was resolved. I updated the ticket to the effect of, "Yup, I guess we can close this ticket and I will open a new one if it crops up again. Thank you for your patience and working with me on this." A few seconds later, I get an email saying that an AI agent decided to close my ticket based on the wording of my update.

Which, you know, is fine I guess. The business wants to close tickets because We Have Metrics, Dammit. But if the roles were reversed and I was the help desk agent, seeing the note of gratitude and clicking that Resolved button would very likely be the only little endorphin hit that kept me plugging away on tickets. Letting AI do ONLY the easy and fun parts of my job would just be straight-up demoralizing to me.

I'd suggestion learning about pgvector and text embedding models. It seems overwhelming at first but in reality the basic concepts are pretty easy to grok.

pgvector is a Postgres extension, so you get to work with a good traditional database plus vector database capabilities.

Text embeddings are easy to work with. Lots of models if you want to do it locally or adhoc, or just use OpenAI or GCP's api if you don't want to worry about it.

This combo is also compatible with multiple vendors, so it's a good onboarding experience to scale in to.

Yes, absolutely. The most effective we I know to develop that sort of intuition (not just in AI/ML, but most subjects) is to try _and fail_ many times. You need to learn the boundaries of what works, what doesn't, and why. Pick a framework (or, when learning, you'd ideally start with one and develop the rest of your intuition by building those parts yourself), pick a project, and try to make it work. Focus on getting the ML bits solid rather than completing products if you want to get that experience faster (unless you also have no "product" experience and might benefit from seeing a few things through end-to-end).

> stay relevant in the long run

Outside of the mild uncertainty in AI replacing/changing the act of programming itself (and, for that, I haven't seen a lot of great options other than learning how to leverage those tools for yourself (keep in mind, most tasks will be slower if you do so, so you'll have a learning curve before you're as productive as before again; you can't replace everything with current-gen AI), and we might be screwed anyway), I wouldn't worry about that in the slightest unless you explicitly want to go into AI/ML for some reason. Even in AI-heavy companies, only something like 10% of developers tangentially touch AI stuff (outside of smallish startups where small employee counts admit more variance). Those other 90% of jobs are the same as ever.

> keep up my learning in these areas

In addition to the general concept of trying things and failing, which is extremely important (also a good way to learn math, programming, and linguistics), I'd advise against actively pursuing the latest trends until you have a good enough mentor or good enough intuition to have a feel for which ones are important. There are too many things happening, there's a lot of money on the line, and there are a lot of people selling rusty pickaxes for this gold rush (many intentionally, many because they don't know any better). It'll take way too much time, and you'll not have a good enough signal-to-noise ratio for it to be worth it.

As one concrete recommendation, start following Yannic Kilcher on YouTube. He covers most of the more important latest models, papers, and ideas. Most of his opinions in the space are decent. I don't think he produces more than an hour per day or so of content (and with relatively slow speaking rates (the thing the normal YT audience wants), so you might get away with 2x frame rate if you want to go a bit faster). Or find any good list of "foundational" papers to internalize (something like 5-20). Posting those is fairly common on HN; find somebody who looks like they've been studying the space for awhile. Avoid advice from big-name AI celebrities. Find a mentor. The details don't matter too much, but as much as possible you'd like to find somebody moderately trustworthy to borrow their expert knowledge to separate the wheat from the chaff, and you'll get better results if their incentive structure is to produce good information rather than a lot of information.

Once you have some sort of background in what's possible, how it works, performance characteristics, ..., it's pretty easy to look at a new idea, new service, new business, ..., and tell if it's definitely viable, maybe viable, or full of crap. Your choice of libraries, frameworks, network topologies, ..., then becomes fairly easy.

>> other people saying to build something simple with LLMs and brag about it

Maybe. Playing with a thing is a great way to build intuition. That's not too dissimilar from what I recommended above. When it comes to what you're telling the world about yourself though, you want to make sure to build the right impression. If you have some evidence that you can lightly productize LLMs, that's in-demand right this second. If you publ...

This said, I have started some experiments and like in all hyped technologies, there is some useful substance there as well.

My recommendation is to start with some very simple and real, repetitive need, and do it with the assistants API.

I started by turning a semi-structured word document of 150 entries into a structured database table. I would have probably done it more quickly by hand, but I would not have learned anything that way.

I think the sweet spot for generative AI right now is not in creative jobs (creating code, creating business communication , content creation etc.) but in mundane, repetivive things, where using generative AI seems like overkill at first.

I would suggest following / reading people who talk about using Claude 3.5 sonnet.

Lots of people developing whole apps using 3.5 sonnet and sometimes cursor or another editor integration. The models are getting quite good now at writing code once you learn how to use them right and don't use the incorrect LLMs (a problem I often see in places other than twitter unfortunately.) They seem to get better almost weekly now too. Just yesterday Anthropic released an update where you can now store your entire codebase to call as part of the prompt at 90% token discount. Should make an already very good model much better.

Gumroad's CEO has also made some good YouTube content describing a lot of these techniques, but they're livestreams so there is a lot of dead air.

https://www.youtube.com/watch?v=1CC88QGQiEA

https://www.youtube.com/watch?v=mY6oV7tZUi0