Ask HN: How do I go from being a generalist to a specialist?

Hi HN. I am at a point in my career where I should start choosing my path. I have a lot of breadth in terms of exposure and knowledge but I lack depth at the moment.

I have worked extensively with SQL, python/flask/django for app development and backend development, spark and airflow for complex jobs and orchestration, python for scripting and automating tasks, many POCs which have been approved for the next phase, which include ELK stack, Kafka, FastAPI, etc. I also have experience with docker and kubernetes for deployment and monitoring. I have experience with Azure services but I believe that principles across vendors (aws/gcp) might remain the same hence would not be an issue to learn them.

Apart from my work experience above, I have dabbled in other languages including java, scala, golang, c/c++.

My huge concern is trying to put a label on myself. I have recently started applying for jobs and the first question I am asked is whether I classify myself as a backend engineer or a data engineer. My role on paper is SE so I am not sure about this. I am also confused how to decide on the skills to learn to advance in my career. On the one hand I saw a post recently on HN where an ex Uber employee explained in detail how complex the software is. This was really amazing and something I would like to do (this I think is backend development). At the same time am very interested when I read about people implementing a big data ecosystem involving tools like kafka, spark, airflow to manage data efficiently (this comes under data engineering I think). I like both the roles when I read the job descriptions but I am confused how to narrow down my choice.

Are there more data points from HN how they chose their careers? Any pointers on backend or data engineering would also be helpful.

Apologies for my english, it's not my first language.