Ask HN: Beyond CRUD and ETL – How to Grow Professionally?

how about learning something related to DevOps?

Stream processing is pretty cool. Instead of doing batch jobs try to set up something real time using Spark.

Work on a product, instead of doing one-off projects for other companies.

You say you want to get to a director level or higher. Given the industries you listed no technical skill is going to put you in the Director or Exec seats at a larger organization. If you were talking SMB then yes, technical skills could assist you to getting that next level, but even then it is more about people skills, planning skills, budget management and most of all how to affect revenue.

What I would say is that people are finally waking up and realizing the data we collect is where the gold is at, not in selling a widget for $5.99. The data tells us what to sell, when to sell it, what to price it etc. So everything you know is driving business today, especially in insurance, finance and advertising. So don't run from what you know, change your focus into leading others to do the "tasks" of what it takes and switch your brain to architecting solutions that make companies more money. Don't be a manager of people, be a thought leader, an inventor that can turn an idea into action and quickly.

As for what to do to grow. Seriously, if you don't know a lot about accounting, learn the basics of how a company runs accounting. Learn how what you are involved in might affect revenue and how you can personally help affect revenue with your team. Stop just taking orders to create this next ETL or report, and invest some extra time to figure out how you can invent a solution (doesn't have to be fancy) that can help senior execs or help the company make more money. That is how you can move up, even if it isn't in your current organization you will be able to say, I thought of, designed and implemented X that brought the company Y more revenue through Z means. Or I created X report that allowed the senior team to see data in a way that they weren't prior and it has lead to Y changes etc. As a last thought, go spend time with sales teams, they will complain a lot about things they need and aren't getting. Help them get some of those done and they will love you and brag about it. They will also then want to see you move up to help them more.

I think if you want a Director or better kind of position in the technical side of any organization, what you need probably isn't necessarily more technical "depth", but a demonstrated track record of delivering value by proactively applying both technical and organizational/managerial skills to solve business problems (or capitalize on business opportunities; two sides of the same thing), preferably problems or opportunities that you identified for your organization. (You'll probably also gain knowledge, at a high level at least, of all the technical areas involved in that solution in the course of doing that, and you can dive into those more deeply if they are interesting to you, but they aren't the things that are likely to be most critical at getting the positions you seem to be looking for the industries you are interested in.)

And, of course, the people skills to sell an organization on both your past track record of doing that and your ability to do the same in their organization.

I don't think you'll get a role as Director or VP simply on technical knowledge.

This is what the Harvard Business Review has to say (pretty light on actionable advice though):

https://hbr.org/2011/03/the-new-path-to-the-c-suite

Another article:

http://www.computerweekly.com/feature/Career-management-The-...

I guess an MBA could be of help, but it's useless unless you already have connections and all the other requirements in place.

http://www.informationweek.com/it-leadership/6-must-have-ski...?

A recurring piece of advice is having been a consultant.

The other is building your relationship-building prowess.:

"Strong project management skills are critical to running a successful IT shop. But if you're hoping to climb that corporate ladder, it's important to know how to build relationships up, down, and sideways. In fact, according to a recent poll from SearchCIO, conducted of 875 senior and mid-level IT executives, CIOs who earn the highest salaries make building relationships with top executives more of a priority than managing IT projects.

"Being a CIO requires an ability to develop relationships in all directions--with your boss, outward with one's peers, with other C-level executives, heads of business units and relationships downwards as well,"

You are a manager but you didn't state anything in your story about the people you hired and developed, the projects you led or rescued, or how you contributed to your peers, managers and companies success. Do that.

This is intended as no way offensive, but your written English isn't great (I'm thinking it isn't your first language?). The first step to getting an executive position would be to improve in that area.

The second thing about being a Director or VP is that it's all about team building, not technical skills. Technical skills are a baseline, but a good executive is an expert at attracting, hiring and retaining good talent. So that is another area where you'd need to prove yourself out (you didn't mention anything about hiring or recruiting).

    near -> nearly

    well familiar - delete the 'well'

    Since last 3 years -> For the last three years.

    I work as manager -> I work as a manager *or* I am a manager *or* I have worked as a manager.

    What are things -> What are the things

off topic. I am a step behind the OP. I am trying to get into the role that the OP describes he is in for the last 3 years. What would be some ways to move into big data and BI other than quitting my current job that doesn't deal much with Big data. Is the solution to this to contribute to Open Source?

Just remember this the hire up you go the more it's about sales. so with that in mind. You need to network with people that can hire a director and impress the shit out of them. Then, when they are looking for one... your the guy.

  "I am tired of ETL , data cleaning , slicing and dicing projects now."

My instinct says that one has to love what one does (the dirty bits especially) for one to evolve naturally into the next greater phase of one's career.

Lots of IT shops nowadays are struggling to contain a BIG MESS while still providing new features that the business needs. So the skills that are most useful at director level and above, are how to cope with the BIG MESS and how to extricate the business from the BIG MESS. Most likely this involves lots of compromises and the usual techniques of managing in the midst of chaos by evolving bit by bit to get closer to some overall architectural vision.

As far as the technology goes, there are many ways to have an IT infrastructure without a BIG MESS. So technology does not help you here. What matters most is deciding on an architectural vision and leading the whole company towards that "better place". If you believe in silver bullets, then you will fail.

If you read Peopleware and it made sense to you, then you are probably on the right track. There is a huge literature on management techniques to guide you. Pick a path that you can walk and just do it.

How to grow professionally? I have no clue. I made some observations of my own.

If you work at a company, you would expect the hierarchy to notice your ambition and guide you to next level. In practice this does not happen or happens to only few people for variety of reasons. The company does not produce enough market value, there are too many skilled people with ambition, there are no networking opportunities, you were not noticeable that nobody took you under their wing, you are too odd to fit in, the industry is too young etc. One could only prepare self to look good, talk the talk and wait for an opportunity to arise. When I see so many managers ass-kissing their bosses, I believe they are doing the same, preparing for receiving. I once noticed an ambitious director making a spectacle while taking pictures of his boss when the boss was speaking to an audience. I really have no clue if these social engineering techniques work, but I notice so many people do it.

If you are perceived technically strong, the hierarchy does not see you as leadership materiel. I witnessed this event that changed my perspective about over stressing my technical abilities. A bunch of people, almost all of them out of touch with technology sat down and the conversation started like this with someone saying -

"I know this large company which takes this amazingly smart technical people and made them managers and directors and they are facing difficulty, in our awesome org we should never do that”.

A bunch of other people around agree and they give their own experiences of horror stories. Such things gain wider acceptance because these meetings have less representation from someone who still see themselves as technically competent and current. I was in such a meeting and asked the group, if leadership is a skill that people are born with and what should technically strong people do for their career advancement? The answers were more ridiculous. Technical skill is seen as something you only need to have up to a limit and any more than that is not useful to the company. There is a ceiling if you are technically smart.

The environment is self selecting what is needed for their business to strive. I found it smart to adapt sometimes but then I see a lot of these young people in silicon valley do amazing things and give themselves great titles. We both know that skills are transferrable between companies but titles may not be. I have more respect for these young people who create their own reality and make everybody believe theirs. Now Zuckerberg is still a CEO and they are still teaching people in MBA the skills to be a CEO or CIO or whatever it is and the magazines writing about the rules to be a CEO or CTO or CIO.