> There is a significant gap between how software is currently developed in this space versus how it should be developed. The vast majority of genomics-related software is not written with speed or reliability in mind.
True, but working in academia is very VERY different working in a tech/product company.
> This state of affairs makes it difficult for anyone other than the original author to contribute to these code bases, further cementing the one-maintainer policy.
Who wants to fix other peoples code mess? This is a no-no if you want to promote a job opening.
I do. It's my bread-and-butter. I call myself a code janitor. I live by books like "Working Effectively With Legacy Code" and "Kill it With Fire". But I have my limits. Academic code has.. coded, in the medical sense, and it can't be revived. Put a DNR on it.
Would love to but I don’t think academia will want masters at least and years of industry experience will be discarded completely. I have 6 years experience in data intensive IoT applications and yet that would not be considered useful by academia
I left academia for that reason; there was no advancement path that didn’t involve more advanced degrees, and that wasn’t something I was interested in at the time.
> "Fundamentally, RSEs build software to support scientific research. They generally don’t have research questions of their own — they develop the computer tools to help other people to do cool things."
Several of the job boards linked don't have any job listings, most don't provide a salary range, several require advanced degrees, and none specify whether remote work is possible.
If I can get a better salary and working conditions at some crappy no-name startup, why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade?
I think your phrase here sums up how many people feel:
> why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade
This is changing in my experience, albeit slowly. And really, this is what I'm calling on us, as a community, to do better on.
The reason you _would_ work at these organizations is because (1) the subject-matter is really interesting, (2) there are hard problems to be solved, and (3) you wake up every morning knowing that you are working on something that will have an impact on the lives of people around the world.
Or, and hear me out here for a second, people go where they’re most valued. Why would someone volunteer to go work somewhere where they’ll get less money, have less flexibility, be treated as second-class employees, have less work perks, etc. when so many alternatives exist? Doubly so in this economy.
Maybe genetic companies should catch up in workplace etiquette instead of recommending that SWE’s lower their expectations.
I agree. With that said, I do think we are getting there. The pay is becoming more comparable, and I think software engineers are becoming more and more valued in these companies/organizations.
Because they find it rewarding in some other way? I agree that that way is not the conventional wisdom, but it exists, it turns out that some people value doing things that provide a demonstrable benefit to humanity.
Also he second class citizen thing has diminished over the years. It still exists but there are plenty of companies where that's no longer true. This is in stark contrast to when the field was getting off the ground, for instance it wasn't uncommon for benefits like PTO to be tied to your degree level & not length of employment.
The problem is that it isn't just about pay, it's about everything: autonomy, flexibility, culture, work quality, respect... If an organization can reliably convince—show, not tell—that it will be a much better place to work overall, I'm certain they can hire highly skilled engineers even if they can't compete on pay.
My experience is that most non-tech organizations can't or won't.
The pay isn't all that important to me, as long as I can live on it, but this. It was so obvious when I was working in academia that because I wasn't myself an academic that I was just lab help, no better than the person who washes the test tubes and beakers.
I don't quite understand this post. In my experience lab technicians in academia are highly valued (however I mostly have experience with clean room staff), however they are support for the researchers, who drive the research agenda (well actually it's the funding primarily). What exactly do you expect to not be what you call "a second class citizen"?
Academia is a hugely elitist pyramid with well-demarcated layers, and the lab technicians are at the very lowest level of the pecking order, down with the cleaning staff.
They might be "needed", but by and large are they really "well respected" or "valued"? Not really, sad to say.
I charge a high rate not because I need the money but because I need to stop people from wasting my time. I’ve worked at these places before where they exploit your altruism and dedication to craft to extract more work out of you for less. Concretely one of the places refused to hire a frontend dev to help so I got stuck wasting time churning frontends. Charging more encourages them not to do that.
Yeah. It’s one thing if the only way the job is worse is in pay, it my experience has been that if the pay is significantly worse, the job is worse off in every way.
I think the honest answer is, it depends where you work. Where I work, we have that autonomy. But we also have leadership that understands and respects the software engineering craft.
I have experience working in academia. I started out working for a medical school in fact, on bioinformatics. It's been a while, so I have kind of forgotten the problems, but I'll do my best.
1. Academic code. Not one institution would pass the Joel Test[1]. You pretty much covered some key points in your first paragraph, so I see not much has changed. The best predictor of how something will perform in the future is how it has performed in the past. Just hiring good software engineers won't change the system in which they work.
2. Academic bureaucracy & administration. I've worked for large Fortune 500 companies with less byzantine org charts. I've been matrix-managed. The siloing in academia is crazy.
3. Advancement. Because it's academia, advanced degrees are everything. My first boss in academia had a PhD. His job? He ran the student computing lab. My second boss was an MD/PhD. Great guy, but treated everyone like a lab assistant. I went to graduate school for one year and realized it wasn't for me.
4. (added after reading other comments). Completely unrealistic understanding of what developing robust, complex software is like. You touched on this by mentioning how many projects have 1 maintainer. I remember seeing a doctor shopping around his project plan. I'd say it would be a challenge for a high-performing 5-person team. He thought it was a job for a single entry-level programmer.
If you don't have a PhD, don't touch academic jobs with a 10 foot pole. They all got one, and they value the credential to a ridiculous degree.
When I last worked with academia, they essentially thought of me as the same as the guy who maintains the lab equipment, not an actual collaborator on their research.
The academic elitism doesn’t end at PhD. A friend asked a Nobel prize winner their thoughts of different person who was recently awarded a Nobel prize in the same field. Their response “Ah, yes that was one of the lesser Nobels”
How are we "as a community" going to be able to improve this?
It's ultimately down to the cultural norms of the field, as well as the realities of academic funding.
I was a research software engineer (RSE) for the best part of a decade. The best thing that happened to me was being made redundant when my funding ran out, and being forced to work in industry. What a difference (and wholly for the better).
The reasons you give are all nice positives, but they all ultimately are very emotionally manipulative. You're asking people to act against their own self-interest. But this isn't really for the benefit of humanity. It's for the benefit of the PIs who run the research groups, and keeping their little empires running. But the cost to the individual is great. You're sacrificing salary, a career track, advancing your own skills to the full, and in many cases the opportunity to have a life: being able to afford a home and support a family.
In retrospect I, and many others like me, do feel that we were taken advantage of to some degree.
I spent several years on a massive grant, then several years on lots of small short-term grants (12 months, 6 months, 3 months). You can't risk getting a mortgage when you have no guaranteed employment. And it's also very stressful not knowing if you'll be employed in three months time every three months. And unlike in a company, RSEs don't really have a proper career track. There's no real progression. You're a hired help.
RSEs are not treated equally with academics. Let's be really honest about that. We're not. I even had a PhD in the subject area and you're still beneath all of the "real" academics. We're not "partners" in their work. We're the dogsbody's.
If these people want software developers with real chops to work in the field then they need to pay a competitive salary, have a proper career track, and really fix the job stability. And they also need to properly respect the expertise RSEs bring. Unless all of those are fixed, a career in industry will continue to be the only rational choice.
This won't happen though. Tenured academics refuse to consider paying the going rate because that would mean the "hired help" would be earning considerably more than they do. I had already topped out the salary band when I left, and I was earning more than most of the junior-mid-career academics. They are, of course, on fairly poor salaries. They too would earn vastly more in industry, but are mostly unwilling to consider that option as a rule. Their loss. If they truly respected the value they were getting, then they would pay for it. It will not happen though. Most of academia is about climbing the greasy pole and not about advancing the state of the art; there's just no way they'll permit others to sit higher on the salary pyramid than they do.
At least in industry skill and competence and the ability to deliver are highly-valued, and companies will gladly pay for people who are proven to deliver. In practice the work I do in industry (biomedical) has far more positive impact upon the world than anything I did in the academic niche I used to occupy, and is also vastly more enjoyable, with a lot more responsibility and technical depth.
Are you personally planning to stick it out for the whole of your career? Because if I could give you the advice I would have liked to have given myself, it's that you should properly think about where you want your career to go in the medium to long term, and decide when (not if, but when) you will exit to move to industry. Use it as an opportunity to gain some useful skills, and move on to where your skills will be properly valued.
I've seen a posted phd position that was extremely weak academically, because they just wanted someone with a CS degree to implement their pre-existing ideas, but didn't want to pay a developer's salary.
The position kept being posted multiple times over a couple of years. Then I moved on and don't know what happened.
This reads a bit like “what can this place do for me?” instead of “how can I make the place I work at better”.
High potential areas like genomics that are behind on software are amazing places for talented software people with a givig attitude to have a big positive impact.
I can have a big positive impact and make the startups I tend to work for better places AND get paid well and be respected. It seems the "potential" here is a result of a lack of care and maintenance for legacy codebases, and not, say, driving innovation.
In general salaries would be lower than what you can get in a standard software role. Some of the ones on that list allow remote work, others are more limited. The tradeoff is that you're working towards the betterment of humanity. Whether or not that's worth the tradeoff for you is a personal decision.
Agree that not everyone will buy this. But it is why many people are working in positions that are otherwise worse on paper. It's up to them to decide if its real or not.
> If I can get a better salary and working conditions at some crappy no-name startup, why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade?
Amen, couldn't have said it better myself. I'm sure it's very worthy and all working on a genomics project that aims to eradicate some killer disease, but you need to live and provide for your family while you're doing it.
I’ve had a growing interest in the power of DNA and what the data can be used for since discovering no less than 3 family secrets (one of which pertaining to me) after taking an Ancestry DNA test. Did I know I was going to find 18 half siblings the moment my results came in? Nope, but yet there they are, listed in order of most shared DNA.
Despite my interest, I’ve found that landing a job in this field at my desired compensation level is very difficult especially if you not have the ”correct” academic background. Who does a double degree for computer science and forensic genealogy? I’m sure some people but for $75k/yr you’d think the companies need to at least adjust their expectations.
Yeah the low pay is one thing, but in my experience a lot of the academic jobs seem to want a domain scientist who can do programming, not the other way around.
Yeah, I agree, I looked into this before, and the pay doesn’t come close to other swe jobs, as far as I have seen whenever I look. It is usually like 2x less, it’s hard to want to choose that just to work on something a bit more interesting. I even have a background in bioinformatics, but I never found anything that compensates it as much as pure swe roles.
I've been a software engineer in this space.
I just want to say that there is exactly 1 job (non-intern) job between Microsoft, Google and Amazon listed according to the search links provided in the article.
I keep thinking of this space but don't know where to start. Any pointers for good resources? - books that are accessible to software engineers with no background in genomics, open source projects which are widely used, etc.... in short a good place to start in exploratory/hobby/learning mode.
Science programming jobs suck. You get all the bad parts of academia, including less money, plus you're seen as a janitor rather than an engineer, and you get to deal with scientists all day.
Tooling roles in SWE in every other field are highly regarded. Why not here?
it depends. I've been in science support roles where people are genuinely grateful for the help, and its _really_ interesting to get to peek in on people's research.
it depends on the role. it worked out really well for me when I got to drop in and do piece work on lots of projects in different fields. working on a larger software development project can be really painful and demoralizing because the people running it don't really understand how the sausage gets made.
Because it's not what sells. It's literally a tool, and if you don't deliver the level of perfection they're used to get from sequencing, NMR or assay testing machines, you're the PITA. You really have to bring something very interesting to the table to earn some status, and software engineering just doesn't. It's too far from the core business. Think of the attitude SWEs have towards sales people...
The thing is many will pay big bucks to contractors/consultants/IT services/LIMS systems, but if you’re an employee, nope.
They have a hard time having someone with a BS or MS making 50-75k more than a freshly-minted PhD.
I just left a job in pharma because I cannot do it anymore (salary being a big one, but my experiences reflect many in this thread).
They spent 500k on a consulting company to build a few NGS processing pipelines. This was built using a framework I was unfamiliar with. I re-factored one of them and was able to increase runtime by 60% in a couple weeks. I was paid in the low 100’s.
They would rather contract out the high-paid work and pay orders of magnitude more for it.
This is very true and I loved working with bioinformaticians but the pay is so much lower than a normal SWE role which is why SWEs will pick tech over genomics companies.
Look, I did this at multiple places for a number of years. The issue is that you often form an adversarial relationship with the scientists. They don't really want you there. They are perfectly happy just organizing everything by hand with post its and excel spreadsheets. They do not want you to mess with their flow with your software, even if it would help them to be more efficient.
Can you elaborate with some anecdotes? Why is their current workflow wrong? Why would an organization hire someone to build software if they can achieve goals with spreadsheets?
I work in this field at a large medical research institution. There is a significant amount of genomics analysis that occurs here on a day-to-day basis. The genomic processing pipeline work all falls directly into my group.
There is next to zero demand for tool development internally. I do it on the side of "normal" IT data management because I love high performance computing, algorithms, and multithreaded hackery. But even at my large, well-funded institution, there isn't a specific role where that is all that you do by design.
I do suck at marketing - meaning, despite having some success with big improvements in research tools that folks have definitely appreciated, no one comes to me asking for help with better engineering of genomic applications. Partly that is due to many researchers maybe only know R, so they will default to whatever packages are already available in Bioconductor, install those, and throw the resulting mash-up for their current research effort onto the compute cluster and simply wait for hours or days for the jobs to finish.
PIs are often insulated from software engineering problems too - if work is completed before the next bi-weekly meeting and update session, well, it must be ok.
Ah, sounds so much like history of programming. At the 50's stage of straight up statistical manipulation.
DNA base units not viewed as base 4 binary number system that can be transformed into an abstract software language, where can select abstraction level of choice to use. Much like musical notation not viewed as numeric system.
Although, most software engineers don't view systems as numerical language development, too.
Great post - which contains the answer to many of the questions raised in this thread. I am working in this area as well. There is "next to zero demand for tool development" because there are great open source tools. Only in rare cases (e.g. Illumina Dragen), a commercial software adds significant value that the audience is willing to pay for.
It may be worth pointing out that several of the leaders in the Genomics field started off in commercial software development. I agree that it does not make monetary career sense to move into genomics -- academic labs cannot pay you more than the lab head makes, which is probably much less than many software developers are worth in other markets.
But I've known several financially successful developers who have gone back for a PhD in bioinformatics and genomics, and, after getting over their distaste for existing tools, have made important and well-recognized contributions. But they did not make more money.
It might seem like an exaggeration but this morning I was thinking of doing more work in scientific software. I lost my mother to cancer. This seems like a way to channel that energy and motivation. Thanks for posting, OP.
PS. Feel free to reach out. Email in profile. I’ll be happy to email around the subject.
Can you provide a list of the top problems in that space? Much rather try to understand them deeply myself and build a company solving them than just getting a job.
Creating pipelines is still a problem. Typically one needs to call a bunch of other tools in order to get to the final result. There could be map/reduce behavior in the middle where chunks of data are processed in parallel in order to gain speed. And you need some kind of data management/tracking as well (putting samples in groups, ingesting raw data, exporting results). And sane monitoring especially if something breaks/fails.
There are probably 100s of tools written for this but no clear winner so far. The traditional software engineering approaches like git, ci/cd seem too heavyweight (or rather too low-level) especially during development. IMHO there could be space for a fully remote/cloud solution where one would code/debug/deploy from the browser optimized for writing/maintaining pipelines.
I also found the quality & proliferation of data pipeline tools to be baffling. Somehow always more painful to put these together than it seemed like it ought to be.
At one point we wrote an internal tool (I think lots of organizations do this, since all the 100s of existing tools somehow don't fit, so you invent #101) and while it was tremendously satisfying getting batch jobs with 1000's of cpu's churning away, that kind of data infrastructure needs to be standardized. I think some companies are doing this, e.g. saw a presentation about Arvados/Curii that seemed interesting (but haven't used it so not sure). Maybe CWL will turn out to be the way forward here?
Protein structure prediction was a huge deal, which is why AlphaFold received so much fanfare. It is actually pretty good. The next step is to predict where multi-protein complexes would interact- which is not just as simple as predicting the structure of two proteins independently and then trying to fit them together like a puzzle, because the the interactions can also change the structure.
While it's not as hard as it used to be to experimentally determine protein targets of, for example, a protein kinase, it's still not an arbitrary or cheap experiment, and to do that for the many thousands of such proteins, across different conditions (stress, presence of co-factors, etc) and in different organisms would be rather a lot of work. Something like alphafold that makes reasonable predictions and can be used to help you focus on what's most likely to be relevant to your disease or process of interest helps quite a bit.
There's also more need for integrating "multi-omics" data, where you have data from multiple assays (gene expression, phospho-proteomics, lipidomics, epigenetics, small RNA expression, etc etc) with the goal of somehow combining all these different assay results from various levels of gene regulation, to get closer to figuring out actual mechanism for complex processes. Building on that, we can also do single-cell multi-omics to some extent- where you have results from different sequencing-based assays on the level of the same individual cell. This is still pretty limited, but it's exciting and advancing pretty quickly. This will eventually be combined with things like spatial transcriptomics, which is useful for mapping out what's going on in heterogeneous tissue samples like tumors, for example, so we'll end up with spatial single-cell multi-omics, at which point you're looking at 1) some quantitative trait for multiple genes/loci/molecules, and often 10k+ of such features at the same time per assay, 2) multiple assays, such as DNA accessibility and gene expression, in 3) single-cells, of which you might have 10k of in a single sample, 4) across a physical tissue sample where individual cells are spatially mapped, and where you probably want to figure out how cells might influence the state of those around them, and 5) in multiple different samples, where you might want to compare disease vs control, or look for correlation to heterogeneity of results within one group.
There's a lot of public data already available for single-cell gene expression projects if you want to get a feel for how these things are structured and how (passable but not amazing) the existing tooling is- one of the main repositories for this data is the NCBI's SRA https://www.ncbi.nlm.nih.gov/sra but you'll quickly note that searching and browsing is not as easy as you might think it would be- because one of the main limiting factors in bioinformatics is how bad everyone is at keeping terminology consistent. For many bioinformaticians, a majority of time is spent in the data cleaning phase. It's awful. Sometimes the experimental parameters make it into SRA or GEO, but sometimes you have to read through the associated paper to pull that out. Often it's only large consortium projects like the The Cancer Genome Atlas (TCGA) or the Genotype-Tissue Expression project (GTEx) - which have enough funding for staff dedicated to data management- end up publishing datasets that are easy to "consume" without having to jump through a whole bunch of hurdles to figure out how the data was produced.
I have a BS/MS in bioinformatics and I'm presently a PhD candidate in genetics and computational biology defending in February.
So if I understood you correctly then further lowering the cost of experimentally determining protein targets could be a viable way forward that is completely orthogonal to computational methods?
20 years ago I got interested in "bioinformatics." I loved learning something about molecular biology, after all those years of hearing about DNA and not understanding it. And "Molecular Biology of the Cell" is, hands down, the greatest textbook ever written.
That said: a lot of the comments are spot on. You're working in a field where the hard scientists and business people rule and you're a helper. Maybe they're grateful for your help OR maybe they regard you as an overpaid lab assistant. After all, they have PhD's and postdocs, and you don't.
I've never actually worked in that field. I'd guess that it might be very satisfying, despite the low pay. Or not.
> That said: a lot of the comments are spot on. You're working in a field where the hard scientists and business people rule and you're a helper
This definitely was the culture when I started working in the field 6 years ago. However, the culture has shifted (at least where I work) to where biologists and engineers are equal partners that work together on solving these problems. For those organizations that are not this way, I think they’re going to have to change if they want to innovate.
Agreed. Huge change over the last 10-15 years. My first job in the space had a view that obviously a mere software developer wouldn't be paid more than even a postdoc scientist. And as postdocs weren't paid all that well, you see where this is going.
These days more biotech companies are computationally/software focused. They understand that to pull in strong talent they're not operating in the same academic science world.
That may be the case for engineers with PhDs and scientific credentials, but I'm not so sure that is true of normal developers who did not play the academic game. I'm not going to take a job based on the eventuality of a culture shift, and I don't think you should either.
This isn't just genomics, by the way. Scientific computing folks are very similar.
That's always been my impression, but it does sound like "software eats the world" has had some effect. At least in some places.
Looking at it from their point of view: CS people tend to think that "everything is just information, and now that we're here you're all going to be working for us."
You can see why a PhD in mol bio would resent that. Everything is not just information.
Fwiw, I studied humanities, do a lot of pop science reading in my spare time, and I'm able to appreciate it. There are more detailed and technical sections I skim or skip, but the overviews are fantastic. Incredible description of, for example, the sheer wonder we should all experience at the fact that all life starts as a single cell.
MBC is readable by someone with an undergraduate background in science. You'd probably want basic knowledge of biology, general chemistry and organic chemistry.
It's essentially an upper-level undergraduate textbook.
Probably just a college level gen chem class. Pretty accessible, albeit technical, textbook from what I remember of reading it for a course a few years ago.
The chemistry requirements are minimal. You should understand the difference between ionic and covalent bonds, how van der Waals forces work, hydrophobicity, solubility, and the effects of catalysts on reaction transition states. It will also be important to understand what reaction kinetics are and what pH means. An understanding of buffers might be useful.
I would argue that, to understand the book, you specifically don't need to know electrochemistry, organic chemistry, analytical chemistry, organometallics, spectroscopy, or even physical chemistry.
Having worked (as a consultant/contractor) for a few businesses in the field, I can say that my experience was closer to "grateful for your help" than to "an overpaid lab assistant". I even recall once, in a meeting, being referred to (by a senior staff scientist with a Ph.D.) as "the technical guy", causing me to wonder at how someone who does gene sequencing thinks of programming as being more technical.
I have worked in a genomics lab after finishing a bioinformatics master's.
It was my first fulltime job, and by far the most chill. People were great. The PI was laid back, the whole lab went out for beers every now and then - and not because of a mandatory startup-style 'bonding' event. We genuinely enjoyed each others company and hung out outside of work. I never had that in any other job, which were/are all commercial operations.
The vibe and the power structure felt very different. More level. There werent any purely managerial roles, everyone was doing at least a bit of 'science'. And even junior ICs like me got to coach undergrads every now and then. Most of the operational budget comes from grants, on which you have to deliver. The pay is not amazing, so most employees really are in it for the science.
Or I was still young and naive and was lucky all of the two layers of management were all nice people.
Ultimately I left, as the grant money coudnt keep up with offers I was getting.
It is still the job I am most proud of. I love talking about it, and it really sucks that even a well funded lab cant really afford market engineering rates.
I'm in a role that is very similar (different field though). However, I know enough about academia to know that alot depends on the culture the PI fosters. Also, I spend a lot of time learning the field.
I contribute to Nextflow core (https://nf-co.re/) It's more of a collection of pipelines than traditional software, but there are users all around the world and a good community.
Most of the packages on bioconda (https://bioconda.github.io/) are open source. But you probably want to find a sub-field that interests you most before finding a project.
In grad school, we also had an ex-google software engineer volunteer with us one day a week. It was very impactful for many members of the lab to learn good engineering practices, and it wasn't at all like the sentiment others in this thread are expressing where engineers were "janitors".
I worked with Bob some ~20 years ago at Berkeley. he showed up one day to check out the seminars and see if he could "help out" after having sold his database company to Intel. he said he'd been trained as a physics guy in the 80s but there were no real jobs so he started a software company instead. He joined my advisor's group (it helped a lot, because at the time most journals wouldn't publish a paper submitted from a home address).
He proceeded to completely understand hidden markov models and protein sequence alignment and was immediately hacking improvements to HMMER. However, Sean Eddy couldn't understand his optimizations (Sean has to know how HMMER works at all times) and so Bob went off and made his own tools like MUSCLE.
One of the reasons he can do this is, well, he's a programmer/math genius, and the other reason is that HMMs and protein alignments are a fairly well understood and programmable thing these days.
Still blows me away we train up all these people to be scientists when there are no jobs for them in that role.
I wish more fields would just start adopting the product/engineer partnership that Software companies have perfected. Engineers are very good at what they do. Product people are very good at what they do. They need each other to build things. Sure, engineers might know enough about product to get by and product people might know enough about coding to get by, but the reason it works is because each one is an expert in what they do and are equal.
Its no different in finance, healthcare, genomics etc. I'd love to work in a setting where I'm paired with an SME product manager in a domain I have no clue about and they respect my work and I respect theirs and we are partners.
This is one of the biggest factors that made software/internet companies explode. They respected people who build software. They didn't need to. A bunch of MBAs could have easily just decided that the best way to run the company was to treat the people building the product as a cost center. Many did. I think that's probably one of the reason for the lack of innovation and down fall in many old tech companies like HP/IBM.
The ones that treated SWEs properly and valued them accordingly, did very well.
I have heard from a friend who's a doctor that in hospitals there's a very adversarial relationship between doctors and MBAs. The MBAs see the doctors as a cost center, and the doctors resent people without MDs being above them.
Your comment reminds me to be thankful that at many software companies engineering, product, and design do respect each other as equal partners. I totally agree that to do otherwise is business suicide.
Molecular Biology of the Cell got me extremely excited about genetics and bioinformatics, highly, highly recommend this book to any software person I meet who is interested in biology.
As to the work environment, it seems to be extremely varied depending on the lab and team your on. I came from a number of years doing web development in marketing and finance before joining an R1 university research lab, and in many ways the day-to-day is quite similar in both fields. You are not the 'go-to' person for most things, but with that said, even as an individual contributor I feel my voice is heard on technical decisions where appropriate. As for pay, it's the biggest aspect that will make me leave at some point. If you do not have a PhD, or even a degree in my case, you can't expect to get paid a lot. As to the speculation on the satisfaction of the work, it is indeed deeply satisfying!
I got to have a conversation with one of the hero donors that gave a kidney biopsy after a life-saving transplant. It's hard to overstate just how impactful your work feels when talking to someone like that. Even as a small cog in the larger machine (our lab is around 50 strong with many people being at the top of their sub-fields), the end results of the effort will be massive improvements in individuals quality of life, this alone makes it quite easy to get out of bed in the morning.
Any particular edition of Molecular Biology of the Cell you’d recommend? I just looked up the 7th edition on Amazon (seems like the latest) and it’s $300 USD. Oof.
I've still got my 3rd edition copy (from 1999 when I was an undergrad molecular biologist). Most of the basic biochemistry and molecular biology will be exactly the same--it hasn't changed much if at all. While there have been lots of additional details added over the last two decades, the fundamentals are unchanged for the most part.
This wouldn't apply to other fields such as Immunology (Janeway's Immunobiology) where I have purchased multiple copies of the years due to the field changing so fast.
I worked in the field. Leaving to work on ads immediately tripled my salary, and gave me more room to grow.
Everyone who says you're the hired help and treated about as well as a secretary that the organization dislikes is dead on. At best, you're viewed as an overpaid cost center.
Which is sad, because I'd love to work in these areas... but I'm not giving up 66% to 75% of my income to as charity to private corporations.
- The compensation absolutely do not match the workload and education required.
- The sheer number of disreputable PIs and their unrealistic goals for software.
- The data is likely questionable and often underpowered.
- Institutional politics. everywhere.
- Marketing ("Curing Cancer"). The role is actually just juggling various bioinformatics file formats.
> just juggling various bioinformatics file formats
Your other points are spot on. This one I want to address specifically. The file formats. Academics love their incredibly over-engineered file formats. MARC. SGML. DICOM. HL7. RDF. Those are just the ones I know. Universally, they try to cover every corner case that anyone could ever imagine. Academics absolutely love their ontologies. Just implementing one of them is a nightmare. Going from one to another is an exercise in the philosophies of ontologies.
Actually I think genomics / bioinformatics is a counterpoint there. One of the things I like about the field is nearly every file format is under-engineered. It's TSV all the way down and if you need compression gzip it. If you need to index that, sort it (literally often with unix sort command) and block-gzip it. Anything more engineered arose specifically because the above failed and something more is actually needed.
The downside is it's a giant hellscape of unstructured, poorly specified formats where data types are barely specified at all or if they are most of the schema is published on some rambling blog post by some rando scientist. You will spend most of your time understanding it by empirical reverse engineering of the data that you are trying to deal with.
Oh, then eventually they'll get a committee together and after a few years they'll produce a unified file format that somehow manages to cover all the cases in the different existing formats (or at least the ones used by well-funded PIs) and is a hellscape of optional properties and required elements so poorly specified that it's impossible for any two implementations to communicate.
Nothing like putting that boilerplate pablum on research grant proposals. Either that or something about green energy. Some PIs just want to play with ligands, man.
If academics embraced software and software developers as heartily as advertisers did, you'd see that result. Until they do, I expect you'll continue to see a bunch of skepticism from developers.
Genomics is still predominantly a research field. In research, software development and hence software engineers are not valued much, because technologies change rapidly, new ideas come every day, so it is about being able to hack together a workable solution enough to write a paper or get funding.
Software development becomes important when certain data processing methods have been standardised, eg mapping sequencing data to mouse or human genome, differential expression analysis, pca visualisations.
Google already axed all job offers, Microsoft and AWS are searching student interns...
I used to work in genomics and computational biology. It was incredibly interesting. But it's university research and gets paid as such. 2-year time-limited contracts, lots of interns and students, extremely low salaries.
The AWS jobs aren’t even related to genomics. They just have genomics in the description of types of workloads performed by customers of AWS. The jobs are hard core CS automated reasoning jobs.
298 comments
[ 2.2 ms ] story [ 253 ms ] threadTrue, but working in academia is very VERY different working in a tech/product company.
Who wants to fix other peoples code mess? This is a no-no if you want to promote a job opening.
The original authors' quirks get enshrined in the code base, and its neigh impossible to fix until they leave the company that commercialized it.
That's their whole life: "where did you do your PhD? Who did you do your postdoc under?"
Many world-class hackers would do pretty poorly on those questions.
https://news.ycombinator.com/item?id=31577376
https://www.nature.com/articles/d41586-022-01516-2
> "Fundamentally, RSEs build software to support scientific research. They generally don’t have research questions of their own — they develop the computer tools to help other people to do cool things."
If I can get a better salary and working conditions at some crappy no-name startup, why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade?
> why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade
This is changing in my experience, albeit slowly. And really, this is what I'm calling on us, as a community, to do better on.
The reason you _would_ work at these organizations is because (1) the subject-matter is really interesting, (2) there are hard problems to be solved, and (3) you wake up every morning knowing that you are working on something that will have an impact on the lives of people around the world.
At least those are my reasons :)
Maybe genetic companies should catch up in workplace etiquette instead of recommending that SWE’s lower their expectations.
Whether or not that's a tradeoff you're willing to make is another question.
People can work for less if they are visibly valued, or where they are doing some heroic stuff that appeals to them personally.
People can for some time withstand being treated as second class, being overworked, etc, if they are paid a lot.
But if it's neither, why would anyone bother?
Because they find it rewarding in some other way? I agree that that way is not the conventional wisdom, but it exists, it turns out that some people value doing things that provide a demonstrable benefit to humanity.
Also he second class citizen thing has diminished over the years. It still exists but there are plenty of companies where that's no longer true. This is in stark contrast to when the field was getting off the ground, for instance it wasn't uncommon for benefits like PTO to be tied to your degree level & not length of employment.
My experience is that most non-tech organizations can't or won't.
The pay isn't all that important to me, as long as I can live on it, but this. It was so obvious when I was working in academia that because I wasn't myself an academic that I was just lab help, no better than the person who washes the test tubes and beakers.
Academia is a hugely elitist pyramid with well-demarcated layers, and the lab technicians are at the very lowest level of the pecking order, down with the cleaning staff.
They might be "needed", but by and large are they really "well respected" or "valued"? Not really, sad to say.
Will I be allowed to fix the parts that result in
> an organization that respects my craft so little
Like, it's one thing if they haven't - that can be fine, it's just more work. It's an entirely different thing if they won't.
1. Academic code. Not one institution would pass the Joel Test[1]. You pretty much covered some key points in your first paragraph, so I see not much has changed. The best predictor of how something will perform in the future is how it has performed in the past. Just hiring good software engineers won't change the system in which they work.
2. Academic bureaucracy & administration. I've worked for large Fortune 500 companies with less byzantine org charts. I've been matrix-managed. The siloing in academia is crazy.
3. Advancement. Because it's academia, advanced degrees are everything. My first boss in academia had a PhD. His job? He ran the student computing lab. My second boss was an MD/PhD. Great guy, but treated everyone like a lab assistant. I went to graduate school for one year and realized it wasn't for me.
4. (added after reading other comments). Completely unrealistic understanding of what developing robust, complex software is like. You touched on this by mentioning how many projects have 1 maintainer. I remember seeing a doctor shopping around his project plan. I'd say it would be a challenge for a high-performing 5-person team. He thought it was a job for a single entry-level programmer.
1 https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
When I last worked with academia, they essentially thought of me as the same as the guy who maintains the lab equipment, not an actual collaborator on their research.
It's ultimately down to the cultural norms of the field, as well as the realities of academic funding.
I was a research software engineer (RSE) for the best part of a decade. The best thing that happened to me was being made redundant when my funding ran out, and being forced to work in industry. What a difference (and wholly for the better).
The reasons you give are all nice positives, but they all ultimately are very emotionally manipulative. You're asking people to act against their own self-interest. But this isn't really for the benefit of humanity. It's for the benefit of the PIs who run the research groups, and keeping their little empires running. But the cost to the individual is great. You're sacrificing salary, a career track, advancing your own skills to the full, and in many cases the opportunity to have a life: being able to afford a home and support a family.
In retrospect I, and many others like me, do feel that we were taken advantage of to some degree.
I spent several years on a massive grant, then several years on lots of small short-term grants (12 months, 6 months, 3 months). You can't risk getting a mortgage when you have no guaranteed employment. And it's also very stressful not knowing if you'll be employed in three months time every three months. And unlike in a company, RSEs don't really have a proper career track. There's no real progression. You're a hired help.
RSEs are not treated equally with academics. Let's be really honest about that. We're not. I even had a PhD in the subject area and you're still beneath all of the "real" academics. We're not "partners" in their work. We're the dogsbody's.
If these people want software developers with real chops to work in the field then they need to pay a competitive salary, have a proper career track, and really fix the job stability. And they also need to properly respect the expertise RSEs bring. Unless all of those are fixed, a career in industry will continue to be the only rational choice.
This won't happen though. Tenured academics refuse to consider paying the going rate because that would mean the "hired help" would be earning considerably more than they do. I had already topped out the salary band when I left, and I was earning more than most of the junior-mid-career academics. They are, of course, on fairly poor salaries. They too would earn vastly more in industry, but are mostly unwilling to consider that option as a rule. Their loss. If they truly respected the value they were getting, then they would pay for it. It will not happen though. Most of academia is about climbing the greasy pole and not about advancing the state of the art; there's just no way they'll permit others to sit higher on the salary pyramid than they do.
At least in industry skill and competence and the ability to deliver are highly-valued, and companies will gladly pay for people who are proven to deliver. In practice the work I do in industry (biomedical) has far more positive impact upon the world than anything I did in the academic niche I used to occupy, and is also vastly more enjoyable, with a lot more responsibility and technical depth.
Are you personally planning to stick it out for the whole of your career? Because if I could give you the advice I would have liked to have given myself, it's that you should properly think about where you want your career to go in the medium to long term, and decide when (not if, but when) you will exit to move to industry. Use it as an opportunity to gain some useful skills, and move on to where your skills will be properly valued.
The position kept being posted multiple times over a couple of years. Then I moved on and don't know what happened.
High potential areas like genomics that are behind on software are amazing places for talented software people with a givig attitude to have a big positive impact.
If only this was actually true.
In reality, you're being taken advantage of and it's very manipulative and deceptive to make this claim.
You're working to give some pharma company something to patent and make millions of $
Amen, couldn't have said it better myself. I'm sure it's very worthy and all working on a genomics project that aims to eradicate some killer disease, but you need to live and provide for your family while you're doing it.
Despite my interest, I’ve found that landing a job in this field at my desired compensation level is very difficult especially if you not have the ”correct” academic background. Who does a double degree for computer science and forensic genealogy? I’m sure some people but for $75k/yr you’d think the companies need to at least adjust their expectations.
I looked. There are zero full-time, remote roles that don't require previous genomics experience at any of the companies listed.
Tooling roles in SWE in every other field are highly regarded. Why not here?
it depends on the role. it worked out really well for me when I got to drop in and do piece work on lots of projects in different fields. working on a larger software development project can be really painful and demoralizing because the people running it don't really understand how the sausage gets made.
They have a hard time having someone with a BS or MS making 50-75k more than a freshly-minted PhD.
I just left a job in pharma because I cannot do it anymore (salary being a big one, but my experiences reflect many in this thread).
They spent 500k on a consulting company to build a few NGS processing pipelines. This was built using a framework I was unfamiliar with. I re-factored one of them and was able to increase runtime by 60% in a couple weeks. I was paid in the low 100’s.
They would rather contract out the high-paid work and pay orders of magnitude more for it.
But this shit is so dumb.
Edit: fixed.
Can you really trust the scientific results if they depend on software made by people who don't care about code quality?
Just at your shop, or the field in general?
[1] https://www.theverge.com/2020/8/6/21355674/human-genes-renam...
There is next to zero demand for tool development internally. I do it on the side of "normal" IT data management because I love high performance computing, algorithms, and multithreaded hackery. But even at my large, well-funded institution, there isn't a specific role where that is all that you do by design.
I do suck at marketing - meaning, despite having some success with big improvements in research tools that folks have definitely appreciated, no one comes to me asking for help with better engineering of genomic applications. Partly that is due to many researchers maybe only know R, so they will default to whatever packages are already available in Bioconductor, install those, and throw the resulting mash-up for their current research effort onto the compute cluster and simply wait for hours or days for the jobs to finish.
PIs are often insulated from software engineering problems too - if work is completed before the next bi-weekly meeting and update session, well, it must be ok.
DNA base units not viewed as base 4 binary number system that can be transformed into an abstract software language, where can select abstraction level of choice to use. Much like musical notation not viewed as numeric system.
Although, most software engineers don't view systems as numerical language development, too.
But I've known several financially successful developers who have gone back for a PhD in bioinformatics and genomics, and, after getting over their distaste for existing tools, have made important and well-recognized contributions. But they did not make more money.
PS. Feel free to reach out. Email in profile. I’ll be happy to email around the subject.
There are probably 100s of tools written for this but no clear winner so far. The traditional software engineering approaches like git, ci/cd seem too heavyweight (or rather too low-level) especially during development. IMHO there could be space for a fully remote/cloud solution where one would code/debug/deploy from the browser optimized for writing/maintaining pipelines.
At one point we wrote an internal tool (I think lots of organizations do this, since all the 100s of existing tools somehow don't fit, so you invent #101) and while it was tremendously satisfying getting batch jobs with 1000's of cpu's churning away, that kind of data infrastructure needs to be standardized. I think some companies are doing this, e.g. saw a presentation about Arvados/Curii that seemed interesting (but haven't used it so not sure). Maybe CWL will turn out to be the way forward here?
There's also more need for integrating "multi-omics" data, where you have data from multiple assays (gene expression, phospho-proteomics, lipidomics, epigenetics, small RNA expression, etc etc) with the goal of somehow combining all these different assay results from various levels of gene regulation, to get closer to figuring out actual mechanism for complex processes. Building on that, we can also do single-cell multi-omics to some extent- where you have results from different sequencing-based assays on the level of the same individual cell. This is still pretty limited, but it's exciting and advancing pretty quickly. This will eventually be combined with things like spatial transcriptomics, which is useful for mapping out what's going on in heterogeneous tissue samples like tumors, for example, so we'll end up with spatial single-cell multi-omics, at which point you're looking at 1) some quantitative trait for multiple genes/loci/molecules, and often 10k+ of such features at the same time per assay, 2) multiple assays, such as DNA accessibility and gene expression, in 3) single-cells, of which you might have 10k of in a single sample, 4) across a physical tissue sample where individual cells are spatially mapped, and where you probably want to figure out how cells might influence the state of those around them, and 5) in multiple different samples, where you might want to compare disease vs control, or look for correlation to heterogeneity of results within one group.
There's a lot of public data already available for single-cell gene expression projects if you want to get a feel for how these things are structured and how (passable but not amazing) the existing tooling is- one of the main repositories for this data is the NCBI's SRA https://www.ncbi.nlm.nih.gov/sra but you'll quickly note that searching and browsing is not as easy as you might think it would be- because one of the main limiting factors in bioinformatics is how bad everyone is at keeping terminology consistent. For many bioinformaticians, a majority of time is spent in the data cleaning phase. It's awful. Sometimes the experimental parameters make it into SRA or GEO, but sometimes you have to read through the associated paper to pull that out. Often it's only large consortium projects like the The Cancer Genome Atlas (TCGA) or the Genotype-Tissue Expression project (GTEx) - which have enough funding for staff dedicated to data management- end up publishing datasets that are easy to "consume" without having to jump through a whole bunch of hurdles to figure out how the data was produced.
I have a BS/MS in bioinformatics and I'm presently a PhD candidate in genetics and computational biology defending in February.
That said: a lot of the comments are spot on. You're working in a field where the hard scientists and business people rule and you're a helper. Maybe they're grateful for your help OR maybe they regard you as an overpaid lab assistant. After all, they have PhD's and postdocs, and you don't.
I've never actually worked in that field. I'd guess that it might be very satisfying, despite the low pay. Or not.
This definitely was the culture when I started working in the field 6 years ago. However, the culture has shifted (at least where I work) to where biologists and engineers are equal partners that work together on solving these problems. For those organizations that are not this way, I think they’re going to have to change if they want to innovate.
These days more biotech companies are computationally/software focused. They understand that to pull in strong talent they're not operating in the same academic science world.
This isn't just genomics, by the way. Scientific computing folks are very similar.
Looking at it from their point of view: CS people tend to think that "everything is just information, and now that we're here you're all going to be working for us."
You can see why a PhD in mol bio would resent that. Everything is not just information.
Tangential, but what are the chemistry prereqs to grasp this book?
It's essentially an upper-level undergraduate textbook.
I would argue that, to understand the book, you specifically don't need to know electrochemistry, organic chemistry, analytical chemistry, organometallics, spectroscopy, or even physical chemistry.
But, YMMV.
Everything you don't understand looks complicated from the outside.
It was my first fulltime job, and by far the most chill. People were great. The PI was laid back, the whole lab went out for beers every now and then - and not because of a mandatory startup-style 'bonding' event. We genuinely enjoyed each others company and hung out outside of work. I never had that in any other job, which were/are all commercial operations.
The vibe and the power structure felt very different. More level. There werent any purely managerial roles, everyone was doing at least a bit of 'science'. And even junior ICs like me got to coach undergrads every now and then. Most of the operational budget comes from grants, on which you have to deliver. The pay is not amazing, so most employees really are in it for the science.
Or I was still young and naive and was lucky all of the two layers of management were all nice people.
Ultimately I left, as the grant money coudnt keep up with offers I was getting.
It is still the job I am most proud of. I love talking about it, and it really sucks that even a well funded lab cant really afford market engineering rates.
Doing fundamental reseach is a taller order. But lots of software, tools, pipelines etc need maintainers, optimizations...
Most of the packages on bioconda (https://bioconda.github.io/) are open source. But you probably want to find a sub-field that interests you most before finding a project.
In grad school, we also had an ex-google software engineer volunteer with us one day a week. It was very impactful for many members of the lab to learn good engineering practices, and it wasn't at all like the sentiment others in this thread are expressing where engineers were "janitors".
[1] http://drive5.com
He proceeded to completely understand hidden markov models and protein sequence alignment and was immediately hacking improvements to HMMER. However, Sean Eddy couldn't understand his optimizations (Sean has to know how HMMER works at all times) and so Bob went off and made his own tools like MUSCLE.
One of the reasons he can do this is, well, he's a programmer/math genius, and the other reason is that HMMs and protein alignments are a fairly well understood and programmable thing these days.
Still blows me away we train up all these people to be scientists when there are no jobs for them in that role.
Its no different in finance, healthcare, genomics etc. I'd love to work in a setting where I'm paired with an SME product manager in a domain I have no clue about and they respect my work and I respect theirs and we are partners.
This is one of the biggest factors that made software/internet companies explode. They respected people who build software. They didn't need to. A bunch of MBAs could have easily just decided that the best way to run the company was to treat the people building the product as a cost center. Many did. I think that's probably one of the reason for the lack of innovation and down fall in many old tech companies like HP/IBM.
The ones that treated SWEs properly and valued them accordingly, did very well.
Your comment reminds me to be thankful that at many software companies engineering, product, and design do respect each other as equal partners. I totally agree that to do otherwise is business suicide.
MD's -> patient interest comes first
MBA's -> company interest comes first
As to the work environment, it seems to be extremely varied depending on the lab and team your on. I came from a number of years doing web development in marketing and finance before joining an R1 university research lab, and in many ways the day-to-day is quite similar in both fields. You are not the 'go-to' person for most things, but with that said, even as an individual contributor I feel my voice is heard on technical decisions where appropriate. As for pay, it's the biggest aspect that will make me leave at some point. If you do not have a PhD, or even a degree in my case, you can't expect to get paid a lot. As to the speculation on the satisfaction of the work, it is indeed deeply satisfying!
I got to have a conversation with one of the hero donors that gave a kidney biopsy after a life-saving transplant. It's hard to overstate just how impactful your work feels when talking to someone like that. Even as a small cog in the larger machine (our lab is around 50 strong with many people being at the top of their sub-fields), the end results of the effort will be massive improvements in individuals quality of life, this alone makes it quite easy to get out of bed in the morning.
An awful lot has changed since 2000. RNA is now a Thing, where it was just a poor stepchild before. Protein folding, of course.
But yeah. The pictures are shining examples of what a scientific diagram can be.
This wouldn't apply to other fields such as Immunology (Janeway's Immunobiology) where I have purchased multiple copies of the years due to the field changing so fast.
Everyone who says you're the hired help and treated about as well as a secretary that the organization dislikes is dead on. At best, you're viewed as an overpaid cost center.
Which is sad, because I'd love to work in these areas... but I'm not giving up 66% to 75% of my income to as charity to private corporations.
- The compensation absolutely do not match the workload and education required. - The sheer number of disreputable PIs and their unrealistic goals for software. - The data is likely questionable and often underpowered. - Institutional politics. everywhere. - Marketing ("Curing Cancer"). The role is actually just juggling various bioinformatics file formats.
I need an advanced degree for that?
Your other points are spot on. This one I want to address specifically. The file formats. Academics love their incredibly over-engineered file formats. MARC. SGML. DICOM. HL7. RDF. Those are just the ones I know. Universally, they try to cover every corner case that anyone could ever imagine. Academics absolutely love their ontologies. Just implementing one of them is a nightmare. Going from one to another is an exercise in the philosophies of ontologies.
DICOM is for radiology.
RDF and SGML, well, they're from the same era as XML, so yeah.
Good thing there are lots of competing implementations! It would be a shame if these files were actually portable.
The downside is it's a giant hellscape of unstructured, poorly specified formats where data types are barely specified at all or if they are most of the schema is published on some rambling blog post by some rando scientist. You will spend most of your time understanding it by empirical reverse engineering of the data that you are trying to deal with.
The person that runs a research lab, which at a university is usually a (tenured) Professor.
Nothing like putting that boilerplate pablum on research grant proposals. Either that or something about green energy. Some PIs just want to play with ligands, man.
Google and Microsoft probably know how to make software?
Side note: why does this page have user-select: none on body? It's annoying; what does it accomplish?
What happens when the world's most brillant minds do something else than making us click on more ads ?
Software development becomes important when certain data processing methods have been standardised, eg mapping sequencing data to mouse or human genome, differential expression analysis, pca visualisations.
I used to work in genomics and computational biology. It was incredibly interesting. But it's university research and gets paid as such. 2-year time-limited contracts, lots of interns and students, extremely low salaries.