Ask HN: Physicists of HN, what are you working on these days?

456 points by sachin18590 ↗ HN
Of late, except for few headline-friendly fields (colliders, quantum computing, gravitational waves and astrophysics in general), I don't get to see/relate with a lot of activities in Physics. Also I have noticed a growing trend of physicists becoming data scientists post phD. Although I understand the money factor, are there any other reasons for this as well?

332 comments

[ 3.0 ms ] story [ 263 ms ] thread
Academic trends are ruled by money. In theoretical physics, computer simulations is where the money is. For the students, that means they are doing computer science related work, frequently the physics part is lacking. The route to leaving physics and using the data science capabilities is then a straightforward one.
I work with a few PhD physicists, one is a technical program manager in applied optics and ML; one is an individual contributor in optics and ML, and the other one is pretty straight stick ML. I bailed on physics after my bachelor's, ended up in medicine and now ... work in ML, with them. I don't think I ever heard anyone tell me, or any of my classmates "you should get a PhD in physics". It was something one did to do it. Because it's there. But even 20 years ago, the guidance was to look for jobs in finance or computers.
>20 years ago, the guidance was to look for jobs in finance or computers

It was 2003 when the company I was working for acquired InterBiz from Computer Associates. By the time we were integrating teams from both companies I met this guy who were the guru, the #1 product manager from one of the most profitable lines of business that the company had. I remember the shock in people's faces when he, a Portuguese guy, said he's got a PhD in Physics, in Germany, in a second language to his own, but then ended up developing Warehouse Management Systems (WMS).

> But even 20 years ago, the guidance was to look for jobs in finance or computers.

Yeah. I keep kicking myself for not taking the Wall Street guys who called me more seriously. That would be one important message I'd like to send backward in time.

I remember some of the Physics departments I knew well commenting that they were getting 250+ applications for each open tenure track position (early/mid 90s). It skyrocketed to 1000+ at one point for a few places.

Based on some back of the envelope math, and a realization that a fresh Ph.D. with ~6 publications (2 PRL, 4 others) would not be a good competitor to a senior FSU physicist with 50+ publications to their name, a solid reputation, and an interest in moving out of the FSU quickly.

I determined that market forces had flooded the supply side of physics with, well, extremely good candidates for the same positions I was applying for. And my ability to compete with them was low based upon the main components that hiring committees cared about.

So I looked elsewhere. 20+ years later, I've made a good career working in computing, but I really do miss physics and research in general.

A surprising number of physics students I studied with, were also mountain climbers. They seem to be very much, 'because its there' sort of people.
Is it really bad that most time is invested in simulations? Seems like a very potent low-hanging fruit (not to mention how many cool things could come out of it).
Depends on what you enjoy doing. I spent a lot of time doing magnetic simulations during the earlier period of my PhD and the reason for doing them isn't that the theory can't explain what will happen, but rather that we don't have the mathematical chops to actually calculate it, so we build little artificial worlds and try to take measurements the old fashion way.

This was certainly a lot fun, but you won't be getting any thing particularly revolutionary out of it. I remember my advisor at the time running a few dozen CPUs of our poor excuse for a cluster (built mostly by yours truly with off the shelf components) for the entire summer to add a couple of digits to the ground level energy estimate of a completely artificial magnetic model :)

On the bright side, it taught me a lot about programming, numerical simulation and optimization

This kind of reminds me of what Jeff Hawkins once claimed was the issue with making sense of how the brain works:

> So why don't we have a good theory of brains? People have been working on it for 100 years. Let's first take a look at what normal science looks like. This is normal science. Normal science is a nice balance between theory and experimentalists. The theorist guy says, "I think this is what's going on," the experimentalist says, "You're wrong." It goes back and forth, this works in physics, this in geology.

> But if this is normal science, what does neuroscience look like? This is what neuroscience looks like. We have this mountain of data, which is anatomy, physiology and behavior. You can't imagine how much detail we know about brains. There were 28,000 people who went to the neuroscience conference this year, and every one of them is doing research in brains. A lot of data, but no theory. There's a little wimpy box on top there.

Not that physics has no theories, but I dropped out of studying physics myself over a decade ago, and at that time it felt a lot like the balance in physics has shifted towards having to measure and process disproportionate amounts of data with so much precision that it has to be automated, or like you said do a ton of really complicated modelling. It feels a bit "stuck" that way.

[0] https://www.ted.com/talks/jeff_hawkins_on_how_brain_science_...

I think you're seeing this in physics because, in terms of experimental accessibility, the low-hanging fruit are mostly picked.

Most parts of the Standard Model are verified to 5σ at all energy levels that are accessible to us in everyday life and in contemporary particle accelerators, so it's increasingly unlikely that we've missed any obvious mechanisms.

If we're going to find some completely new physics (in the same way that quantum theory and relativity theory were completely new at the turn of the 20th century), it's probably going to be at energy levels that contemporary particle accelerators cannot yet explore.

EDIT: That's not to say that we know everything that is to know about the accessible energy levels. There's so much in our theory that's yet unexplained, e.g. for all I know, we still don't have any idea why time works the way it does.

> Not that physics has no theories, but I dropped out of studying physics myself over a decade ago, and at that time it felt a lot like the balance in physics has shifted towards having to measure and process disproportionate amounts of data with so much precision that has to be automated, or like you said do a ton of really complicated modelling. It feels a bit "stuck" that way.

The reason for is just how damn good all the Physics Theory is. People keep having to look closer and closer to try to find a place, any place, where the theory "fails" (in a sufficiently spectacular fashion) hoping that that might point the way to new phenomena and new theory. There's a giant multibillion dollar hole on the ground in Switzerland who's main purpose was to find a chink in the armor of HEP or help point the way to new theories. It failed spectacularly, but they're already talking about building a bigger one :)

> The reason for is just how damn good all the Physics Theory is.

> (majewski's comment below) I think you're seeing this in physics because, in terms of experimental accessibility, the low-hanging fruit are mostly picked.

I mean, yes and no. The Standard Model is amazing. But I think we're also kind of focused too much on the Big Questions, and in the arena of the not-so-big questions there are still a ton of gaps and missing insights.

Perhaps in the not-so-big questions, most of the low-hanging fruit may appear to be picked as well, but I suspect that is more a kind of myopia that can be solved by collaborating outside of our field.

Whenever I see an article these days about physics that really excites me, it tends to be a story where one or more physicists got excited about a seemingly small question, or ate a bit of humble pie and joined forces with chemists, biologists or some other field, and or both, and that working it out turned out to bring far-reaching consequences and novel insights. Same with maths, really.

I'm aware this is completely personal of course, and that there is a likely bias in that this kind of research tends to be more accessible and thus more fun to read for me, but still.

> It failed spectacularly, but they're already talking about building a bigger one

Awww it isn't that bad. They found the Higgs and at least verified we need to look at higher energy levels to find anything new.

Nothing, as they don't seem to be hiring physicists.
(comment deleted)
You posted this in a thread full of Physicists in Industry, maybe you should reevaluate what you are doing.
When I still pursued my PhD in physics, I worked on all-optical amplifiers with phase noise reduction for fiber communication.

Now I develop software for a medium-sized ISP and IT outsourcing company.

I'm not a physicist but studied physics before jumping to programming.

A surprising amount of people whom I studied with ended up as programmers themselves, after finishing the physics degree.

My first degree was in physics as well, and I also ended up in IT. At SAP, to be exact. Urban legend among German SAP employees has it that SAP expands to "Sammelplatz arbeitsloser Physiker" (gathering place for unemployed physicists).
I know a few people at SAP (NL/DE) who are overqualified for the job they are doing and seem bored/uninterested. What's an insiders perspective of this?
I guess it depends on where you are in the company. I have been under two different managers since I joined 7 years ago, and they both keep their employees on a relatively loose leash. As long as the required work gets done in an orderly fashion, we get to choose our own pacing and attack angles. Makes for a pleasant work experience most of the time.

Then again, I guess I value stability and predictability in my day job a bit more than most Silicon Valley startup employees.

(comment deleted)
Are you happy with the path you took? Are you still happy with the time you spent studying physics?
Absolutely, but I had been programming before entering uni and got most joy out of programming related courses.

But I enjoyed my time as a physics student. Physics is an interesting field to study even if you don't end up working in it, in my opinion. :)

When I got my PhD (1999) the American Physical Society said that you had a 2% chance of getting a permanent job in the field with a PhD. At that point you are not being judged on your merits but on your connections, ability to navigate politics, etc. (The job is way too valuable compared to the value you can give to it.)

I saw a postdoc who is now rather well known struggling with anxiety over his career even though he had written half a book and done a lot of great work. When we were both at Cornell I'd come to the conclusion that many papers involving "power law" distributions were bogus because nobody knew how to test for them with any rigor. It was years later, after he had tenure, that he published something about it in a statistics journal.

Seeing that made me run for the exit after my first year as a postdoc.

> I'd come to the conclusion that many papers involving "power law" distributions were bogus

> after he had tenure, that he published something about it in a statistics journal

If I'm reading you right, you're saying he struggled for awhile doing bogus things only to question those things outside the relevant field?

(Also, can you say more about power law papers..?)

I read the comment as implying that the colleague had published a paper on power law distributions in order to publish (pressure) while the poster knew that the colleague did not believe in the subject.
I read the comment as saying that the postdoc kept his reservations about power law distributions to himself until after he had tenure, so as to not upset the applecart I suppose... ?
I wrote one (experimental) bogus paper. He wrote quite a few papers, I'm not sure if any of them used the experimental non-technique that was common at the time.

So there is a theory based on the renormalization group that can explain many (but not all) "fractal" phenomena that are observed.

I was part of the false paradigm of "bin up a probability distribution", "plot it up on log-log paper", and "draw a straight line."

Sometimes when you do that you will get an answer that has something to do with reality but it is not a valid answer to the problems of: "is a power law distribution a good model of this phenomenon", "what is a good estimate of the exponent", "are these two power law charts drawn from the same distribution" not to mention how to handle the problems that turn up at the highly frequent and highly rare ends of the distribution.

The root cause is this attitude

http://thinkexist.com/quotation/if_your_experiment_needs_sta...

and an academic system where power is too concentrated, where people who write review articles do the most good but get the least career advancement, etc.

There was a similar issue with power laws in biology, mostly driven by a guy named Barabasi. I don't think any of his results have held up when someone did the math right, but he's gone so far as to try to politically pressure people to keep quiet about it because they were hurting his career.

I've been out for a few years, so I don't know what the current state of affairs is.

Surprised to hear this about Barabasi, I'm no physicist but enjoyed reading his books on network science as a layperson.

Is there any published criticism against his work? His papers get cited everywhere, I know citations are more metric of popularity than scientific quality but I find it puzzling that his work continues to get attention if the results don't hold up.

I am about to defend my thesis in physics and will probably start a data science job afterwards (have had a few second round interviews, waiting for offers/rejections). Note that my thesis work did not contain big data or machine learning, but I have completed a couple of outside projects. I am in the US and in a top tier university/program.

My perspective is that there is just not many academic positions available. I realized I could do a couple of post docs for a few years and hope that something opens up, or not delay the most probable outcome and start an industry career sooner rather than later.

(PhD, 1996 in general relativity)

Embedded graphics drivers for real-time systems.

I keep the physics part of my brain alive by developing physics based Unity assets (nbodyphysics.com) and supporting a package for GR on github (grtensor).

I still buy WAY too many physics books. Current aspiration is to work through "Modern Classical Physics" Thorne/Blandford.

I bought Modern Classical Physics last summer as a birthday gift to myself and I am also slowly going through it.
Oh wow! This looks like the physics book that I always wanted to read. Phd 2016 in condensed matter physics - now working in a mid-size (~600 people) software company as a data scientist.
Modern Classical Physics is a great book - I've recently started working through it (just about to move on to chapter 2). I'd be interested in chatting about it with others/cross-checking solutions. Anyone who's interested, drop me an email (address in my profile).

If you're thinking of getting it but want to check it out, there's a 2012 draft version that the authors have previously taught from here: http://www.pmaweb.caltech.edu/Courses/ph136/yr2012/. It's not the same as the book, of course, but from a skim it seems quite similar.

Thanks! This looks like what I will start with. Can't find your email in your profile, though.
Ex-PhD student in general relativity here, also slowly going thru that book. If anyone wants to start a reading (slack?) group together, I’d be interested. I have no one to talk to about this stuff IRL.
I would love to join a reading group (on slack or another messaging platform) on this book.
And is this book a good read for someone re-learning math having only taken high school physics?
Depends on what the math re-learning covers. You'd need differential equations and 2nd year calculus.

A good choice for less math might be "Gravity from the ground up" by Schutz.

Have you ever considered working on some low-hanging fruit problem in a non-physics field?
Some of the Unity asset work might fit that description e.g. provide a library of chaotic attractors packaged for use in a game.

My grad work had no connection to data/stats and I have not bumped into any low hanging fruit where Riemannian geometry might be the answer!

I don't know that differential geometry ever made anything simple, so I don't know if there is any obvious fruit to be had. But there might be some, weird, unexpected fruit in the back; for instance, there exists a formulation of thermodynamics in terms of differential geometry: https://ui.adsabs.harvard.edu/#abs/arXiv:physics%2F0604164
As far as I know, a good differential-geometric understanding of nonequilibrium thermodynamics still hasn't been achieved.

The central issue is understanding how changes in control parameters (for instance concentrations of catalysts in a chemical system, or local fields in a spin system) affect the evolution of the probability distribution over states. Some work has been done in close to steady state (for instance [1,2,3]) but it's far from resolved.

This has some nice applications - designing efficient protocols for microscale devices, for instance.

[1] https://arxiv.org/abs/1603.07758 [2] https://arxiv.org/abs/1507.06269v1 [3] https://arxiv.org/abs/1201.4166

I know what you mean on keeping the physics part alive. I've written a bunch of code for doing simple molecular dynamics and varying the interaction types. I've also gone off and discovered I like number theory.

None of this pays the bills, right now I help build clouds, and I used to build supercomputers, and high performance storage systems.

Love your website, when I get home I am going to boot up my PC and play around with some of your code :)
I’d imagine there could be some really interestingly massive simulation possibilities integrating nbody with unity’s new ECS system
Mind reader!

Yeah, I'm keen to give that a go. Interestingly it is not a problem that is very easy to parallelize, since every mass will affect every other mass. There may still be some cool ways to use ECS though.

Yeah I can see that... at least all the other non-rigidbody stuff is (theoretically) much more efficient.
I was actually at Pitt when Rovelli and Newman were still team teaching GR and always regretted not taking this course. Would Modern Classical Physics be enough in the GR section to get the basics down? It looks decent for the stuff I know about (optics and so on).
I have a degree in physics and work as a software dev (c++). No one cares for physics in the industry, but it's indirectly taken as a sign that people aren't morons. I never used anything I learned at university at work. About half of my colleagues have physics or Math degrees, other half EE or CS.
I worked in numrel before and was even offered a tenure track job focused on teaching. I didn't want to work 70 hour weeks though. Now I focus on my family and data pipelines seem like a nice hobby.
I did my undergraduate in Aerospace Engineering and Astrophysics, then graduate work in Space Systems Engineering and regular Systems Engineering. I also did plasma physics research at Princeton.

Now I'm doing aerospace vehicle modeling and simulation in MATLAB and C++ and primarily work with other physicists. This is by far the most enjoyable work I've done and the pay is excellent - my salary is 3-4x the average household income for the city I live in.

(PhD, 2008 in Complex Systems)

While my PhD did start out as supposedly being on Spin Glasses it quickly diverged to Complex Networks and what people would now call Data Science. Since then, I've worked on Social Science, Epidemiology, Human Behavior, etc... For the last year or so, I've been doing Data Science and Finance in one of the Big Banks.

(5th year PhD student)

Working on my thesis analysis (collider based HEP), squeezing in time for OSS development and prepping for a jump to software dev.

I love physics, but academia is in a rough place right now. Almost everyone I know pursuing the academic career path has nightmare stories.

> nightmare stories Salary/job finding wise?
[ghost of tenure track searches past]

The physics job market has been, and will remain in a "rough place". As with all things, it is who you know.

If you want to stay in the field, network like mad. Get people to know your name. Make sure you have work known to them. Get a great Postdoc with someone who will make calls for you when you are done with your project, to help your search.

Otherwise, computing is nice :D

Money aside, there is a sort of more systematic reason we see physics MS/PhDs bailing to industry.

Though theory groups in general tend to use computational simulations as a tool to complete calculations, groups that develop novel computational methods and techniques tend to be headed by younger, more junior professors. These groups are typically well-funded and do very exciting (trendy? cutting edge?) work with distributed computation, machine learning, neural networks, etc, so they tend to pull quite a few students.

While these computational groups tend to bring in funding and are well-staffed by excited grad students, the junior professors leading them tend to be marginalized by the more traditional, seniority-focused establishment. Which is to say, a new PhD might have a lot of trouble landing a prestigious postdoc because a) their adviser might have been too young to have high name recognition outside their field and b) departments might place limits the amount of staff for these more junior professors/young groups doing exciting computational work. This is, of course, on top of the overall scarcity of jobs in academia.

But there's no such job scarcity in industry-- especially not for stats-smart programmers with years of experience a) wrangling data in python, b) writing fortran that runs on distributed clusters, or c) designing algorithms to solve /approximate hilariously expensive problems. Advisers know this and point some of their students who might thrive more in industry than academia towards that route.

(Anecdote: And of course, as a physicist who builds models/simulations in industry, I can speak a personally a little re: thriving. If you're someone in love with solving disparate problems, you're unlikely to find that in academia. Some of us learn in graduate school that we can't spend our whole lives-- or in my case, more than a few months-- solving one problem. Academia just... didn't seem like something that would be worth fighting for.)

I assume that this will gradually change as there's turnover within physics departments and we get more computational-first professors with seniority (or even in leadership). There are a few departments with better-known professors you can see it happening now. Universities are spinning up incubators and institutes for computational research. Physics departments are just slower to adapt to new developments, and the hierarchy of theorists can have more to do with seniority and internal politics than it does with technology.

And to answer the question, (MS 2016 Condensed Matter Theory), Did risk assessment for two years, now I simulate systems of systems.
I don't think you're necessarily wrong re: difficulty getting prestigious post docs after graduating from a young lab in a newer field, but I think you're way overselling it's importance.

The fundamental issue is the field is not growing (very much). Each professor will graduate 20-50 students over their career, but only one will get their job when they retire (on average).

From someone who might be pursuing a physics degree soon.

> Some of us learn in graduate school that we can't spend our whole lives solving one problem. Would you please expand on this. I am not sure if you meant that problems are hard enough or what.

The nature of a PhD is to study one sub-discipline long enough to reach the edge of human knowledge, and then expand it. Often this is so difficult that a set of multiple diverse projects is not practical. But not always; in my PhD I worked on both astrobiology and solar cells. It helps to learn versatile methods that apply to a range of problems.
PhD 2008, now a data "science" generalist
I am a physicist working as a physicist! My field of experimental particle physics, at least, is very hacker-ey: we develop instrumentation hardware, low-level data acquisition software, database and web apps for experiment operations and monitoring, and the like, beyond the data analysis. Of course, there are many industry opportunities for people with this kind of experience. In addition to the issues pointed out in other comments (small number of academic positions, salary differences), I think there are two cultural factors that are helping people transition out of academia: an increasing awareness on the part of advisors and institutions that students need more broadly marketable experience, and a corresponding decrease in the stigma of leaving academia. As one example, I routinely see notices distributed for Insight data science programs within our community.
(comment deleted)
> particle physics, at least, is very hacker-ey: we develop instrumentation hardware, low-level data acquisition software, database and web apps for experiment operations and monitoring, and the like, beyond the data analysis

Sounds like engineering and SE, not physics

Would you be interested in talking to me about a new miniature particle collider device I've come up with? I haven't yet built a prototype, but I need to ensure that I gather the correct data to determine if it functions the way I believe it will.
I got my masters, then pursued a PhD for 6 months after which an opening in computer graphics came up in the industry. I abandoned my PHd and started a career as software engineer.

The reason was I sucked in physics, did not find my PhD position motivating and loved computer graphics.

13 years later I'm a valued technical contributor in a team in an ISV creating a valuable global software packages in the CAD field.

Middle class income, could probably make lot more in US with my skillset but family situation really is not awesome for expatriation.

I'm doing OK.

With physics skillset you can pretty much make your career what you want it to be. You just need a proactive attitude to read on other fields. You need to understand the other guys mindset so you understand the overall game going on. It's usually not nefarious (although it can be), but most of the time the rules that are used are not the vocalized ones. In this empirical physics is a wonderfull philosophical background. Organizations have certain dynamic rules which half of the people are not aware of. You don't need to "play the game", but you need to understand the rules so that when the wind blows into the direction you want to go into you can grab the opportunity.

I had a pretty good idea where I wanted to be 13 years ago (R&D in a position that values quality over quantity and speed with a great team) and that's pretty much where I am now.

You need to know where you want to go, and go there. No one will guide your path. Physicists are an outlier but that math and mindset is really an asset. Just don't get stuck in fixing bugs in some legacy monstrosity, that is absolutely soul crushing. But such a position can function as a stepping stone if you are operating in the industry you want to work in.

Fixing bugs in a legacy monstrosity checking in! Updating my resume this week.

What's an ISV?

That's what I was saying - you need to understand the game. Part of this is knowing the core business terminology in you field :)

ISV - independent software vendor. An entity having an ownership of a software product, usually developing the software as well.

The other entity this is often compared against is the ESP - external service provider. I.e. consultants.

I spent years with legacy monstrosity and it does develop important skills. If the organization is otherwise ok it might not be that bad - but generally production software is absolutely horrible. The thing is legacy maintenance is important, and there's quite nothing like it that will teach you about the lifecycle requirements of software development. But I find it much more fullfilling to maintain and develop software that is alive and well and actively kept out of the 'legacy' label.

Not all old software is 'legacy'. One definition is that does it have tests. I've found an equally good definition of 'you are not afraid to modify it' and 'you can understand what changes in one place do elsewhere'.

So to be clear, I was not saying "anyhing but greenfield development sucks".

(PhD 2010 in Nuclear Physics) I stayed in Academia, right now I'm trying to survive the first year as an Assistant Professor. Teaching is a lot more fun than I thought it would be. My research is two fold: One half of my research is focused on precision measurements of the proton form factors (proton radius puzzle, form factor ratio puzzle), the other half on streaming readout for next-gen experiments.
Interesting research... I hadn't given much consideration to the interior structure of protons and neutrons before.
It is! If you look around you, essentially all the mass you see is generated dynamically by QCD inside the nucleons. And while we believe we can write down all the rules for QCD, we can not solve the equations well enough that we can reliably predict outcomes (emergent properties, like the masses, size etc), except if we do lattice QCD, which is essentially a brute-force numerical solution.
Physicist, working as a physicist (PhD 2015). Precision tests of gravity, primarily. We test the equivalence principle, Newton's inverse square law, hunt for exotic forms of dark matter, and build/invent cutting-edge sensors for LIGO.

I have begun looking about at jobs outside of academia in order to decide whether to stay or go. Data science is one of the easiest transitions a physicist with data-analysis can make, which I think explains the prevalence of physicists in that role. We have the training in both the techniques and a sober assessment of uncertainties, which makes us desirable.

So far, in my search for outside employment, I haven't found anything that draws me as much as my present work, but if you're in the Seattle area and looking for an experimental physicist with a broad range of experience, please get in touch.

> We test [...] Newton's inverse square law

Imagine taking Newton down in the Replication Crisis Wars!

Newton would not be taken down by this experiment because Newton's laws have a known precision bound. If we did not include the precision bound in our holistic idea of "Newton's laws," they would already have been falsified by relativity and quantum mechanics. Every physical theory has a precision bound that is determined by the most precise experiment that has ever tested it.
Well, I think that's a problem with calling this situation "The Replication Crisis". The basic laws of physics have been verified to an extraordinary degree of accuracy on the that people deal with them daily and even at the ordinary atomic level - if anything, physics has the opposite crisis where it's theories are not fully self-consistent on every hypothetical condition but where in every condition we can produce, one or another theories works with apparently perfect precision.

"The Replication Crisis" has basically involved rather different fields - psychology and human biology/medicine most often. The crisis could be called "human and animal testing crisis" imo.

https://en.wikipedia.org/wiki/Replication_crisis

Apologies for the delayed reply -- HN's noprocrast setting is a powerful taskmaster.

What we do is search for deviations from Newton's inverse square law. Depending upon the distance over which an experiment is performed, the ISL can be correct to a precision as high as ~10^-10 [1]. In particular, our group works to test whether the inverse-square law (or indeed gravity at all) happens as predicted at distances shorter than 100 microns [2]. It still amazes me that at distances shorter than half the diameter of a human hair, nobody has any idea whether gravity even happens.

Newton's early observations (and Kepler's) have been replicated many times -- there is no risk of a major replication-crisis upheaval there.

[1] https://arxiv.org/abs/hep-ph/0307284 (Figure 4)

[2] https://arxiv.org/abs/hep-ph/0611184

I did undergrad at UW, several of my classmates did grad work on LIGO and were on the gravity wave teams when they had their first detection. Amazing what timing can do for a career. I visited Hanford LIGO in 2012 before the detection, really interesting back then to hear their frustrations and steady commitment.

I graduated and did a few strtuos, now working on autonomous self-delivering ebikes in SODO.

How about the going the other way? I'm an engineer (hydrodynamics and optimization, and econometrics way back before all that.. and.. aerospace before that) who just finished a PhD and who has been writing software professionally for a large-ish engineering-software-company for 5 years while working on said degree. I have a serious-for-an-amateur physics side habit and really want to bring my work and play closer together. At work I look after a code base that is a half million lines of ancient f77/c++ so maybe that is a factor in my wanting something new!

Any room for scientific software developer at LIGO? Gravity is a beast... I will leave it at that. But I can signal process and I am not afraid to ask stupid questions. Looking for growth industries, I've had my eye on quantum computing and am just putting resumes together now. I'm really into differential geometry and topology but my understanding is amateurish and what I've learned of QFT, geometric physics, topology, etc., is uneven at best.

If you really are interested in software dev / data science I can ask around my company. They have a Seattle office* and a broad range of needs for different developer backgrounds --- at least when all offices are considered. Typically there is wiggle room to work from one office with a group based elsewhere. I can't guarantee anything though -- it's been hard to get anyone hired lately!

Back to gravity and such: I doubt you find anything, without physics in it, that draws you in as much as your present work, but it can't hurt to look around! How about a switch to focus on quantum computing if data science doesn't draw you? Bar that, what about numerical simulation?

*I'm not in Seattle.

It is much more challenging to go the other way -- there are a lot of people deeply in love with the field, with field-specific training, who would like to stay in (at significant financial and temporal sacrifice).

That said, if one wanted to move toward LIGO from a software background, the chink in the armor is that the signal-processing is hard and computationally bound. Helping signal-analysis groups to find efficiencies there would have an impact.

I continue to keep an eye on quantum-computing infrastructure hires in the Seattle area -- so far, I've proven to be too physics-focused to want to work solely on provisioning the (awesome) cryostats that house the quantum-computing devices under test. That may prove to be a sub-optimal life-decision -- time will tell :).

Fair enough. And yep, I plan to find a chink in the ol' armor somewhere. Just get me 20% time thinking physics at work and I will be happy... (even if it we never "get there" with the device) Or more likely my insatiable appetite will only increase..
I'll leave this here, since many comments are about the transition from physics to data science.

"For now, however, in hard-core physical science at least, there is little evidence of any major BD-driven breakthroughs, at least not in fields where insight and understanding rather than zerosales resistance is the prime target: physics and chemistry do not succumb readily to the seduction of BD/ML/AI. It is extremely rare for specialists in these domains to simply go out and collect vast quantities of data, bereft of any guiding theory as to why it should be done. There are some exceptions, perhaps the most intriguing of which is astronomy, where sky scanning telescopes scrape up vast quantities of data for which machine learning has proved to be a powerful way of both processing it and suggesting interpretations of recorded measurements. In subjects where the level of theoretical understanding is deep, it is deemed aberrant to ignore it all and resort to collecting data in a blind manner. Yet, this is precisely what is advocated in the less theoretically grounded disciplines of biology and medicine, let alone social sciences and economics. The oft-repeated mantra of the life sciences, as the pursuit of ‘hypothesis driven research’, has been cast aside in favour of large data collection activities [7]. And, if the best minds are employed in large corporations to work out how to persuade people to click on online advertisements instead of cracking hard-core science problems, not much can be expected to change in the years to come. An even more delicate story goes for social sciences and certainly for business, where the burgeoning growth of BD, more often than not fuelled by bombastic claims, is a compelling fact, with job offers towering over the job market to anastonishing extent. But, as we hope we have made clear in this essay, BD is by no means the panacea its extreme aficionados want to portray to us and, most importantly, to funding agencies. It is neither Archimedes’ fulcrum, nor the end of insight."

https://royalsocietypublishing.org/doi/full/10.1098/rsta.201...

> For now, however, in hard-core physical science at least, there is little evidence of any major BD-driven breakthrough

> Yet, this is precisely what is advocated in the less theoretically grounded disciplines of biology and medicine, let alone social sciences and economics. The oft-repeated mantra of the life sciences, as the pursuit of ‘hypothesis driven research’, has been cast aside in favour of large data collection activities

The thing is, I've just spent two years working for molecular neurobiologists in the field of Single Cell RNA Sequencing, and large data collection has definitely lead through tons of breakthroughs there.

We can now classify cell types based on gene activation, on top of the previously existing morphology and location the cells are found. That can then be used to discover new subtypes, the origins of cells during embryonic development, and even predict which cells will evolve into others[0][1][2][3]. All of this requires vast amounts of data to ensure there is enough statistical power. In fact, the insistence on using unbiased samples before applying clustering algorithms is a big part of overcoming biases based on pre-existing expectations.

(Also, may I request that you edit your comment and break up that block of text into sub-paragraphs, for the sake of readability?)

[0] http://mousebrain.org/

[1] https://linnarssonlab.org/osmFISH/

[2] http://gioelelamanno.com/post/velocitynature/

[3] https://www.nature.com/articles/d41586-018-05882-8

The lack of a guiding theory has been something I've talked about with regard to big data and ML in general. This said, if you look closely at the math behind ML, it looks, not to surprisingly, like statistical mechanics.

So if we take some of the thought processes behind Stat Mech (or S&M as we used to call it in grad school), and you kinda squint your eyes hard to blur the less robust discussions you read about, you get the sense that ML is more about the "thermodynamics of information" than anything else.

I find this intriguing and definitely want to spend more time on this stuff.

It also looks exactly like decision theory from statistics done in high dimensional spaces. Mostly because it is.
> And, if the best minds are employed in large corporations to work out how to persuade people to click on online advertisements instead of cracking hard-core science problems, not much can be expected to change in the years to come.

This is so sad

Nice quote. I'll have to read that paper later.

You can merge ML and theory in at least one way. I attended a talk by Prof. Karen Willcox of the University of Texas at Austin (I'm a PhD student in mechanical engineering there) where she argued that in fluid dynamics and combustion at least, it's better to use "model order reduction" instead of machine learning. The problem with many models (e.g., Navier-Stokes equations) in these fields is that they are computationally expensive. Model order reduction looks for ways to reduce the computational cost of the model while maintaining accuracy, and it uses many of the same techniques as machine learning. Based on the examples she gave it seemed to be the closest thing I've seen to merge the two.

https://en.wikipedia.org/wiki/Model_order_reduction

http://kiwi.ices.utexas.edu/

>And, if the best minds are employed in large corporations to work out how to persuade people to click on online advertisements instead of cracking hard-core science problems, not much can be expected to change in the years to come.

This makes me so incredibly depressed.

(HEP Phd, finishing)

I'll also be joining the data science world.

There's some debate about whether it's the best or worse time ever to be in particle physics. Either way I see a field that is overstaffed. Add to that the fact that CERN accelerators are shut down for two years and we're in the middle of the european strategy review.

Seems like a good moment to take some time out

My prediction is that the stagnation of physics will continue until the money and prestige has dropped to 1800s levels. Then, the only people who do it will be "true" in that they are motivated purely by curiosity. At that point the next set of breakthroughs will come around because all of the emotional and funding-treadmill induced blindspots that we have presently will be gone. Of course this is assuming that somehow the experiments can be kept up even through the funding decline.
Meaning, science as a pastime for the idle wealthy, with no pressure to publish or perish?
I'm not a physicist, but a physics drop-out. However, I am still in touch with many of the people I studied with. Only a few stayed in physics in the end.

One thing I noticed is that because physics has multiple decades more experience with dealing with big data compared to just about every other scientific field, a lot of physicists who jump ship tend to end up in a position where they can apply that expertise.

I worked as a programmer for a molecular neurobiology research group for two years. Biology is going through a kind of Cambrian explosion of new data (especially when it comes to anything that involves genetics). So it's probably not surprising that a number of people at work told me that it is extremely common to see physicists switch to biology because that's where all the exciting new research is happening, with new theories and discoveries, and lots of people who are very happy to steal whatever the physicists have already figured out about how to process and interpret mountains of data.

there is a long and distinguished history of physicists in biology (w some nobel winners). but ultimately, i wonder if a physicist could do xray crystallogrphy on the microbial fossils found on mars to determine their genetic basis. (dna fossils?)
BioPhysics was all the rage in my university few years ago. I haven't heard much after, but genetics and big data have been at the forefront for a while now. There have been a bunch of recent YC companies in this intersection as well.
Deep learning consulting ("I teach people to teach machines").

Finished my PhD in quantum optics in 2014, but immediately moved to data science.

Why not physics? Full version: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html

tl:dr: I wanted a fast-pasted field with more freedom. Physics is now stale (no fundamental changes in the last decades, compared to each year in deep learning; cf. physics in 1900-1920), and academia offers a rigid framework of grants and feudal dependence. In data science, as a freelancer/consultant, I get much more freedom. Even in companies, one is able to migrate in a matter of weeks, not years.

Money was a nice perk, not anywhere near to the main motivation.

Hey, thanks for the link. I've had a vaguely similar path (quantum info PhD -> short software stint -> lectureship -> software/data job). Your list of benefits really resonated with me.

For myself at least I would add job security as another benefit. I struggled in vain for 6-7 months during my lectureship applying to any and all postdocs I could find, so the prospect of cycling through 2-3 year postdocs in various locations eventually overwhelmed me. I loved the research but realised I didn't love it enough to potentially live apart from my partner and regularly go through months-long application processes.

On the flip side, for me freedom is actually something I miss from academia. Admittedly that's partly because I'm a regular employee rather than a freelancer. Outside of teaching, my time was basically my own and I was free to work from home or anywhere else. I'm somewhat resentful of having to justify what I'm spending my time on day by day. Not that my manager would particularly mind if I spent a day researching something slightly blue-sky, but the fact that I might have to respond to questions about it creates this background tinge of anxiety that I find prevents deep creative thought.

(comment deleted)
I'm a physicist, with a PhD in physics. I worked for a while as a post-doc in Italy and recently I quit University, working now in the private sector as a Data Scientist.

I'm happy with the change, my life is more relaxed now. I've time for my family, for me. I've also better and stable earnings.