Hackers and painters but not physicists?

There are two common routes that people go down after a physics PhD. The first is, of course, to stay on in academia and get a postdoc, and then hopefully another postdoc or two, and then hopefully a permanent job.

Many people fail one of these steps, and many others don’t fancy the whole process in the first place. So the second common option is to leave and look for a job that uses the same skills in some way. This could be in, for example, data science, algorithmically intensive areas of programming, quantitative finance, or industrial research and development. These are challenging jobs that require a lot of thinking, normally with long work hours attached. There isn’t much time and energy left to spare for learning physics, so mostly people don’t do that any more.

I wasn’t particularly good at my PhD, so playing the first game would have been a struggle. And there was enough that annoyed me about academia that I was pretty OK with leaving.

But I also didn’t like the look of the second game. I don’t want to do a challenging job that ‘uses my technical skills’ in some other field. I don’t care about my technical skills. They’re not even very good! I’m completely lacking in the kind of sharp, focussed intellect that excels at rapid problem solving in unfamiliar contexts. I will not pass your whiteboard interview.

I just want to think about physics. I have specific questions that are stuck in my head, and I want to work on those. I may not succeed in doing this very well or finding anything useful, but it’s not going to stop me thinking about them. Getting to use some decontextualised ‘problem solving skills’ elsewhere is not much of a consolation prize, because everything I care about is in the context itself. (This mindset actually makes most academic postdocs look quite unappealing too.)

So in my case it makes more sense to be relatively unambitious career-wise and try to free up time for learning physics. The main useful features of my current job are that it’s reasonably not horrible, has sane hours, and leaves me with mental energy to spare. Eventually I want to do better, and figure out how to cut down the hours I work.


This seems to be an unusual choice in physics. I can only think of two vaguely relevant archetypes: cranks of the classic ‘retired electrical engineer who’s just realised relativity is WRONG’ variety, and the occasional seriously impressive person who ends up the news for making important contributions to number theory while working in Subway. I feel embarrassed admitting to people what I’m doing, because I worry that the subtext they’ll pick up is either ‘I’ve deluded myself into thinking I’m an unrecognised genius’, or ‘I’ve gone full crackpot and no longer care what anyone thinks, also did you know that Einstein was wrong?’ Neither of those exactly sounds good, so I tend to talk about my plans like it’s just a big joke.

The annoying thing is that what I’m trying to do is not at all rare in other fields. Everyone understands the concept of people in the arts having day jobs, for example. The musician who works in a coffee shop to pay the bills is a standard cultural stereotype.

I like several things about this stereotype. For a start, it’s nice to simply have it available, so you can explain what you are doing to others quickly without looking too odd. (Possibly someone will give you a bit of ‘get a real job!’ grief for it, but that’s also part of the script! You already know how to play along in that exchange, so you escape a lot of awkwardness.)

There’s also no requirement for you to have any particular level of ability, so you escape the genius/crackpot dichotomy. You can be a brilliant musician, but it’s also OK to be sort of mediocre, or even downright awful, but love it anyway.

And best of all, it’s expected that your interests will be specific and contextual, that you’ll be driven by a particular obsession. If someone advises you to ‘apply your technical skills’ to writing advertising jingles, it’s a pragmatic suggestion for how to pay rent, not an indication that it could be a comparably fulfilling way to spend your time.


Maybe it’s not so surprising that the idea of people working on physics in their spare time is not well represented in the wider culture. Most people aren’t especially interested in physics, and might not even realise that you can actually care this much. But I’m not just talking about cultural representation. I’m confused about why hardly anyone seems to be doing it at all.

One very obvious point is that the financial situation is so much better than in the arts. All those jobs in data science and industrial research pay very well. This is not something I want to complain about! But it means there are strong forces pulling people into these careers, and away from having time for physics. This probably makes it harder to notice the idea in the first place.

I don’t think this is the whole story, though. In this respect it’s interesting to compare us against a different population of STEM nerds. Programmers are also well paid. But, like musicians, they still manage to have a robust culture of day jobs and side projects. It might not be such a sitcom stereotype, but within the field it’s not seen as so unusual for someone to spend the day at their dull enterprise Java job, and then go home and contribute to an open source project they really care about.

In fact, once you’ve thought of it, the financial situation makes the ‘day job’ idea easier, not harder. The most challenging and well-paid jobs may not leave you much extra time for thinking, but there are other less impressive-sounding options that still pay pretty well. Writing line-of-business applications at Large Dull Company may not be the most exciting way to spend your day, but that means you’re likely to have brainpower to spare. And if you want to reduce your hours further or have more flexibility with your time, there are reasonable pathways in consulting and contracting.

As with artists and musicians, you don’t have to be brilliant to fit this pattern. You just have to be fascinated enough with a specific idea to want to work on it even if nobody is paying you. Writing a basic CRUD app in your spare time is fine if there’s something in there that really interests you.

Paul Graham famously made the same comparison in his Hackers and Painters essay:

The other problem with startups is that there is not much overlap between the kind of software that makes money and the kind that’s interesting to write…

All makers face this problem. Prices are determined by supply and demand, and there is just not as much demand for things that are fun to work on as there is for things that solve the mundane problems of individual customers. Acting in off-Broadway plays just doesn’t pay as well as wearing a gorilla suit in someone’s booth at a trade show. Writing novels doesn’t pay as well as writing ad copy for garbage disposals. And hacking programming languages doesn’t pay as well as figuring out how to connect some company’s legacy database to their Web server.

I think the answer to this problem, in the case of software, is a concept known to nearly all makers: the day job. This phrase began with musicians, who perform at night. More generally, it means that you have one kind of work you do for money, and another for love.


So why is this missing from physics? One good reason is that working independently doesn’t make much sense for a lot of people if they want to still produce original research. If you enjoy being an experimentalist in a big collaboration, you’re out of luck. You can’t build a small version of the LHC in your shed and hope to keep contributing to high energy physics. Some areas of theory would also not work well, such as big numerical simulations, or fast-moving subfields developing highly technical methods that would be hard for an outsider to keep up with.

I think that still leaves a reasonable number of options, though. It’s the same for programmers, surely: you won’t have access to Google-sized datasets in your one-person side project, either. And I’m not necessarily even talking about original research anyway. In fact, I’m explicitly in favour of a much wider conception of what constitutes a useful contribution to physics. I talked about some of the things I value briefly in this ‘niches’ comment on David Chapman’s Meaningness blog:

What I’d like to see is more niches, in the ecological sense: more acceptable ways to be ‘successful’ in academia so that everyone isn’t stuck trying to shove each other down the same boring hill in their quest for the summit. Off the top of my head I would like all these things to be valued as highly as ‘high-impact’ research: teaching, reproduction of experiments, new ways of visualising or conceptualising existing results, communication of existing concepts to people outside of your speciality, programming new tools to make research easier, digging around in the historical archives of the field for interesting lost insights…

Many of these are completely feasible to work on as an individual outside of academia. So I don’t think this is the whole problem.

It’s true that a lot of people are in physics mostly for the problem solving element. In that case, applying your skills elsewhere for more money might look extremely attractive, as you aren’t really losing anything by solving the same kinds of problems in a different field. This could actually be enough to explain the situation completely – maybe there just are very few weirdos like me out there who care about specific questions in physics rather than broadly applicable techniques, but who also don’t try to stick it out in academia. But in that case I don’t really understand why the situation in programming is so different.

I do have to wonder, though, whether one reason that people don’t do this in physics is simply that, well, people don’t do this in physics. Nobody sees anyone else doing it (apart from a few crackpots and geniuses who are easily discounted) so they don’t think to try it themselves. This would be the most optimistic explanation, because it looks so easily changeable! Maybe if more of us continued doing physics outside of academia it would become a thing. Maybe there are already a fair number of people trying this, but they’re currently keeping rather quiet about it.

Mostly I’m just confused, though. Suggestions gratefully received!

8 thoughts on “Hackers and painters but not physicists?

  1. gavinrebeiro May 26, 2017 / 10:38 am

    Dear drossbucket,

    You’ll probably find that there are a lot of people out there who are holding day jobs (quite a few in software) and pursuing their interest in physics. A good, but perhaps not very representative, place to see it is in book reviews on sites like amazon and goodreads. You’ll see even medical doctors talk about pursuing mathematics, physics, etc., in their spare time.

    There are lots of people realising that they can pursue their intellectual interests in their spare time and earn a good living by working for (insert tech position here) instead of 5+ years in graduate education being paid next-to-nothing fighting for a dwindling number of positions.

    It’s a very sad situation really. Weighing the options as a Bsc maths student myself, I’d rather work in a programming gig. Stuff like algorithms, error-correcting codes, and discrete probability are not the most boring topics in the world, and knowledge of them definitely puts one in a good position to land a well-paying programming job.

    I think another thing to consider in terms of free time is the long-term plan. Working with not much free time for a few years programming for, say, a hedge fund is a lot of hours but after working for a couple years and saving up for a house one can cut down hours by switching to contractual work.

    The academic world, to me at least, looks to be in a toxic state. Too much for very uncertain reward in a degrading environment which doesn’t even guarantee intellectual freedom.

    Best wishes,
    Gavin

    Like

    • drossbucket May 27, 2017 / 6:02 am

      Thanks for the reply! That’s a good point about book reviews – I hadn’t really thought about it much but it’s true that you do often find good reviews by people self-studying in their own time.

      > Weighing the options as a Bsc maths student myself, I’d rather work in a programming gig.

      Yeah, programming seems to be a good choice. Doing a PhD can still be good – a lot of them still give you plenty of freedom – but after that academia rapidly gets less fun.

      > I think another thing to consider in terms of free time is the long-term plan.

      Ah, my old nemesis, the long term plan! I’m not very good at those 🙂

      Good luck finding your own route through this!

      Like

  2. robotpliers May 26, 2017 / 3:20 pm

    This is interesting, and somewhat mirrors my experience. I didn’t go all the way to a PhD, but got a BS in Applied Physics. I’ve gradually realized that you either need a job that you love to devote your time to, or a job that mostly leaves you alone to pursue your own interests. I’ve managed to get the latter, and have been working on a project sort of at the nexus of physics, programming, and gaming–trying to build a solar system / galaxy generator that roughly uses published research on exoplanets, star system formation, and climate to build much more realistic, procedurally generated universes for games.

    I suspect that physicists don’t work as hobbyists as much as in programming, art, etc because physics doesn’t have the kind of culture those examples do. My impression is that physics is thought of as something you do at a university or institution of some sort. It’s “big science.” Programming you can do on a computer at home. The physics you can do at home is closer to mathematics or philosophy, and that strays into the area of geniuses and cranks as hobby practitioners.

    Like

    • drossbucket May 27, 2017 / 6:03 am

      Thanks for the comment – your own project sounds really cool!

      Yes, I also suspect it’s a cultural thing. I’m not sure whether it has to be this way, though – certainly a lot of physics is ‘big science’ you can’t do by yourself, but a lot of programming is also ‘big programming’ you can’t do by yourself.

      > The physics you can do at home is closer to mathematics or philosophy, and that strays into the area of geniuses and cranks as hobby practitioners.

      Yes. This is a good way of putting it. I still want to clarify my thoughts on this more, but I think I agree that the ‘closer to mathematics and philosophy’ parts of physics *are* the bits you can most feasibly do by yourself (certainly that matches my own interests). I do believe that this leaves a lot of options outside of either ‘genius’ or ‘crank’ territory, but that will probably be another post sometime.

      Like

      • robotpliers May 31, 2017 / 7:42 pm

        I agree that there are definitely more options than “genius” or “crank,” but perhaps the perception is that those are the main options, which could have two possible implications: (1) People are discouraged from independent study/work in physics because they believe it is for geniuses or cranks or (2) people *do* pursue independent study/work in physics, but they are aware that *others* perceive it as being for geniuses or cranks and so they don’t discuss it with other people for fear of being mistakenly categorized as such. No one wants to be thought of as a crank (well, maybe some actual cranks do), and I think most people with roughly normal-sized egos are embarrassed to be labeled as geniuses, even if they really are. Someone else I read online has remarked that being labelled as a genius is actually a bad thing in our society (jealousy, high expectations, different treatment, etc.) and that self-labeling, which admitting to practicing physics/math in your spare time may be perceived as doing, is really frowned upon as arrogant.
        Whereas in programming, the attitude is that everyone should be taught to code and that most coding is easily learned, a hobby for the masses.

        Like

      • drossbucket June 1, 2017 / 5:30 pm

        robotpliers: Completely agree with this analysis. And yeah, it’s weird to compare with the ‘everyone should learn to code!’ thing in programming. I mean, part of that is economically driven, but there was a strong existing hobby culture anyway.

        Like

    • gavinrebeiro May 31, 2017 / 6:22 am

      Dear robotpliers,

      You make very good points. One can’t really control what one is interested in, so it is unfortunate that those interested in “big science” definitely draw the short end of the stick if academia is not for them but they still have a desire to keep studying.

      I’m a bit lucky as my interest is mostly in mathematical logic and a few areas close to it. It would be very demoralising indeed to be interested in something and have a barrier because of cost and related factors.

      From what I’ve seen it also makes a big difference on how much support one gets inside academia depending on how ‘popular’ the topic is. I’d be interested in reading more about this from @drossbucket.

      Like

      • drossbucket June 1, 2017 / 5:38 pm

        Hm, not sure I have anything special to say on popularity. I was fairly lucky in being in an area that was middle-of-the-road in terms of popularity (general relativity) so I avoided the silliness of a really trendy topic, but also there was an OK amount of funding around. Probably more now gravitational waves have been detected at last (just heard that there’s been a third detection!)

        Pure maths tends to be a lot less fad-driven than physics, so you may be in luck there as well!

        Like

Leave a comment