This really resonated: "Instead, I found that simply wanting to help my trainees succeed brought the pressures of these science metrics back." It's funny that I didn't see that coming in my own career.
I like the idea of engineering in ways to update/improve goal posts regularly. Maybe no matter what the solutions are for how you quantify, keeping things dynamic is important because almost everything can be gamed. Red Queen hypothesis FTW. Will be interesting to think about how this could be implemented at a large scale though!
Your Red Queen framing is exactly right, and I wish I had put it that way in the piece. I guess the arms race isn't really between scientists and the metric designers; it's really between good-faith scientific norms and the emergent, decentralized behavior of people optimizing locally. That's what makes static solutions so fragile.
The large-scale implementation question is the one I lose sleep over. My intuition is that you'd want an institution with genuine independence from both publishers and funders, but that raises its own governance problems. I'm curious whether you see a lighter-weight version that could still be robust enough to matter.
yeah, it's a tough problem for all areas. my general strategy is to first obsess over whether it's actually demonstrably useful. and then usually if there's a will, there's a way... my opinion is that building dynamism into any solution will be an important feature to normalize with the world changing as fast as it is today. govt isn't exactly known for that!
This really resonated: "Instead, I found that simply wanting to help my trainees succeed brought the pressures of these science metrics back." It's funny that I didn't see that coming in my own career.
I like the idea of engineering in ways to update/improve goal posts regularly. Maybe no matter what the solutions are for how you quantify, keeping things dynamic is important because almost everything can be gamed. Red Queen hypothesis FTW. Will be interesting to think about how this could be implemented at a large scale though!
Your Red Queen framing is exactly right, and I wish I had put it that way in the piece. I guess the arms race isn't really between scientists and the metric designers; it's really between good-faith scientific norms and the emergent, decentralized behavior of people optimizing locally. That's what makes static solutions so fragile.
The large-scale implementation question is the one I lose sleep over. My intuition is that you'd want an institution with genuine independence from both publishers and funders, but that raises its own governance problems. I'm curious whether you see a lighter-weight version that could still be robust enough to matter.
yeah, it's a tough problem for all areas. my general strategy is to first obsess over whether it's actually demonstrably useful. and then usually if there's a will, there's a way... my opinion is that building dynamism into any solution will be an important feature to normalize with the world changing as fast as it is today. govt isn't exactly known for that!