Once upon a time A told me: your jacket is brown. “No it’s not! It’s black!” I insisted. A bet was placed. Independent 3rd party evaluators were called in, and the verdict was: yes, the jacket is black.
And yet somehow I viscerally felt for a very long time afterwards, that the jacket was black. From a rational standpoint, there was no reason to imagine that A would lie to me about this, or that her perception of color would be worse than mine.
Having the fact that I might be wrong be very salient to me seems highly valuable.
In some domains I think I’m pretty good at this. For instance, in math I mostly know better than to be confident in any sort of conclusion without a proof, and even when I have given a proof to still be careful (was the proof correct? do the assumptions hold? is the conclusion what I needed?)
When I started doing EA stuff a year ago, I thought “cause prio” was very simple: alignment and control are good; capabilities is bad. Well, a year later I feel thoroughly stripped of this innocence, and tempted by cluelessness. I haven’t decided to give up yet, but I do now have a tic of saying “sign extremely uncertain” whenever I or someone else proposes an intervention.
I could still be better at placing more credence in other people being importantly right in discussions with me. Not only will this tend to decrease costs from me being wrong, but also in many circumstances it’s better to listen to someone than to just tell them why I think they’re wrong. For instance, if I’m talking to a collaborator, and we disagree it can be very good to understand where they’re coming from (which is probably at least a somewhat useful place).
I’ll close with the end of the jacket story. My friend T asked to see my jacket, and T concluded that it in fact used to be black and was still black in some places. Mostly the back was brown because it was worn out and old. So, in some sense A and I were both right about the color of the jacket (although A was certainly more correct). It’s often the case that both people’s intuitions for the answer have some correct parts.
tag:none