The social sciences are going through what has been described as a 'reproducibility crisis'(1,2). Highly influential findings derived from accessible populations, such as laboratories and crowd-sourced worker platforms, are not always replicated. Less attention has been given to replicating findings that are derived from inaccessible populations, and recent high-profile replication attempts explicitly excluded such populations(3). Pioneering experimental work(4) offered a rare glimpse into banker culture and found that bankers, in contrast to other professionals, are more dishonest when they think about their job. Given the importance of the banking sector, and before academics or policy-makers rely on these findings as an accurate diagnosis of banking culture, an exploration of their generalizability is warranted. Here we conduct the same incentivized task with bankers and non-bankers from five different populations across three continents (n = 1,282 participants). In our banker studies in the Middle East and Asia Pacific (n = 148 and n = 620, respectively), we observe some dishonesty, although-in contrast to the original study(4)-this was not significantly increased among bankers primed to think about their work compared to bankers who were not primed. We also find that inducing non-banking professionals to think about their job does not have a significant effect on honesty. We explore sampling and methodological differences to explain the variation in findings in relation to bankers and identify two key points. First, the expectations of the general population regarding banker behaviour vary across jurisdictions, suggesting that banking culture in the jurisdiction of the original study(4) may not be consistent worldwide. Second, having approached 27 financial institutions, many of which expressed concerns of adverse findings, we expect that only banks with a sound culture participated in our study. The latter introduces possible selection bias that may undermine the generalizability of any similar field study. More broadly, our study highlights the complexity of undertaking a high-fidelity replication of sensitive, highly publicized fieldwork with largely inaccessible populations resulting from institutional and geographical barriers. For policy-makers, this work suggests that caution should be exercised in generalizing the findings of the original study(4) to other populations.