Electrically conductive metal organic frameworks (MOFs) are emerging as a subclass of porous materials that can have a transformative effect on electronic and renewable energy devices. Systematic advances in these materials depend critically on the accurate and reproducible characterization of their electrical properties. This is made difficult by the numerous techniques available for electrical measurements and the dependence of metrics on device architecture and numerous external variables. These challenges, common to all types of electronic materials and devices, are especially acute for porous materials, whose high surface area make them even more susceptible to interactions with contaminants in the environment. Here, we use the anisotropic semiconducting framework Cd-2(TTFTB) (TTFTB4- = tetrathiafulvalene tetrabenzoate) to benchmark several common methods available for measuring electrical properties in MOFs. We show that factors such as temperature, chemical environment (atmosphere), and illumination conditions affect the quality of the data obtained from these techniques. Consistent results emerge only when these factors are strictly controlled and the morphology and anisotropy of the Cd-2(TTFTB) single-crystal devices are taken into account. Most importantly, we show that depending on the technique, device construction, and/or the environment, a variance of 1 or even 2 orders of magnitude is not uncommon for even just one material if external factors are not controlled consistently. Differences in conductivity values of even 2 orders of magnitude should therefore be interpreted with caution, especially between different research groups comparing different compounds. These results allow us to propose a reliable protocol for collecting and reporting electrical properties of MOFs, which should help improve the consistency and comparability of reported electrical properties for this important new class of crystalline porous conductors.