Separations in biological systems remain a challenging problem and can be particularly so in the case of biofuels, where purification; can use a significant fraction of the energy content of the fuel. For small-molecule biofuels like ethanol, reverse osmosis (RO) membranes show promise as passive purifiers, in that they allow uncharged small molecules to pass through while blocking most other components of the growth medium. Here, we examine the use of RO membranes in developing biohybrid fuel cells, closely examining the case where a direct ethanol fuel cell (DEFC) is coupled with an ongoing yeast fermentation across an RO membrane. We show that, contrary to initial good performance, the acetic acid produced by the DEFC readily diffuses back across the RO membrane and kills the fermentation after a few days. We introduce an amelioration chamber where the acetic acid is converted to acetate ions. The RO membrane rejects the acetate ions due to their charge, preventing acetic acid buildup in the fermentation. We also show that some small, charged components of the fermentation such as amino acids are imperfectly rejected by RO membranes. Because of the high sensitivity of DEFCs to low concentrations (10s of mu M) of amino acids, even a very slow diffusion of amino acids across the RO membranes can limit biohybrid fuel cell lifetimes.