The combination of nanoparticles with proteins to form functional hybrid systems is receiving undiminished attention because of its many biotechnological and medical applications. The separation of these hybrid materials from unbound free biomolecules has posed difficult challenges to fractionation and purification. Here, a model study has been carried out by removing proteins (bovine serum albumin (BSA) or lysozyme (LYS)) from the dispersion mixtures with silica nanoparticles (nominal size 20 nm) using ultrafiltration (UF) membranes. Regenerated cellulose (RC) and polyethersulfone (PES) membranes with nominal molecular weight cut-off (NMWCO) of 100 kDa, and a PES UF membrane (NMWCO 300 kDa) functionalized with UV -grafted amphoteric polymer hydrogel layer consisting of N[3-(dimethylamino)propyl]-acrylamide (DMAPAA) and 2-acrylamido-2-methyl-l-propanesulfonic acid (AMPS) and having an experimentally determined cut-off of 180 kDa (identical with the experimental data for PES 100 kDa) were studied. Membrane properties and filtration conditions, in particular pH value and flux, were selected or adapted based on data for single component feeds to achieve maximum protein transmission, complete silica retention and, hence, maximum silica/protein selectivity. Batch dead-end and continuous diafiltration processes were used for fractionation and purification. Overall, the performance of PES UF membranes was inferior compared to the other membranes because of too strong fouling. With membrane RC 100, the transmission data of LYS and BSA from the mixture with silica were 80% and 30%, respectively. With the hydrogel-functionalized PES membrane, the respective transmissions from the mixture were similar to 35% and similar to 15% for LYS and BSA, respectively. In both cases, quantitative rejection of silica could be achieved. Using continuous diafiltration, membrane RC 100 had better purification efficiency, removing a total of 91% of LYS using 6 diavolumes (DV) in 2.4 h and 84% of BSA using 10 DV in 5.5 h. With the hydrogel-functionalized PES membrane, 82% of LYS and 74% of BSA were removed using 6 and 10 DV within larger time, i.e. 4.0 and 6.8 h, respectively. Importantly, the retained silica nanoparticles remained stable in the dispersion, without any indication of aggregation. Overall, this study will add valuable knowledge to the most efficient use of ultrafiltration sieving properties for the removal or purification of proteins from systems comprising other colloidal particles having a size which is larger by a factor of only 3-10. (C) 2016 Published by Elsevier B.V.