This paper attempts to study the constrained average optimality for continuous-time Markov decision processes in the class of randomized history-dependent policies. The states and actions are in general Polish spaces, and the transition rates are allowed to be unbounded. The optimality criterion to be optimized is expected average costs, multiple constraints are imposed on similar expected average costs, and all costs may be unbounded from above and from below. Under suitable conditions, we first show the existence of a constrained optimal policy by improving the concept of a stable policy in the previous literature and using the analogue of the forward Kolmogorov equation. Then, we develop a linear program (LP), which is equivalent to the constrained optimality problem and is used to obtain a constrained optimal policy. By introducing suitable operators and conditions, we further establish the dual program (DP) of the LP, show that the LP and DP are solvable, and show that there is no duality gap between them. Finally, we use a cash flow model and a controlled birth and death system to illustrate the applications of our main results.