Atomic interference experiments can probe the gravitational redshift via the internal energy splitting of atoms and thus give direct access to test the universality of the coupling between matter-energy and gravity at different spacetime points. By including possible violations of the equivalence principle in a fully quantized treatment of all atomic degrees of freedom, we characterize how the sensitivity to gravitational redshift violations arises in atomic clocks and atom interferometers, as well as their underlying limitations. Specifically, we show that: (i) Contributions beyond linear order to trapping potentials lead to such a sensitivity of trapped atomic clocks. (ii) Bragg-type interferometers, even with a superposition of internal states, with state-independent, linear interaction potentials are at first insensitive to gravitational redshift tests. However, modified configurations, for example by relaunching the atoms, can mimic such tests under certain conditions and may constitute a competitive alternative. (iii) Guided atom interferometers are comparable to atomic clocks. (iv) Internal transitions lead to state-dependent interaction potentials through which light-pulse atom interferometers can become sensitive to gravitational redshift violations.