We study the prospects for the detection of continuous gravitational signals from normal Galactic neutron stars, i.e., nonrecycled stars. We use a synthetic population generated by evolving stellar remnants in time, according to several models. We consider the most recent constraints set by all-sky searches for continuous gravitational waves and use them for our detectability criteria. We discuss the detection prospects for the current and the next generation of gravitational-wave detectors. We find that neutron stars whose ellipticity is solely caused by magnetic deformations cannot produce any detectable signal, not even by third-generation detectors. The currently detectable sources all have B ≲ 1012 G and deformations that are not solely due to the magnetic field. For these, we find in fact that the larger the magnetic field, the higher the ellipticity required for the signal to be detectable, and this ellipticity is well above the value induced by the magnetic field. Third-generation detectors such as the Einstein Telescope and Cosmic Explorer will be able to detect up to ≈250 more sources than current detectors. We briefly treat the case of recycled neutron stars with a simplified model. We find that continuous gravitational waves from these objects will likely remain elusive to detection by current detectors, but should be detectable with the next generation of detectors.