In many aspects, elastomers and soft biological tissues exhibit similar mechanical properties such as a pronounced nonlinear stress–strain relation and a viscoelastic response to external loads. Consequently, many models use the same rheological framework and material functions to capture their behavior. The viscosity function is thereby often assumed to be constant and the corresponding free energy function follows that one of the long-term equilibrium response. This work questions this assumption and presents a detailed study on non-Newtonian viscosity functions for elastomers and brain tissues. The viscosity functions are paired with several commonly used free energy functions and fitted to two different types of elastomers and brain tissues in cyclic and relaxation experiments, respectively. Having identified suitable viscosity and free energy functions for the different materials, numerical aspects of viscoelasticity are addressed. From the multiplicative decomposition of the deformation gradient and ensuring a non-negative dissipation rate, four equivalent viscoelasticity formulations are derived that employ different internal variables. Using an implicit exponential map as time integration scheme, the numerical behavior of these four formulations are compared among each other and numerically robust candidates are identified. The fitting results demonstrate that non-Newtonian viscosity functions significantly enhance the fitting quality. It is shown that the choice of a viscosity function is even more important than the choice of a free energy function and the classical neo-Hooke approach is often a sufficient choice. Furthermore, the numerical investigations suggest the superiority of two of the four viscoelasticity formulations, especially when complex finite element simulations are to be conducted.