Ceramic composites composed of oxide materials have been synthesized by reaction sintering of Ca3Co4O9 with BiCuSeO nanosheets. In situ x-ray diffraction and thermogravimetric analyses of the compound powders were conducted to understand the phase transformations during heating up to 1173 K. Further thermogravimetric analyses investigated the thermal stability of the composites and the completion of reaction sintering. The microstructure of the formed phases after reaction sintering and the composition of the composites were investigated for varying mixtures. Depending on the amount of BiCuSeO used, the phases present and their composition differed, having a significant impact on the thermoelectric properties. The increase of the electrical conductivity at a simultaneously high Seebeck coefficient resulted in a large power factor of 5.4 μW cm−1 K−2, more than twice that of pristine Ca3Co4O9.