Averrhoa carambola L. is reported for its anti-obese and anti-diabetic activities. The present study aimed to investigate its aqueous methanol leaf extract (CLL) in vivo anti-obese activity along with the isolation and identification of bioactive compounds and their in vitro α-glucosidase inhibition assessment. CLL improved all obesity complications and exhibited significant activity in an obese rat model. Fourteen compounds, including four flavone glycosides (1–4) and ten dihydrochalcone glycosides (5–12), were isolated and identified using spectroscopic techniques. New compounds identified in planta included (1) apigenin 6-C-(2-deoxy-β-D-galactopyranoside)-7-O-β-D-quinovopyranoside, (8) phloretin 3′-C-(2-O-(E)-cinnamoyl-3-O-β-D-fucopyranosyl-4-O-acetyl)-β-D-fucopyranosyl-6′-O-β-D fucopyranosyl-(1/2)-α-L arabinofuranoside, (11a) phloretin3′-C-(2-O-(E)-p-coumaroyl-3-O-β-D-fucosyl-4-O-acetyl)-β-D-fucosyl-6′-O-(2-O-β-D-fucosyl)-α-L-arabinofuranoside, (11b) phloretin3′-C-(2-O-(Z)-p-coumaroyl-3-O-β-D-fucosyl-4-O-acetyl)-β-D-fucosyl-6′-O-(2-O-β-D-fucosyl)-α-L-arabinofuranoside. Carambolaside M (5), carambolaside Ia (6), carambolaside J (7), carambolaside I (9), carambolaside P (10a), carambolaside O (10b), and carambolaside Q (12), which are reported for the first time from A. carambola L. leaves, whereas luteolin 6-C-α-L-rhamnopyranosyl-(1-2)-β-D-fucopyranoside (2), apigenin 6-C-β-D-galactopyranoside (3), and apigenin 6-C-α-L-rhamnopyranosyl-(1-2)-β-L-fucopyranoside (4) are isolated for the first time from Family. Oxalidaceae. In vitro α-glucosidase inhibitory activity revealed the potential efficacy of flavone glycosides, viz., 1, 2, 3, and 4 as antidiabetic agents. In contrast, dihydrochalcone glycosides (5–11) showed weak activity, except for compound 12, which showed relatively strong activity.