Short answer
With the engine running above idle, measure voltage at the battery. It should be meaningfully higher than the resting voltage and within the range your regulator and battery chemistry specify. If it sits at battery voltage, the alternator is not contributing; if it climbs far above specification, the regulator needs attention immediately.
You will need: Multimeter, torch
A charging warning lamp tells you about one failure mode. Plenty of charging faults never light it.
The check
Measure at the battery with the engine running above idle and a modest load applied. Charging voltage should be clearly above resting voltage and within the window specified for your regulator and battery type — and those windows differ substantially between flooded, AGM, gel and lithium, so use your own specification.
At battery voltage: the alternator is not contributing. Check the belt, the field or excitation circuit, the warning-lamp circuit where it forms part of excitation, and the connections.
Far above specification: stop. Overcharging damages batteries quickly and is a fire and gassing risk. This is a regulator fault and needs attention before the system is used again.
Belt and output
A glazed or loose belt slips under high output, which shows as poor charging and belt dust. See belts, hoses and mounts.
Remember that a standard alternator at idle produces well under its rating, which is why idling to charge works so poorly — see can I run a diesel at idle to charge batteries.
Where it stops being owner work
Regulator replacement, alternator rebuilds, and anything involving lithium battery management integration should be done by someone who works on marine charging systems. Getting the regulator profile wrong on a lithium bank is expensive.