Electrolytic capacitors are polarity sensitive; note the "-" symbol on one side of the capacitor package.
These capacitors typically have a larger capacity than their ceramic counterparts and are used for bulk storage; this one is used to assist the battery when the LEDs turn on.
This is necessary because button cell batteries are not designed to provide as much power as the LEDs can demand.