That number is the identifier for your key - and is directly related to the first 12 characters of your Yubikey output.
There's a calculator at
http://demo.yubico.com/php-yubico/Modhex_Calculator.php that can convert between the "Number" and the "Modhex" output of the first 12 characters. Unfortunately it looks like the Number -> ModHex conversion is broken (or at least, it is for my key), but if you try the other way around it should work.
ie, put the first 12 characters of a one-time password into the string box, select ModHex and "Convert to all formats". The "Number" field returned should match the serial number you're talking about.