hardly any.
The first 12 could be considered the 'public' part of a keypair like in PGP / GnuPG or PKI. There also, you would openly publish the public part (look for public pgp keys at
http://pgp.mit.edu/ for example). Still the PGP secrets are safe as long as the user doesn't publish those. You could run into trouble if you were going to post your AES secrets from your Yubikey on the internet. I don't think you're going to do that...
theoretically there one risk as far as I can see:
if you use those 12 chars as a static password at some sites (you shouldn't)