It seems to me that if I was to use a Yubikey to authenticate to multiple sites, and if one of those sites was compromised or the key was captured in-flight then the attacker could use it to login to other servers.

Realistically, few people have a threat model that involves an attack of that complexity (for basic 0wnage the simpler attacks work well enough that the bad guys are kept busy and wealthy). Also for the case of ssh access a skilled attacker could own the terminal and take over a session once it was authenticated, fake a BSOD and make the user think that they just lost access. A few hours of access while the legitimate user thought that they were having technical problems could do a lot of damage.

Also even if a device using Yubikey technology (USB keyboard based) had a battery and included a time-stamp, this would not prevent an attacker from logging in to server B within a matter of seconds of me logging in to server A. So the battery backed keys that display a number are also vulnerable to the same attack. It seems that solving this properly (in a cryptographic sense) would require that the key receive a challenge from the server.

As the caps-lock light is used for sending data to the key, I wonder if (using hardware that is more advanced than the current Yubikey) it would be possible to have an enhanced mode of operation which uses the caps, scroll, and num lock lights to transmit 3 bits of data per baud to send a challenge from the server, that should allow sending a 32bit challenge in a small amount of time but would require client software to manipulate the LEDs.