The way the OTP works there is no logical way a device compromise would work. The server implementing the login would be the only viable point of entry for a government agency.
The way the static password works, there is no communication outside of the HID interface so no way to exploit a compromise any way.
Any off the apps can be tested using other applications to verify that the signing results work as expected. (PGP and other tokens)
The reason the firmware isn't readable or re-flashable is because that interface is closed off entirely. This is the only reason it isn't vulnerable to infection by malware (government or otherwise), do some research on 'Bad USB' for similar issue that every USB flash drive is vulnerable to because the firmware CAN be updated. If anything this is a great security feature because malware can not be loaded by intercepting packages.
Biggest checks are that all of the apps as well as the static password can be verified with other application to verify the results. For most of them a third party app dose the actual security. The yubikey is just the credential.
The main issue with using a device like this for securing data against law enforcement is that in most jurisdictions they would not require a seperate warrant to use any yubikeys that are seized when executing a warrant or are on your person if you are arrested.
For more information you might want to check out Security Now Episode 143
http://twit.tv/show/security-now/143 they go a lot into the actual technology. Its the older device so the apps were not a part at the time.
If there are any specific features of the device you would like more help understanding how you can verify using external applications I would be happy to point you in the direction of specific resources.