I think I've sorted it out - one thing I didn't realize was that SSH doesn't use PAM if you do key based auth... in default setup key auth is first, then interactive password prompt and that's exactly where PAM comes to play.
As usual the answer is "RTFM and if you still don't get it, go RTFM even more"
Now I'll try to do even more... add a backup (disconnected mode) to have three entry otpions:
1. SSH with key auth (works for both on/off-line systems)
2. SSH with password and OTP (for on-line systems, testing against Yubico cloud)
3. SSH with password and OTP (for off-line systems, using
http://www.securixlive.com/yubipam)
Chances that I'll be at the machine and won't have my ssh key are rather very very slim, but better be safe than sorry
and setting it up looks like good fun
Comments/ideas welcome!
TQM