Openssh, operator controlled authorized_keys

I want to control access to my machines via public keys. I'm
implanting the private key in a smartcard-like token, and giving the
tokens to people for access. They'll use the tokens like smartcards,
and ssh-agent can use those RSA keys on the tokens.

However, I don't want people to authorize other public keys (ie, not
on physical tokens) after they've logged in.

How do I configure openssh so that it'll permit a public key for a
user, without giving the user the oppertunity to change/add public
keys to the authorized list?

Also, is there a way to have a single file with the authorized keys
for *all* users? Like /etc/shadow, but for public keys rather than
passwrods.

Thanks.

Steve