Recently, I’ve been starting to explore and get back into public key crypto thanks to some really cool uses of Keybase. Today, I decided it was time to enable the “Sign new emails” setting in GPG Mail.

GPGMail Settings

After enabling it, I attempted to send a new email and discovered a rather ungraceful error.

Inappropriate ioctl for device

Once that error popped up, I couldn’t close the new mail message at all. The only fix was to quit Mail.

Findings

After some searching, some suggested setting the environment variable of GPG_TTY=$(tty). In that same thread, a part of the solution I used is also discussed.

What I ended up doing was setting the pinetry-program setting, just not to the Homebrew version. I already had MacGPG2 installed, which came with it’s own version of pinentry-mac.

To set your pinentry program to the MacGPG version, I believe you need the GPG Tools installed first. Once that’s done, simply add the following line to your ~/.gnupg/gpg-agent.conf:

pinentry-program /usr/local/MacGPG2/libexec/pinentry-mac.app/Contents/MacOS/pinentry-mac

Once that was added, I killed the GPG agent (killall gpg-agent) and it worked as expected!

For good measure, I did also add the setting of the GPG_TTY environment variable to my .zshrc:

GPG_TTY=$(tty)
export GPG_TTY

Why not use the Homebrew version?

Good question. To be honest, I don’t have a solid reason but the version that brew just installed was a little older than the version that GPG Tools provided.

# GPG Tools
$ /usr/local/MacGPG2/libexec/pinentry-mac.app/Contents/MacOS/pinentry-mac --version
pinentry-mac (pinentry) 0.9.7

# Installed via Homebrew
$ pinentry-mac --version                                                                                        
pinentry-mac (pinentry) 0.9.4

What I think was happening

As far as I can tell, GPG is trying to ask for the passphrase to unlock my key. When it does this, it doesn’t have a TTY to open, which causes it to throw the inappropriate ioctl for device error. When you explicitly tell GPG Agent which program to use to ask for your passphrase, it now knows how to handle the lack of TTY in GUI applications.