There is no magic, as the credential helper is for storing https credentials, not ssh passphrase for private keys.
Try the ssh agent as described in Working with SSH keyword phrases . "
Please note that ssh asks for your password (not a passphrase), which may also be due to the fact that your ssh public key is not correctly registered on the remote site (and it returns to the user / password authentication method)
Vonc
source share