できなくて悲しい。
Developing on Remote Machines using SSH and Visual Studio Code
Remote - SSH limitations If you clone a Git repository using SSH and your SSH key has a passphrase, VS Code's pull and sync features may hang when running remotely. Either use an SSH key without a passphrase, clone using HTTPS, or run git push from the command line to work around the issue.