How to fix error error-ipsec-ike-invalid-auth-alg in Windows 11

Solution Steps

1. 1 First, check the VPN connection properties. Press Windows key + R, type 'ncpa.cpl', and press Enter. Right-click your VPN connection and select 'Properties'.
2. 2 Go to the 'Security' tab. Click 'Advanced settings' under the 'IPsec' section.
3. 3 In the 'Authentication' tab, ensure that the authentication method is set to 'Pre-shared key' or 'Certificate' as per your server's configuration. If using a pre-shared key, verify it's correct.
4. 4 Check the 'Encryption' settings. Try changing the encryption level to 'Maximum encryption strength' or 'Require encryption' to match the server's requirements.
5. 5 If the server uses a specific algorithm like AES-256, ensure your Windows supports it. Windows 11 supports most modern algorithms, but you can enable legacy ones via Group Policy. Press Win+R, type 'gpedit.msc', and press Enter. Navigate to Computer Configuration → Administrative Templates → Network → IP Security Policy. Look for settings that allow weaker algorithms.
6. 6 Alternatively, you can use PowerShell to check the IPsec algorithms. Open PowerShell as administrator and run 'Get-VpnConnection' to see your VPN settings.
7. 7 If the issue persists, try changing the VPN protocol to IKEv2, which is more flexible. In the VPN properties, under 'Type of VPN', select 'IKEv2'.
8. 8 Finally, reboot and try connecting again.