Solved Windows Built in VPN

ThomasHoey · Jul 2, 2023

Hi Forum,

I have setup a L2TP/IPSec VPN using a synology NAS and Widows 11 client.

I have successfully setup 1 laptop and it connects without an issue. My seconed laptop tho is having issues

event view has error 789
"the user system dialed a connection named VPN name which failed. the error code returned on failure is 789"

I have tried reinstalling all network devices Wi-Fi and Mini port
I have tried adding registry key AssumeUDPEncapsulationContextOnSendRule value 2 + ProhibitIpSec value 0
I have ensured all settings are fine LCP Extensions on, MS-CHAP v2 on,
I have tried reinstalling Windows OS
I have opened ports 500,1701,4500 in firewall for UDP
I have ensured IPSec Policy Agent service is running
I have ensured IKE and AuthIP IPSec service is running
I have updated all drivers

Anyone any advice.

BobOmb · Jul 2, 2023

Are you sure its not a PSK issue? 789 is a negotiation error

Are you using type: L2TP/IPSec with pre-shared key?

ThomasHoey · Jul 2, 2023

BobOmb said:
Are you sure its not a PSK issue? 789 is a negotiation error

Are you using type: L2TP/IPSec with pre-shared key?

Hi Bob,

I am using a PSK and the PSK is correct. Like I said in post I have 2 laptops 1 connects fine and the other isn't.

BobOmb · Jul 2, 2023

~~Is the "Routing and Remote Access" service running in the service list?~~ 789 says it is...

Because of the error i keep leaning towards negotiation, on the WAN adapter in network connections, click properties and make sure on the security tab, the settings are identical between the working Win11 machine that connects properly and the new Win11 machine that does not. (to allow protocols etc)

ThomasHoey · Jul 2, 2023

BobOmb said:
Is the "Routing and Remote Access" service running in the service list?

Hi Bob,

Yes this is running.

Error I am getting in Windows is:
"The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with remote computer"

ThomasHoey · Jul 2, 2023

BobOmb said:
~~Is the "Routing and Remote Access" service running in the service list?~~ 789 says it is...

Because of the error i keep leaning towards negotiation, on the WAN adapter in network connections, click properties and make sure on the security tab, the settings are identical between the working Win11 machine that connects properly and the new Win11 machine that does not. (to allow proto

ThomasHoey · Jul 2, 2023

Hi Bob,

Yes both laptops have the same settings in security.

BobOmb · Jul 2, 2023

Does credential manager store the key for VPNs? Maybe try clearing it out and re-entering it if it does.

I'm starting to grab at straws now though, based on what you have already tried, especially on a fresh install of a known good image it should be working.

ThomasHoey · Jul 2, 2023

BobOmb said:
Does credential manager store the key for VPNs? Maybe try clearing it out and re-entering it if it does.

I'm starting to grab at straws now though, based on what you have already tried, especially on a fresh install of a known good image it should be working.

Hi Bob,

I cannot see anything in credential manager.

I cleared all anyway and still same.

I think I am just going to do a clean install instead of doing a reset

BobOmb · Jul 2, 2023

ThomasHoey said:
I have tried reinstalling Windows OS

ThomasHoey said:
I think I am just going to do a clean install instead of doing a reset

Ah..

, this brings doubt back into play.. lol - Def let us know if this resolves the issue. Im genuinely curious what the fix will be.. You could try changing the PSK on the NAS as well to see if you have been entering the wrong key (or disconnect the original machine then try to reconnect it)

As a last resort, if you just want to make it work, (emergency production) you could alternatively use PPTP and the diskstation creds to connect, but it is less secure

ThomasHoey · Jul 2, 2023

BobOmb said:
Ah.. , this brings doubt back into play.. lol - Def let us know if this resolves the issue. Im genuinely curious what the fix will be.. You could try changing the PSK on the NAS as well to see if you have been entering the wrong key (or disconnect the original machine then try to reconnect it)

As a last resort, if you just want to make it work, (emergency production) you could alternatively use PPTP and the diskstation creds to connect, but it is less secure

Hi Bob,

I will keep you up-to-date. Thanks for all you help upto this point.

ThomasHoey · Jul 2, 2023

BobOmb said:
Ah.. , this brings doubt back into play.. lol - Def let us know if this resolves the issue. Im genuinely curious what the fix will be.. You could try changing the PSK on the NAS as well to see if you have been entering the wrong key (or disconnect the original machine then try to reconnect it)

As a last resort, if you just want to make it work, (emergency production) you could alternatively use PPTP and the diskstation creds to connect, but it is less secure

Hi Bob,

Fresh install of Windows resolved issue.

Thanks for your help

BobOmb · Jul 2, 2023

Glad to hear, sorry I didnt have an out of box answer.

ThomasHoey · Jul 2, 2023

BobOmb said:
Glad to hear, sorry I didnt have an out of box answer.

No sorry required. Thanks again

Solved Windows Built in VPN

New member

My Computer

Well-known member

My Computer

New member

My Computer

Well-known member

My Computer

New member

My Computer

New member

My Computer

New member

My Computer

Well-known member

My Computer

New member

My Computer

Well-known member

My Computer

New member

My Computer

New member

My Computer

Well-known member

My Computer

New member

My Computer

Similar threads