Solved PFSense Routing?

---

Hello ITPro.TV crew and members. I've spent an entire week trying to resolve this issue with no luck.

Short version:
All inbound traffic that starts on the router is not going to any of the PFSense subnets unless it was started by an IP on PFSense box.
Strange thing is this worked once before. Yes, I have internet connectivity.

For ease of troubleshooting I have allowed "ANY"(ports, IP, source, and destination) across all ports via the Firewall.
Firewall logs don't show anything being blocked (logging turned on for all rejected packets,each interface, and rule)
All IPs are static except the WAN. NAT is set to dynamic. "Block private networks and loopback addresses" is unchecked across all interfaces.

Any help is greatly appreciated and willing to answer any questions.

Long Version (detials):
I have a Netgear(C7000-100NAS) router that's connected via WAN to the WAN port of my PFSense box. The router I'm assuming doesn't know what IPs are behind the WAN connection.

Router:
192.168.0.0/24 (Subnet)
192.168.0.1 (Gateway)
192.168.0.12 (WAN ---> PFSense WAN port)
192.168.0.13 (Cellphone)

PFSense Ports:
192.168.0.12 (PFSense WAN port ---> WAN Router port)
10.10.2.1 (NAS Gateway) ---> FreeNAS(10.10.2.2)
192.168.1.1 (LAN/PFSense IP)---> 192.168.1.2(PC)

I CAN:

• Ping ANY IP listed under "Router" from NAS OS
• Ping ANY IP listed under "Router" from NAS Gateway (10.10.2.1) using PFSense ping tool.
• All devices physically connected to PFSense can all ping eachother:

I CANNOT:

• Any IP that resides on the router (the 192.168.0.0/24 network) is not able to ping any of the other subnets/ports on PFSense box.
• Using traceroute it will only go one hop (the default gateway of the router) and it doesn't know where the next hop is located.

@Chris-Taylor227 said in PFSense Routing?:

Hello ITPro.TV crew and members. I've spent an entire week trying to resolve this issue with no luck.

Short version:
All inbound traffic that starts on the router is not going to any of the PFSense subnets unless it was started by an IP on PFSense box.
Strange thing is this worked once before. Yes, I have internet connectivity.

For ease of troubleshooting I have allowed "ANY"(ports, IP, source, and destination) across all ports via the Firewall.
Firewall logs don't show anything being blocked (logging turned on for all rejected packets,each interface, and rule)
All IPs are static except the WAN. NAT is set to dynamic. "Block private networks and loopback addresses" is unchecked across all interfaces.

Any help is greatly appreciated and willing to answer any questions.

Long Version (detials):
I have a Netgear(C7000-100NAS) router that's connected via WAN to the WAN port of my PFSense box. The router I'm assuming doesn't know what IPs are behind the WAN connection.

Ok. Help me out logically. is the Netgear router directly connected to your ISP? or is the pfSense firewall what is connected to your ISP (Cable modem, DSL modem, etc?)
is the following connection the picture?

ISP <----->Netgear<----->pfSense<------>LANs

Do you have the same IP address (192.168.0.12) on both the Router WAN port and the pfSense WAN Port? is this the port that is doing NAT and dynamically assigned?

Just trying to get a better topology to help troubleshoot

Router:
192.168.0.0/24 (Subnet)
192.168.0.1 (Gateway)
192.168.0.12 (WAN ---> PFSense WAN port)
192.168.0.13 (Cellphone)

PFSense Ports:
192.168.0.12 (PFSense WAN port ---> WAN Router port)
10.10.2.1 (NAS Gateway) ---> FreeNAS(10.10.2.2)
192.168.1.1 (LAN/PFSense IP)---> 192.168.1.2(PC)

I CAN:

• Ping ANY IP listed under "Router" from NAS OS
• Ping ANY IP listed under "Router" from NAS Gateway (10.10.2.1) using PFSense ping tool.
• All devices physically connected to PFSense can all ping eachother:

I CANNOT:

• Any IP that resides on the router (the 192.168.0.0/24 network) is not able to ping any of the other subnets/ports on PFSense box.
• Using traceroute it will only go one hop (the default gateway of the router) and it doesn't know where the next hop is located.

Yes, you are correct. Also it is a router/cable modem.

ISP <----->Netgear<----->(WAN)pfSense<------>LANs

We can ignore the LAN as I believe that not to be the issue since they all can talk to one another within the PFsense.

I may have found the issue or at least hopefully I'm headed in the right direction. After testing multiple things, from the router diagnostic tool I can ping my WAN (192.168.0.12) which connects to the WAN port of the PFsense. Maybe I don't understand but when I did a traceroute using the same tool in my router using the same IP I got:

"Performing traceroute to the (192.168.0.12) from (My Public IP)" <---- The trace was successful by the way.

I see 2 problems here:

1. If the routing table is correct it shouldn't be trying to find how to get to my WAN (192.168.0.12) via my public IP
2. The IP should be coming from the gateway IP of the router (192.168.0.1), especially if it's on the same interface.

Something config wise has changed and I have do idea what it is. I have started from scratch on the PFsense and router with no luck. Also even went back to a previous config on both devices as I back them up.

The only way I see resolving this issue is create static routes in my router. Unfortunately I can't do this as there is no option for it.

@Chris-Taylor227,

It looks as if the gateway of your router is on the same network as the IP address of your WAN that is connected to pfSense.

e.g.

ISP192.168.0.1<------------->(Router)<--------------->192.168.0.12(pfSense)<------------->LANs

is this correct? if so what is the IP of the Router Interface connecting to pfsense? What is the IP connecting your router interface connecting to your router? is this your dynamically assigned ip?

Yes, that's correct. My Gateway IP and WAN IP are in the same subnet on my router.

192.168.0.1 /24 (Gateway)
192.168.0.12 /24 (Wan)

The IP on the on the router that connects to pfSense is my wan 192.168.0.12 /24 so it's the same IP on both ends with the same subnetmask since it was assigned via dhcp. The device is even listed in the router.

The gateway on the router is in charge of handing out dhcp leases (allowed range is 192.168.0.10 to 192.168.0.255)

It's something to do with the WAN connection on the PFSense port, I believe. My next step is to bring out the packet analyzer to see what's going on.

@Chris-Taylor227,

I think I would see two problems, without looking at the actual configurations.

On the Router, at the simplest configuration, each port should be on a different subnet. The router may not know when it receives a packet which port to send it out. The Routing table will look at 192.168.0.0/24 on both interfaces that are directly connected. Usually a router doesn't allow both interfaces to be configured within the same subnet. I'm kinda of surprised it allowed you to do this, doesn't mean it can't do it, obviously.

With the same IP on both ends, it's duplicate IPs on a single subnet. This is always problematic.

Please just try something for me, on connection between the router and pfSense. Set the IP differently, just as a test.

Router (192.168.1.10/24) <------------------->pfSense (192.168.1.11/24)

on pfSense set the gateway of the WAN port to 192.168.1.10 and save the configuration.

@Ronnie-Wong

Dude, I love you! I did as you said and now I can ping not only my WAN ip from my router but even another LAN ip on a different port.

I have another LAN port that I can't ping even though the rules allow all traffic.

I can ping from my router:
192.168.1.1/24 (LAN/PFSense IP)

I can't ping from my router:
10.10.2.1/29 (NAS Gateway) ----> 10.10.2.1/29(NAS/PFSense IP)

For the WAN interface on the pfSense(192.168.1.11) I input 192.168.1.10 for the "IPv4 Upstream gateway" and it worked. I can ping the IP from the router.

Should I follow the same process by adding whatever the IP is on the opposite end of the cable? I tired adding WAN interface ip on the pfSense(192.168.1.11) as the "IPv4 Upstream gateway" on the NAS interface but it didn't work(not pinging). I can't use 192.168.1.10 because I get an error message that says "The gateway IP address already exists? The WAN of pfSense 192.168.1.11 can ping all my LANs though.

Appreciate the the help thus far. Gave you a shoutout of the SY0-501 this morning on how helpful you have been.

@Chris-Taylor227 said in PFSense Routing?:

@Ronnie-Wong

Dude, I love you! I did as you said and now I can ping not only my WAN ip from my router but even another LAN ip on a different port.

Glad it helped!

I have another LAN port that I can't ping even though the rules allow all traffic.

I can ping from my router:
192.168.1.1/24 (LAN/PFSense IP)

I can't ping from my router:
10.10.2.1/29 (NAS Gateway) ----> 10.10.2.1/29(NAS/PFSense IP)

For the WAN interface on the pfSense(192.168.1.11) I input 192.168.1.10 for the "IPv4 Upstream gateway" and it worked. I can ping the IP from the router.

Should I follow the same process by adding whatever the IP is on the opposite end of the cable? I tired adding WAN interface ip on the pfSense(192.168.1.11) as the "IPv4 Upstream gateway" on the NAS interface but it didn't work(not pinging). I can't use 192.168.1.10 because I get an error message that says "The gateway IP address already exists? The WAN of pfSense 192.168.1.11 can ping all my LANs though.

Ok, so here's the default setup for your pfSense interfaces if you've purchased a pfSense hardware box. The WAN port is the only port that needs a gateway; the remaining ports are essentially the default gateway ports for LANs (LAN, OPT1, OPT2...etc) so on the pfSense port connected to your NAZ, set it up as your default gateway for that subnet pfSense( 10.10.2.1/29) <---------------->(10.10.2.2/29)NAZ (or NAZ LAN--10.10.2.3-10.10.2.6)

You may need to check if you've got NAT for that interface also configured if you're allowing connections from the Internet coming in.

If you've configured it correctly, the pfSense box will route each "LAN" port through to your WAN port building the routing table of directly connected networks!

Appreciate the the help thus far. Gave you a shoutout of the SY0-501 this morning on how helpful you have been.

We're glad to help and thank you for the shout out! There's a southern proverb that I remind my ITProTV teammates: "Even a blind hog finds an acorn every once in a while." So you just asked me on a day I was lucky. The team on CompTIA Security+, they are good all the time!

@Ronnie-Wong
I've changed a few IPs for my sake but nothing that would cause issues. It still tests the same afterwards. Also I let DHCP on my router decide my WAN and it has the same IP on both ends (router--->pf) and I have no problems with that.

I did as you said. I should have been more clear; I have no problem routing traffic out.
From any device behind pfSense I CAN ping my router no problem.

My issue has to do with only being able to ping the devices in the LAN behind the pfSense.
I feel like it's something simple. I'm still troubleshooting this issue. As of right now I can do the following when it comes to traffic coming in.

Ping:(I'm showing it as each packet goes through each interface)
Router(ping tool in device)---->pf gateway/wan---->pf lan gateway/PC---->PC (Doesn't respond to ping)

Just to make sure it wasn't a Windows firewall issue a made an inbound rule to allow any protocol and destination just as a test and it still failed to ping. If I can fix one I can fix the rest. I decided to use my PC as a test as it's my only lan device that allows me to ping that far in. When pinging my NAS it will only reach the pfSense/WAN and not the local subnet for the NAS unlike my PC. The config are exactly the same just different ip/subnet.

My outbound NAT works fine; it's the traffic coming in. Is there something I need to do for inbound? I thought in the packet the source and destination just swap places when returning my request.

Honestly something is wrong. If we don't find it, I fine with resolving this issue. When I first got this pfSense setup I had no issues with pinging but one day it just changed. There may be a hardware issue. I say that because at random times I would ping my NAS and it would respond immediately and other times intermittently or not at all. Doing a fresh reload of everything does not fix it but I'll get it work one day.

Maybe trying to eventually be able to access this publicly anywhere through my router, double NAT, and a firewall is asking for too much.

@Chris-Taylor227 said in PFSense Routing?:

@Ronnie-Wong
I've changed a few IPs for my sake but nothing that would cause issues. It still tests the same afterwards. Also I let DHCP on my router decide my WAN and it has the same IP on both ends (router--->pf) and I have no problems with that.

I did as you said. I should have been more clear; I have no problem routing traffic out.
From any device behind pfSense I CAN ping my router no problem.

My issue has to do with only being able to ping the devices in the LAN behind the pfSense.

Let me make sure I'm understanding the above statement. Are you not able to ping between different LAN ports? For example, a PC on one subnet and your NAZ on another subnet?

You may need to some firewall rules to allow traffic from your LAN subnets to get access to other LAN Subnets.

I feel like it's something simple. I'm still troubleshooting this issue. As of right now I can do the following when it comes to traffic coming in.

Ping:(I'm showing it as each packet goes through each interface)
Router(ping tool in device)---->pf gateway/wan---->pf lan gateway/PC---->PC (Doesn't respond to ping)

Just to make sure it wasn't a Windows firewall issue a made an inbound rule to allow any protocol and destination just as a test and it still failed to ping. If I can fix one I can fix the rest. I decided to use my PC as a test as it's my only lan device that allows me to ping that far in. When pinging my NAS it will only reach the pfSense/WAN and not the local subnet for the NAS unlike my PC. The config are exactly the same just different ip/subnet.

My outbound NAT works fine; it's the traffic coming in. Is there something I need to do for inbound? I thought in the packet the source and destination just swap places when returning my request.

Not necessarily so, between subnets behind the WAN ports.

Honestly something is wrong. If we don't find it, I fine with resolving this issue. When I first got this pfSense setup I had no issues with pinging but one day it just changed. There may be a hardware issue. I say that because at random times I would ping my NAS and it would respond immediately and other times intermittently or not at all. Doing a fresh reload of everything does not fix it but I'll get it work one day.

Maybe trying to eventually be able to access this publicly anywhere through my router, double NAT, and a firewall is asking for too much.

It should work just fine. It may take a little time to figure out.