Custom NixOS Routerfails to function as expected

I have configured a router using a Mini PC and NixOS. This was tested behind my old router and worked as expected.

When the router is connected directly to the modem, connectivity issues occur with Iot devices. Some devices are offline and other devices are slow.

What could cause this?

The strange thing is adding the extra hop fixes the problem.

My current setup with NixOS is a basic declarative approach with variables, replaced with the real values in this code block:

    boot.kernel.sysctl = {
        # forward IPv4 on all interfaces
        "net.ipv4.conf.all.forwarding" = 1;
        # deny martian packets
#        "net.ipv4.conf.default.rp_filter" = 1;
#        "net.ipv4.conf."enp1s0".rp_filter" = 1;
#        "net.ipv4.conf."br-lan".rp_filter" = 1;
        # Not using IPv6 yet
        "net.ipv6.conf.all.forwarding" = 0;
        "net.ipv6.conf.all.accept_ra" = 0;
        "net.ipv6.conf.all.autoconf" = 0;
        "net.ipv6.conf.all.use_tempaddr" = 0;
    };

    networking = {
        hostName = "${hostName}";
        enableIPv6 = false;
        nameservers = [ "198.18.1.2" ];

        nftables.enable = true;

        firewall = {
            enable = true;
            trustedInterfaces = [ "br-lan" ];

            # Allow wireguard connections through firewall
            checkReversePath = "loose";

            interfaces."br-lan" = {
                # Leave port 22 explicitly open to always allow ssh
                allowedTCPPorts = [ 22 ];
            };
        };

        nat = {
            enable = true;
            externalInterface = "enp1s0;
            internalInterfaces = [ "br-lan" ];
            internalIPs = [ "198.18.1.1/24" ];
            forwardPorts = [
                { sourcePort = 80; destination = "198.18.1.3:80"; }
                { sourcePort = 8123; destination = "198.18.1.3:8123"; }
                { sourcePort = 51820; proto = "udp"; destination = "198.18.1.19:51820"; }
            ];
        };

        bridges = {
            "br-lan" = {
                interfaces = [
                    "enp2s0"
                    "enp3s0"
                    "enp4s0"
                ];
            };
        };

        interfaces = {
            "enp1s0" = {
                useDHCP = true;
            };
            "br-lan" = {
                ipv4.addresses = [ { address = lanAddress; prefixLength = 24; } ];
                useDHCP = false;

                # Make sure the MAC address is unique
                macAddress = "00:d0:4c:10:2a:18";
            };
        };
    };

The router is connected to a cable modem in bridge mode. Behind this router are three Ubiquiti access points. The offline IoT devices are three PetSafe Automatic Pet Feeders and Samsung SmartThings devices. The Pet Feeders are 'offline', but come 'online' as soon as the old router is placed in between the new router and the modem. The Samsung devices are slow, but become 'snappy' again when the old router is present.

I tried iptables, custom nftables with only masquerading, custom iptables with only masquerading.

I tried rebooting, rebooting the modem and resetting the modem.

I tried a UPnP daemon on my new router.

I used tcpdump extensively with and without the old router in between and verified that the devices are sending packets to remote servers and also receiving packets back from those remote servers. It's hard to determine any more issues using tcpdump.

I spent a lot of time investigating the issue and am at a loss! It is really strange that adding the extra hop (the old router) fixes issues instead of creating them.

Does anyone have any ideas what I could do to find and/or fix the problem?