Kubernetes: Port Forwarding configuration

[IngwiePhoenix]

Hello there!

I am working towards putting i2pd into my k3s cluster - the long-term plan is to mimic Traefik's TraefikServce to possibly put together an I2PService; but, this is way in the future. For now, I just want to get my router working.

I enabled port-forwarding on my modem to my k3s node on my desired port, and after using the WebUI to cause a re-test, it now says Network status: Firewalled - Symmetric NAT.

I couldn't find any documentation about that - so, here is my current deployment (well, DaemonSet):

YAML DaemonSet

apiVersion: v1
kind: Namespace
metadata:
  name: i2pd
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: i2pd-data-pvc
  namespace: i2pd
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: i2pd
  namespace: i2pd
  labels:
    app: i2pd
spec:
  selector:
    matchLabels:
      app: i2pd
  template:
    metadata:
      labels:
        app: i2pd
    spec:
      tolerations:
      # these tolerations are to have the daemonset runnable on control plane nodes
      # remove them if your control plane nodes should not run pods
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      containers:
      - name: i2pd-sys
        image: purplei2p/i2pd:latest
        args:
          # Tunnel config
          # TODO: Gen tunnel confs? o.o
          #- --tunconf=...
          # Logging
          - --log=stdout
          - --loglevel=error
          # Basics
          - --port=4206
          - --ipv4
          #- --ipv6=false
          - --ssu
          - --bandwidth=X
          - --share=100
          # [ntcp2] section
          - --ntcp2.enabled=true
          - --ntcp2.published=true
          # [ssu2]
          - --ssu2.enabled=true
          - --ssu2.published=true
          # [http]
          - --http.enabled=true
          - --http.address=0.0.0.0
          - --http.port=7070
          - --http.strictheaders=true
          - --http.hostname=i2pd.birb.it
          # [httpproxy]
          - --httpproxy.enabled=true
          - --httpproxy.address=0.0.0.0
          - --httpproxy.port=4444
          - --httpproxy.addresshelper=true
          # TODO: Proxy to Tor daemonset!
          - --httpproxy.outproxy=http://false.i2p
          # [socksproxy]
          - --socksproxy.enabled=true
          - --socksproxy.address=0.0.0.0
          - --socksproxy.port=4447
          #- --socksproxy.outproxy.enabled=tor.daemonset:9080
          # [bob]
          - --bob.enabled=true
          - --bob.address=0.0.0.0
          - --bob.port=2827
          # [sam]
          - --sam.enabled=true
          - --sam.address=0.0.0.0
          - --sam.port=7656
          - --sam.singlethread=false
          # [i2cp]
          - --i2cp.address=0.0.0.0
          - --i2cp.singlethread=false
          # [upnp]
          - --upnp.enabled=false
          # [persist]
          - --persist.profiles=true
          - --persist.addressbook=true
          # [i2pcontrol] TODO: Needs Cert Manager.
          #- --i2pcontrol.enabled=true
          #- --i2pcontrol.address=0.0.0.0
          #- --i2pcontrol.port=7650
          #- --i2pcontrol.password=
          # [reseed]
          - --reseed.verify=true
          # [trust]
          #- --trust.family=ingwiephoenix # TODO: add family files
          # [addressbook]
          - --addressbook.enabled=true
          - --addressbook.defaulturl=http://shx5vqsw7usdaunyzr2qmes2fq37oumybpudrd4jjj4e4vk4uusa.b32.i2p/hosts.txt
          - --addressbook.subscriptions=http://reg.i2p/hosts.txt,http://identiguy.i2p/hosts.txt,http://stats.i2p/cgi-bin/newhosts.txt,http://rus.i2p/hosts.txt,http://joajgazyztfssty4w2on5oaqksz6tqoxbduy553y34mf4byv6gpq.b32.i2p/export/alive-hosts.txt
          # [limits]
          - --limits.zombies=5.00
          # [exploratory]
          - --exploratory.inbound.length=2
          - --exploratory.inbound.quantity=16
          - --exploratory.outbound.length=2
          - --exploratory.outbound.quantity=16
        ports:
          # HTTP server
          - containerPort: 7070
            name: webui
            protocol: TCP
          # Proxy & Service
          - containerPort: 4206
            name: i2pd-out
            hostPort: 4206
          - containerPort: 4444
            name: proxy-http
            hostPort: 4444
          - containerPort: 4447
            name: proxy-socks5
            hostPort: 4447
          - containerPort: 7656
            name: i2pd-sam
            hostPort: 7656
          - containerPort: 2827
            name: i2pd-bob
            hostPort: 2827
          - containerPort: 7654
            name: i2pd-i2cp
            hostPort: 7654
          - containerPort: 7650
            name: i2pctrl
            hostPort: 7650
        resources:
          limits:
            memory: 1Gi
          requests:
            memory: 200Mi
        volumeMounts:
        - name: i2pd-data-vol
          mountPath: /home/i2pd/data
      terminationGracePeriodSeconds: 90
      volumes:
      - name: i2pd-data-vol
        persistentVolumeClaim:
          claimName: i2pd-data-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: i2pd-webui-svc
  namespace: i2pd
spec:
  type: ClusterIP
  ports:
    - targetPort: webui
      port: 7070
      name: webui
  selector:
    app: i2pd
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: i2pd-web
  namespace: i2pd
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`i2pd.birb.it`)
      kind: Rule
      services:
        - name: i2pd-webui-svc
          port: webui
          passHostHeader: true
          scheme: http

Got any idea what I can do here - or if this message is alright?

Thanks and kind regards,
Ingwie

[r4sas]

(Really never used k3s, but...) You must port forward hostPort: 4206 to your k3s cluster, directly to i2pd.

Maybe this can help you: https://stackoverflow.com/questions/68547804/how-to-expose-two-apps-services-over-unique-ports-with-k3d

Add: btw, you must forward both TCP and UDP.

[yhaenggi]

You have a double-NAT due to kube-proxy and your "modem".
Either change how your NAT works, or use the host network (IP of your node will be used, so it can vary. will be more complex to port-forward on your router).
You'll also need to forward traffic for your public i2pd port (svc type LoadBalancer), if you continue to use a NAT inside the cluster.

[IngwiePhoenix]

I switched to using a LoadBalancer service instead; since k3s ships with Klipper (ServiceLB), it allocated the ports and iptables rules.
But that didn't work.

So, I switched to just outright using hostNetwork: true in the podspec - and that did quite something. However, it still said "firewalled". Well, now that it was running on the host's network ns, I looked at the stats - and I saw the actual problem right away:

# netstat -tunlp | grep i2pd
tcp        0      0 0.0.0.0:7656            0.0.0.0:*               LISTEN      1387652/i2pd
tcp        0      0 0.0.0.0:2827            0.0.0.0:*               LISTEN      1387652/i2pd
tcp        0      0 0.0.0.0:7070            0.0.0.0:*               LISTEN      1387652/i2pd
tcp        0      0 0.0.0.0:4206            0.0.0.0:*               LISTEN      1387652/i2pd
tcp        0      0 0.0.0.0:4447            0.0.0.0:*               LISTEN      1387652/i2pd
tcp        0      0 0.0.0.0:4444            0.0.0.0:*               LISTEN      1387652/i2pd
udp        0      0 0.0.0.0:4206            0.0.0.0:*                           1387652/i2pd
udp        0      0 0.0.0.0:7655            0.0.0.0:*                           1387652/i2pd

...it's also a UDP port. I did not know that. I had always assumed it was just TCP. However, I had already configured it as both TCP and UDP:

So... I eliminated most of the NATing; at this point it's slimmed down to: Internet -> router -> node; and there is no node-internal NATing anymore, because it runs with the host namespace now.

Are there tools or something I can use to verify the workingness of the port? I have a remote VPS that I can use to send stuff to my public IP and look at the response. Is there something like a "get version" command I could send as a ping message?

I am quite stumped now...

[IngwiePhoenix]

I couldn't sit still so I started pushing buttons - best thing to do when you run out of ideas!

Turns out DrayTec can't handle TCP/UDP combos at all. I changed the port to just UDP, later to just TCP - and after the latter, it finally showed me this:

Before closing however; is there a list of which ports are associated to which protocol? This'd be super handy to have to make sure all other components are marked appropriately.

Thanks!

[yhaenggi]

The 2 ports you forward are for communication between routers. The others are for your local client services, check the i2pd.conf and tunnels.conf

[yhaenggi]

FYI, you shouldnt post your "public" port publicly, as it narrows down the possible routers, leading to de-anonymization. better change it ;)