Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when setting up Envoy as a forward proxy. #34226

Open
ahcognmm opened this issue May 17, 2024 · 3 comments
Open

Error when setting up Envoy as a forward proxy. #34226

ahcognmm opened this issue May 17, 2024 · 3 comments
Labels
area/configuration area/forward proxy question Questions that are neither investigations, bugs, nor enhancements

Comments

@ahcognmm
Copy link

Error when setting up Envoy as a forward proxy.

Hi there, i want to use envoy as a proxy to handle all out going traffic. This is my set up:

Envoy config:

admin:
  address:
    socket_address:
      protocol: TCP
      address: 127.0.0.1
      port_value: 9901
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 10000
    listener_filters:
    - name: envoy.filters.listener.tls_inspector
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
              path: /dev/stdout
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: dynamic_forward_proxy_cluster
          http_filters:
          - name: envoy.filters.http.dynamic_forward_proxy
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
              dns_cache_config:
                name: dynamic_forward_proxy_cache_config
                dns_lookup_family: V4_ONLY
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: dynamic_forward_proxy_cluster
    lb_policy: CLUSTER_PROVIDED
    connect_timeout: 5s
    http2_protocol_options: {}
    cluster_type:
      name: envoy.clusters.dynamic_forward_proxy
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
        dns_cache_config:
          name: dynamic_forward_proxy_cache_config
          dns_lookup_family: V4_ONLY
        allow_insecure_cluster_options: true
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        common_tls_context:
          tls_params:
            tls_minimum_protocol_version: TLSv1_2
            tls_maximum_protocol_version: TLSv1_3
          alpn_protocols:
            - h2
            - http/1.1
          validation_context:
            trusted_ca: {filename: /etc/ssl/certs/ca-certificates.crt}

And my iptables run:

iptables -t nat -N PROXY_INIT_OUTPUT
iptables -t nat -A PROXY_INIT_OUTPUT -o lo -j RETURN
iptables -t nat -A PROXY_INIT_OUTPUT -p tcp -j REDIRECT --to-port 10000
iptables -t nat -A OUTPUT -j PROXY_INIT_OUTPUT

When I try something like curl google.com it returns this:

[2024-05-17 08:16:56.274][17][trace][connection] [source/common/network/connection_impl.cc:474] [Tags: "ConnectionId":"0"] raising connection event 2
[2024-05-17 08:16:56.274][17][trace][connection] [source/common/network/connection_impl.cc:619] [Tags: "ConnectionId":"0"] socket event: 3
[2024-05-17 08:16:56.274][17][trace][connection] [source/common/network/connection_impl.cc:742] [Tags: "ConnectionId":"0"] write ready
[2024-05-17 08:16:56.274][17][trace][connection] [source/common/network/connection_impl.cc:659] [Tags: "ConnectionId":"0"] read ready. dispatch_buffered_data=0
[2024-05-17 08:16:56.274][17][trace][connection] [source/common/network/raw_buffer_socket.cc:25] [Tags: "ConnectionId":"0"] read returns: 73
[2024-05-17 08:16:56.274][17][trace][connection] [source/common/network/raw_buffer_socket.cc:39] [Tags: "ConnectionId":"0"] read error: Resource temporarily unavailable, code: 0
[2024-05-17 08:16:56.274][17][debug][connection] [./source/common/network/connection_impl.h:98] [Tags: "ConnectionId":"0"] current connecting state: false
[2024-05-17 08:16:56.275][1][debug][upstream] [source/extensions/clusters/dynamic_forward_proxy/cluster.cc:300] Adding host info for google.com:443
[2024-05-17 08:16:56.275][1][debug][upstream] [source/extensions/clusters/dynamic_forward_proxy/cluster.cc:279] adding new dfproxy cluster host 'google.com:443'
[2024-05-17 08:16:56.275][1][debug][upstream] [source/common/upstream/upstream_impl.cc:458] transport socket match, socket default selected for host with address 142.251.175.138:443
[2024-05-17 08:16:56.275][10][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][11][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][17][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][13][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][11][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][17][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][13][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][10][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][15][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][22][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][15][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][25][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][1][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][1][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][25][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][18][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][28][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][18][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][28][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][32][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][32][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][24][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][22][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][24][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][29][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1542] membership update for TLS cluster dynamic_forward_proxy_cluster added 1 removed 0
[2024-05-17 08:16:56.275][29][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:1548] re-creating local LB for TLS cluster dynamic_forward_proxy_cluster
[2024-05-17 08:16:56.275][17][debug][connection] [./source/common/network/connection_impl.h:98] [Tags: "ConnectionId":"2"] current connecting state: true
[2024-05-17 08:16:56.275][17][debug][connection] [source/common/network/connection_impl.cc:1021] [Tags: "ConnectionId":"2"] connecting to 142.251.175.138:443
[2024-05-17 08:16:56.275][17][debug][connection] [source/common/network/connection_impl.cc:1040] [Tags: "ConnectionId":"2"] connection in progress
[2024-05-17 08:16:56.276][17][trace][connection] [source/common/network/connection_impl.cc:619] [Tags: "ConnectionId":"2"] socket event: 2
[2024-05-17 08:16:56.276][17][trace][connection] [source/common/network/connection_impl.cc:742] [Tags: "ConnectionId":"2"] write ready
[2024-05-17 08:16:56.276][17][debug][connection] [source/common/network/connection_impl.cc:751] [Tags: "ConnectionId":"2"] connected
[2024-05-17 08:16:56.276][17][trace][connection] [source/common/tls/ssl_handshaker.cc:93] [Tags: "ConnectionId":"2"] ssl error occurred while read: WANT_READ
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/connection_impl.cc:474] [Tags: "ConnectionId":"3"] raising connection event 2
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/connection_impl.cc:619] [Tags: "ConnectionId":"3"] socket event: 3
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/connection_impl.cc:742] [Tags: "ConnectionId":"3"] write ready
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/connection_impl.cc:659] [Tags: "ConnectionId":"3"] read ready. dispatch_buffered_data=0
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/raw_buffer_socket.cc:25] [Tags: "ConnectionId":"3"] read returns: 247
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/raw_buffer_socket.cc:39] [Tags: "ConnectionId":"3"] read error: Resource temporarily unavailable, code: 0
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/connection_impl.cc:534] [Tags: "ConnectionId":"3"] writing 145 bytes, end_stream false
[2024-05-17 08:16:56.276][15][trace][connection] [source/common/network/connection_impl.cc:534] [Tags: "ConnectionId":"3"] writing 11 bytes, end_stream false
[2024-05-17T08:16:56.276Z] "- - HTTP/1.1" 400 DPE 0 11 0 - "-" "-" "-" "-" "-"

I'm just a newcomer with 1 week of reading documents. Can any one point out why I got this errors and how to fix it. I have already tried to google but nothing can help.

@ahcognmm ahcognmm added the triage Issue requires triage label May 17, 2024
@ravenblackx ravenblackx added question Questions that are neither investigations, bugs, nor enhancements area/configuration area/forward proxy and removed triage Issue requires triage labels May 17, 2024
@ravenblackx
Copy link
Contributor

@phlax might be able to help, or to ping someone who is.

@phlax
Copy link
Member

phlax commented May 17, 2024

the problem/solution is not immediately obvious to me - but i have limited dfp knowledge

cc @alyssawilk @mattklein123 as codeowners

@wbpcode might also have some idea

@moderation
Copy link
Contributor

Not sure about your iptables stuff but here is a simplified config that I've been using for ages with all the custom access log and tracing stuff elided. I update my git, rust apt configs etc to leverage localhost:9904 as a proxy and it works well. You can export HTTPS_PROXY to point to this for adhoc. I never worked out how to have this work with HTTP/3

admin:
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 9903
static_resources:
  clusters:
  - cluster_type:
      name: envoy.clusters.dynamic_forward_proxy
      typed_config:
        '@type': type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
        allow_coalesced_connections: true
        dns_cache_config:
          dns_lookup_family: ALL
          name: dynamic_forward_proxy_cache_config
    connect_timeout: 2s
    dns_lookup_family: ALL
    lb_policy: CLUSTER_PROVIDED
    name: dynamic_forward_proxy_cluster
  listeners:
  - additional_addresses:
    - address:
        socket_address:
          address: ::1
          port_value: 9904
    address:
      socket_address:
        address: 127.0.0.1
        port_value: 9904
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: AUTO
          http2_protocol_options:
            allow_connect: true
          http_filters:
          - name: envoy.filters.http.dynamic_forward_proxy
            typed_config:
              '@type': type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
              dns_cache_config:
                dns_lookup_family: ALL
                name: dynamic_forward_proxy_cache_config
          - name: envoy.filters.http.router
            typed_config:
              '@type': type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          route_config:
            name: local_route
            virtual_hosts:
            - domains:
              - '*'
              name: local_service
              routes:
              - match:
                  prefix: /
                route:
                  cluster: dynamic_forward_proxy_cluster
              - match:
                  connect_matcher: {}
                route:
                  cluster: dynamic_forward_proxy_cluster
                  upgrade_configs:
                  - connect_config: {}
                    upgrade_type: CONNECT
          stat_prefix: dynamic_forward_proxy_upgrade
    name: dynamic_forward_proxy_upgrade
    traffic_direction: OUTBOUND

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/configuration area/forward proxy question Questions that are neither investigations, bugs, nor enhancements
Projects
None yet
Development

No branches or pull requests

4 participants