Thrift Rate Limiting with Envoy + Istio

Question

I'm trying to add very basic rls[1] support to Istio for Thrift protocol stacks[2] by allowing a user to provide an external ratelimit service[3] as an environment variable to Pilot. I've run into an issue where the config seems to be rejected by the client.

Here is an isolated and simplified version of my Istio control plane code:

import (
    "github.com/envoyproxy/go-control-plane/envoy/api/v2/core"
    "istio.io/istio/pilot/pkg/model"
    "istio.io/istio/pilot/pkg/networking/util"
    ratelimit "github.com/envoyproxy/go-control-plane/envoy/config/ratelimit/v2"
    thrift_proxy "github.com/envoyproxy/go-control-plane/envoy/config/filter/network/thrift_proxy/v2alpha1"
    thrift_ratelimit "github.com/envoyproxy/go-control-plane/envoy/config/filter/thrift/rate_limit/v2alpha1"
)

// thriftListenerOpts are options for a Thrift listener
type thriftListenerOpts struct {
    // stat prefix for the thrift connection manager
    // DO not set this field. Will be overridden by buildCompleteFilterChain
    statPrefix  string
    transport   int
    protocol    int
    routeConfig *thrift_proxy.RouteConfiguration
}

func buildRatelimtedThriftProxy(node *model.Proxy, thriftOpts *thriftListenerOpts, ratelimitServiceUri string) *thrift_proxy.ThriftProxy {
    var ratelimitService *thrift_ratelimit.RateLimit
    var ratelimitFilter  *thrift_proxy.ThriftFilter
    var proxy *thrift_proxy.ThriftProxy

    proxy = &thrift_proxy.ThriftProxy{
        StatPrefix:  thriftOpts.statPrefix,
        Transport:   thrift_proxy.HEADER,
        Protocol:    thrift_proxy.BINARY,
        RouteConfig: thriftOpts.routeConfig,
    }

    ratelimitFilter = &thrift_proxy.ThriftFilter{
        Name: "config.filter.thrift.rate_limit.v2alpha1.RateLimit",
    }

    ratelimitService = &thrift_ratelimit.RateLimit{
        RateLimitService: &ratelimit.RateLimitServiceConfig{
            GrpcService: &core.GrpcService{
                TargetSpecifier: &core.GrpcService_GoogleGrpc_{
                    GoogleGrpc: &core.GrpcService_GoogleGrpc{
                        TargetUri: ratelimitServiceUri,
                    },
                },
            },
        },
    }

    if util.IsXDSMarshalingToAnyEnabled(node) {
        ratelimitFilter.ConfigType = &thrift_proxy.ThriftFilter_TypedConfig{TypedConfig: util.MessageToAny(ratelimitService)}
    } else {
        ratelimitFilter.ConfigType = &thrift_proxy.ThriftFilter_Config{Config: util.MessageToStruct(ratelimitService)}
    }

    proxy.ThriftFilters = append(proxy.ThriftFilters, ratelimitFilter)

    return proxy
}

When checking the /config_dump endpoint of Envoy's admin interface, when the ratelimit configuration is not supplied, the proxy is present and configured correctly. As soon as I try to add the ratelimit config, it disappears entirely.

What am I doing wrong?

Source code that defines Thrift ratelimit api: https://github.com/envoyproxy/envoy/blob/master/api/envoy/config/filter/thrift/rate_limit/v2alpha1/rate_limit.proto#L9
See official docs here: https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/filter/thrift/rate_limit/v2alpha1/rate_limit.proto
Reference implementation here: https://github.com/lyft/ratelimit

Hi, can you provide the following information?: Cluster platform/infrastructure, Istio version, Envoy version. — Piotr Malec

pnovotnak pnovotnak · Accepted Answer · 2019-10-30T15:55:18

Turns out I had my GrpcService configured incorrectly. It was missing the StatPrefix field. I was able to figure this out by looking at the envoy logs. I was also able to check for this in my code by running ratelimitService.Validate() which validates recursively.

Thrift Rate Limiting with Envoy + Istio

2 Answers