Varnish 301 redirecting loop Wordpress website

Question

Every few hours Varnish caching 301 redirection loop to domain of site itself, its happen only to one of my wordpress sites.

The problem began from the wordpress plugin JF3_Maint_Redirect, by mistakenly it was set up to redirect the site to domain itself in condition when the admin is logged off.

After Varnish cached this 301 redirect, the site stopped work only while admin not logged, because the site trying to redirect to itself again and again.

To try fix that, I removed the redirection condition in the plugin, after that removed the plugin itself and purge Varnish cache, then the redirection cleared and the site began to work normally, until few hours later Varnish again caching 301 redirection loop to domain itself, so to resolve this I need to purge Varnish every few hours.

After deleting the issued plugin, I scanned every files in wordpress folder, scanned all the wordpress DB to find the redirection rule, I found no trace of it, it's freaking out.

Where else could the redirection rule have been saved that Varnish caching it again and again?

IN ADDITION:

my etc/varnish/default.vcl:

vcl 4.0;
backend default { .host = "MY.SERVER.IP.ADDRESS"; .port = "8181";}
include "/etc/varnish/conf.d/vhosts.conf";

etc/varnish/varnish.params:

# Varnish environment configuration description. This was derived from
# the old style sysconfig/defaults settings

# Set this to 1 to make systemd reload try to switch VCL without restart.
RELOAD_VCL=1

# Set WARMUP_TIME to force a delay in reload-vcl between vcl.load and vcl.use
# This is useful when backend probe definitions need some time before declaring
# configured backends healthy, to avoid routing traffic to a non-healthy backend.
#WARMUP_TIME=0

# Main configuration file. You probably want to change it.
VARNISH_VCL_CONF=/etc/varnish/default.vcl

# Default address and port to bind to. Blank address means all IPv4
# and IPv6 interfaces, otherwise specify a host name, an IPv4 dotted
# quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=192.168.1.5
VARNISH_LISTEN_PORT=82

# Admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082

# Shared secret file for admin interface
VARNISH_SECRET_FILE=/etc/varnish/secret

# Backend storage specification, see Storage Types in the varnishd(5)
# man page for details.
VARNISH_STORAGE="malloc,256M"

# User and group for the varnishd worker processes
VARNISH_USER=varnish
VARNISH_GROUP=varnish

# Other options, see the man page varnishd(1)
#DAEMON_OPTS="-p thread_pool_min=5 -p thread_pool_max=500 -p thread_pool_timeout=300"

/etc/varnish/conf.d/vhosts/MYSITE.COM.conf:

backend cwp481c0dbdeb3a6f576863ddf5da5066ff {
    .host = "MY.SERVER.IP.ADDRESS";
    .port = "8181";
}

sub vcl_recv {
    if (req.http.host ~ "MYSITE.COM") {
       set req.backend_hint = cwp481c0dbdeb3a6f576863ddf5da5066ff;
    }

    # Always cache the following file types for all users.
    if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$") {
        unset req.http.Cookie;
    }

    # Remove any Google Analytics based cookies
    set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");

    # Do not cache AJAX requests.
    if (req.http.X-Requested-With == "XMLHttpRequest") {
        return(pass);
    }

    # Post requests will not be cached
    if (req.http.Authorization || req.method == "POST") {
        return (pass);
    }
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Do not cache Authorized requests.
    if (req.http.Authorization) {
        return(pass);
    }

    # LetsEncrypt Certbot passthrough
    if (req.url ~ "^/\.well-known/acme-challenge/") {
        return (pass);
    }

    # Forward client's IP to the backend
    if (req.restarts == 0) {
        if (req.http.X-Real-IP) {
            set req.http.X-Forwarded-For = req.http.X-Real-IP;
        } else if (req.http.X-Forwarded-For) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

    ### Wordpress ###
    if (req.url ~ "(wp-admin|post\.php|edit\.php|wp-login)") {
        return(pass);
    }
    if (req.url ~ "/wp-cron.php" || req.url ~ "preview=true") {
        return (pass);
    }

    # WP-Affiliate
    if ( req.url ~ "\?ref=" ) {
        return (pass);
    }

    set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "PHPSESSID=[^;]+(; )?", "");

    return (hash);
}

Thijs Feryn Thijs Feryn · Accepted Answer · 2020-12-14T08:43:53

What you're probably experiencing is Varnish caching the HTTP version of a WordPress page, and not keeping the request scheme into account.

I wrote a blog post about it: https://feryn.eu/blog/mixed-content-and-err-too-many-redirects-in-wordpress/

What's going on?

WordPress will create URLs for hypermedia content such as images, CSS, Javascript, and more. If the HTTPS=on superglobal is set in PHP, the https:// scheme will be used, otherwise the http scheme will be used.

When a WordPress site is HTTPS-only, a 301 redirect will take place to redirect an http:// request to an https:// request.

The problem is that Varnish is HTTP-only, and all request coming from Varnish will result in a redirect.

The trick is to expose the original protocol via an X-Forwarded-Proto header, and make sure your webserver, either Apache, or Nginx, takes this into account.

An other problem you'll probably face, is the fact that without a proper cache variation, the redirection might be cached.

To avoid these issues there are 2 things to be done:

Make your WordPress setup aware of the original protocol
Create a cache variation in Varnish for the protocol

Exposing the X-Forwarded-Proto header

You need to make sure your TLS termination software exposes the X-Forwarded-Proto: https header. That way, WordPress will know which URL scheme it should use, based on that value.

For plain HTTP connections coming straight into Varnish, you can use the following snippet to set X-Forwarded-Proto: http:

vcl 4.0;

sub vcl_recv {
    if(!req.http.X-Forwarded-Proto) {
        set req.http.X-Forwarded-Proto = "http";
    }
}

This example assumes you're using layer-7 TLS termination software like Nginx, HaProxy, or any other application loadbalancer.

Apache

If you're using Apache, you can add the Vary: X-Forwarded-Proto using the following configuration syntax:

SetEnvIf X-Forwarded-Proto "https" HTTPS=on
Header append Vary: X-Forwarded-Proto

This will set the HTTP=on environment variable that WordPress uses to determine the scheme. This is based off X-Forwarded-Proto.

On the response side, a Vary: X-Forwarded-Proto is set, which Varnish uses to create cache variations for each scheme.

Nginx

If you're using Nginx, you can add the Vary: X-Forwarded-Proto using the following configuration syntax:

add_header Vary X-Forwarded-Proto;

But of course, you need to make sure PHP gets the proper HTTPS environment variable from Nginx.

We can map the value of the $fastcgi_https variable based on $http_x_forwarded_proto, as illustrated below:

map $http_x_forwarded_proto $fastcgi_https {
  default '';
  https on;
}

And then it's just a matter of setting the HTTPS FastCGI parameter to the $fastcgi_https value in your location ~ \.php$ {} block as illustrated below:

fastcgi_param HTTPS $fastcgi_https;