Blog

Earlier this year at Black Hat 2016 there was a lot of buzz around “endpoint security”. In this post I’m going to explain a few issues with a cloud approach to web firewalls. Then I’ll explain the benefits of endpoint security and why Wordfence takes an endpoint approach to protecting your investment.

The Cloud WAF Bypass Problem

Cloud firewall providers like Sucuri and Cloudflare have servers that live out on the internet. When you configure a cloud firewall (or cloud WAF), the provider will ask you to point your website at their servers by making a DNS change. Once you’ve done that, the cloud provider configures their servers so that when your web traffic arrives there, it is filtered using their firewall rules.

Once filtered, the traffic is forwarded over the public Internet to your website. In this configuration your original website is known as the “origin” or your “origin server”. That is where your site originates from and is actually hosted.

Your origin server may be hosted at Bluehost, Hostgator, Godaddy, on your own server or with one of the many other hosting providers.

In theory once you have configured your website with a cloud firewall (or cloud WAF) you are protected by it. The expectation is that attackers will try to access your website using your domain name of www.example.com, they will be pointed to the cloud WAF and their attacks will be filtered out.

In reality, if an attacker can discover your origin IP address, they can simply bypass the cloud WAF as the diagram below shows:

In the case of a cloud WAF, you aren’t actually “behind” a firewall because the server is still on the public internet. Anyone on the net can access your server directly if they discover your origin server IP address. Knowing your origin server IP allows them to simply go around the cloud firewall provider and attack your origin server.

Cloudflare have acknowledged this problem in a blog post and they provide various suggestions on how to keep your origin server IP address “secret”. They also explain how to access your origin server directly for testing if you have the IP address.

We refer to this problem as the “Cloud WAF Bypass Problem”.

The Cloud WAF Bypass Problem is Well Documented

This issue is well documented by many security bloggers. Tools like CloudPiercer.org and CrimeFlare exist to help attackers bypass a Cloud WAF.

CrimeFlare lets you look up a Cloudflare customer’s origin IP address or download an entire database of 1.5 million Cloudflare customers and what CrimeFlare detected as their origin IP address.

CloudPiercer uses an array of techniques to reveal a targets real IP address. They estimate that 70% of sites protected by cloud WAF providers have their origin IP address exposed. More detailed stats and our own data is included below.

Keeping the Origin IP Address Secret is Difficult

There are many ways to discover a site’s origin IP address. You can:

• Look up the IP address of subdomains like mail.example.com and ssh.example.com. In many cases they point to the origin IP address. (See below for detailed statistics)
• Use an IP history database like viewdns.info to look up where the origin IP was hosted before the website started using a cloud WAF.
• Use the site certificate and a search engine like Censys.io or Shodan.io to locate the origin IP address where the certificate is installed.
• Perform an action that gets the site to connect somewhere, revealing its origin IP address. For example, on WordPress you can initiate a pingback to a site which will cause it to connect back to you, revealing its origin IP address.
• Use DNS records like SPF which might reveal the origin IP or adjacent addresses.
• Examine site HTML source which may include subdomains that point to the origin IP address.
• Examine public source or log files which may include the origin IP address or subdomains pointing to the origin.

The original designers of the net never intended IP addresses to be secret. There are no standards or conventions on the net that describe how an IP address might stay secret. In fact standards like SSL certificates and de facto standards like WordPress Pingback make it virtually impossible to keep an IP address hosting a website secret.

Search engines like Censys.io and Shodan, well known in the hacker community, go around and index the internet at a network level, indexing SSL certificates and network service information. They provide a treasure trove of information to attackers who are looking for targets.

Censys.io lets you easily search for the origin IP of any website by using that sites certificate. The results below are from a search on Censys using the certificate common name (website hostname) of Reddit.com which is behind Fastly and Cloudflare. An attacker would use the origin server addresses to launch an attack that bypasses the cloud WAF.

 

34% of Cloudflare sites can be bypassed using just one of these techniques

To determine the scale of this problem we did an automated survey of 30,753 randomly selected Cloudflare customers. We tested how many sites are accessible by simply bypassing Cloudflare and accessing the site directly. We only used one technique to discover the origin IP: Looking up a common subdomain. We didn’t use the pingback technique, site certificates or any of the several other techniques detailed above.

We found that we could directly access 34% of Cloudflare customers, or one in every three customers, bypassing any security that Cloudflare provides, by simply looking up a common subdomain.

To perform the audit on subdomains, we took the following steps:

1. Verify the customer is using Cloudflare and store a copy of their website home page. We checked HTTP and HTTPS and the site with and without the www. prefix to locate it.
2. Look up common subdomain IP addresses.
3. Verify the subdomain IPs don’t point to Cloudflare.
4. Remove duplicate IPs.
5. Try to fetch the site from the origin IPs directly by connecting directly and specifying a ‘Host’ header in the HTTP request.
6. Where we received a response, compare the original page title and first asset (first item in a src= attribute) with the page we just fetched directly.
7. If they match, then count it as a successful bypass.

As you can see from the chart below, looking up the IP address using the ‘mail’ subdomain is by far the most successful technique. Over 50% of cloud WAF customers that reveal their origin IP do so through the ‘mail’ subdomain. The ‘ssh’ domain is also a common culprit, likely because Cloudflare’s own documentation use it to illustrate how you can connect directly to your own origin IP address.

The Fix: Protect the Endpoint and Prevent Bypass

Endpoint protection has taken the industry by storm during the past year. Almost every major vendor is providing some form of endpoint security. So what is the endpoint exactly? An endpoint can be defined simply as: the final target a hacker is after. In the case of desktop security, it is the workstation a user works on. In the case of mobile platforms, it’s the smartphone a user has in their pocket.

In the case of WordPress, the endpoint is the actual WordPress installation that the attacker is trying to compromise. We believe that to best protect a website, you need to protect the endpoint.

The first benefit of protecting a website at the endpoint is there is no way for an attacker to bypass the security mechanisms. They can’t go around the Wordfence Firewall because it is an integral part of the endpoint. To use the endpoint application, you have to interact with our firewall.

The second major benefit is that, by protecting the endpoint, we can provide a defense in depth strategy. We don’t just provide a firewall. We also include a malware scanner and a range of other features. It’s not feasible to include a malware scan unless you are executing on the endpoint itself.

Conclusion

To fully protect your investment you need to employ an endpoint strategy that takes a defense in depth approach to security. Wordfence takes this approach.

As our company has evolved we have had to consider whether we would invest in “cloud” security or focus on protecting our customers where their assets are. We chose to stay on the endpoint and the industry has now also shifted their focus to protecting the endpoint.

In our opinion, using a cloud WAF is like hiring a security firm in Los Angeles and asking all of your visitors to go through their Los Angeles offices before visiting you in New York. We believe in posting guards where the assets are and putting additional defenses behind that first layer of security. This proven approach works for the over 1.5 million sites we protect and it is where the industry is headed.

The post Endpoint vs Cloud Security: The Cloud WAF Bypass Problem appeared first on Wordfence.