Regardless of how you feel about ad blocking software, it’s obvious that content creators, publishers, advertisers and online users are all being affected to varying degrees. In this debate, one audience hasn’t been mentioned at all: Marketing Automation users. Here’s how I uncovered a bug in the HubSpot Marketing Automation platform that is probably hurting you, too, and how you can fix it!
Move fast; don’t break things. It’s our mantra at Rainforest QA. While we usually apply it to software development, I adopted the same mindset when I joined the company a few months ago and set out to build the marketing department from scratch. I wanted to quickly get some programs in place and bring positive impact, and do so at breakneck speed. Oh, and do all that without breaking anything along the way.
A little background: Rainforest QA is growing like a weed, and unsurprisingly the big marketing KPI right now is lead generation — specifically, sales-qualified leads (SQLs). Everything else takes a back seat.
Early on, I wanted to set up marketing automation to best use the scarce resources available while building the marketing team. I decided HubSpot would be the easiest to set up and quickest to ROI. The platform uses landing pages and forms to collect leads while tracking the success of individual marketing channels like email, social media, and advertising. To gain better insight into lead sources and the conversion of our web traffic, one of my first moves was to replace our existing form — not the whole landing page — with a HubSpot form.
Before HubSpot, our lead flow was straightforward. Prospects simply completed a demo request form on our website. The form itself was standard HTML5 and CSS that would post data to Salesforce via web-to-lead. It had been working flawlessly for months, capturing many of the leads that fueled our amazing growth.
When you work at a software QA company, the bar is set a little higher for bug tracking and site quality. So we use the Rainforest Continuous QA Platform to test the Rainforest site. It’s meta, I know. After having a few hundred HubSpot demo form submissions come through without a problem in our testing environment, we deployed the new form to production.
For the first time, we were able to drill down on converting traffic and parse out the lead sources that were driving the most conversions, and marketing nerd nirvana ensued.
Then something weird happened.
For weeks, things seemed to be going okay. Overall site traffic was increasing. Demo landing-page traffic was increasing. But demo submissions dropped to just a third of what they had been previously. Cue screeching-tire sound effect.
As our team started to handle the few leads that were coming in, we noticed an alarming decrease in lead quality. Cue car-crash sound effect.
What was going on? It was mid-September — were our prospects trapped among the army of suit-and-tie-wearing Dreamforce attendees? Was this just a one-week fluke?
The drop in demo-page conversions came after we implemented HubSpot at the beginning of September. When this happens, you’d better believe your sales team will have some concerns.
The same drop, shown week by week. By September 26, we had ripped out and replaced the HubSpot form with a standard HTML form. Notice the big jump the week after the drop. Our theory is that these might be previous submission failures coming back to the site to resubmit after not hearing from us.
After the double whammy of low lead volume and questionable quality, the whole team dove in to diagnose the problem. The HubSpot form must be broken, we thought. We tested it again, first with the Rainforest Continuous QA Platform and then I personally tested the form across a few different browsers. Each time I hit submit, the lead would populate into HubSpot and sync with Salesforce.
I was convinced HubSpot couldn’t be the problem. After all, we were still getting leads. Yes, there were far less of them and the quality of lead had dropped, but we were still getting leads.
I starting reviewing all the usual suspects when lead volume and quality dry up. The sources of the traffic compared to previous weeks was proportionally about the same. There were no new referral sites dumping low-quality traffic onto the site and none of our existing channels had fallen off the map. There was no smoking gun.
Our engineers inadvertently found the problem. I started getting email from the engineers on our team: “Hey, Dan, when I click the submit button, nothing happens.” When more than just a couple mentioned it, I knew we had a problem.
Diving into the issue further, we found that each user who had a failed submission with the Hubspot form had something in common: ad blocking. More precisely, they were using Ghostery, an adblocker popular with more technical users, the exact audience we were trying to reach.
My first move was to immediately rip and replace the HubSpot forms on our site. We switched back to the regular form that ported leads right into Salesforce. Tracking of marketing would have to take a back seat for a while. Lead flow instantly almost doubled what it had been before switching to HubSpot. The quality of leads returned as well, with the conversion rate between leads we contacted and those that became SQLs returning to around 50% compared to less than 10% the two weeks prior.
Leads that use ad blocking, and especially stringent ones like Ghostery, are going to be our core buying audience of CTOs and VPs of engineering. Our best prospects were being blocked while lower quality leads were able to sign up. It was a perfect storm of blocking the leads we wanted, and getting the ones we didn’t.
Still determined to make the HubSpot solution work, I started to research the problem. The web had nothing about HubSpot forms having problems with any ad-blocking software. Ironically, HubSpot had just published a blog post covering a related topic, “Ad Blocking is Becoming a Real Threat: Here’s what Publishers Can Do About It”, but didn’t mention any problems they themselves were having. HubSpot’s support forums and knowledgebase had no additional information either. I contacted HubSpot’s support team and promptly received the standard “It’s not HubSpot, it’s a third-party software problem” response.
As a last resort, I contacted my customer-success representative and implementation specialist to let them know we were considering other platforms. After a day or so of looking into it, HubSpot finally confirmed what we had learned the hard way: Ghostery was blocking HubSpot form submissions. HubSpot reps told us the company was working to get whitelisted with Ghostery and that it would happen with the next HubSpot release. There’s no word when exactly that will be.
Unfortunately, the cost of landing on the final solution was incredibly high. We lost two weeks’ worth of good leads. These visitors had the frustrating user experience of hitting submit on the demo request form and not receiving any feedback on the page — and no one ever got back to them because we never received their form submission. When you’re growing 20% month over month like we are, two weeks’ worth of leads can theoretically represent a missed opportunity for 50% growth. (Spoiler alert: We were still able to hit our number but only after some scrambling by our sales team.)
We’ve solved our own problem with HubSpot but what about everyone else using HubSpot forms to collect leads? How big of a problem is this really? It’s huge. For example, when a visitor is using Ghostery for ad blocking, HubSpot forms don’t work...even on HubSpot’s own site. Everyone running HubSpot is at risk of losing leads. If you’re a HubSpot customer, your conversion rate might actually be much higher than you think.
Ad blockers aren’t going away and we at Rainforest QA don’t believe they should. Online privacy is an important issue. But for this reason, all HubSpot users, including HubSpot itself, should be QA-testing their landing-page flows for issues with ad-blocking software. For many companies, that means testing dozens of landing pages and resulting flows, including thank-you page success and content delivery. That’s a lot to bite off for most in-house QA teams.
This isn’t just an issue for landing pages. Snipcart recently found that uBlock was breaking a key feature inside its product. All SaaS companies need to be aware of the effects of ad blocking not just on their marketing site but on the product itself.
HubSpot content offer pages don’t work while the Ghostery plugin is active. Once blocking is paused, the form works as intended.
The HubSpot form embedded on the Rainforest QA demo page works with Ghostery.
That’s why today we at Rainforest QA are excited to announce the release of Ad-blocker Testing. We are launching three new testing environments that include the most common ad blockers: Ghostery, AdBlock, and uBlock, with more to come in a future release. All existing Rainforest QA customers have access to these new VMs starting today.
Have you had trouble with adblockers and Hubspot forms? Try Rainforest for yourself and we'll show you how Rainforest can help you prevent plugins like Ghostery from putting a kink in your leadflow.
Today, we’re launching our no-code QA platform with a free-forever plan, making software test automation accessible to any product contributor in any company.
Let's talk about some crucial key components of company culture which are necessary for remote work to thrive.
Amazon’s Mechanical Turk, or mTurk is a powerful, under-appreciated crowdsourcing platform that allows businesses to allocate work to humans programmatically and at scale.
How do we keep the Rainforest tester network secure? In this post, we’ll dive into how we secure the actual crowd to make sure they are real, trustworthy people (and not robots).