Our special guest for episode 371 of the award-winning EDGE of the Web podcast was guest Will Critchlow, Co-founder and CEO of SearchPilot. Host Erin Sparks spoke with Will about the development of SearchPilot, how SEO has evolved over the last 15 years, and how machine learning is changing the SEO landscape. Here’s what we learned:
Will Critchlow: His Background and Experience
Will Critchlow is the CEO of SearchPilot, a company owned by the founders of Distilled. SearchPilot focuses on their SEO A/B testing and meta-CMS technology (previously known as Distilled ODN). Since separating SearchPilot (the technology previously known as Distilled Optimization Delivery Network, or ODN) out from Distilled and selling the consulting and conference elements of Distilled to Brainlabs at the end of January, Will’s role is now primarily running SearchPilot, though he continues to have ongoing involvement with Brainlabs as SEO Partner to guide and support the growth of SEO as a channel within that business.
And of course, COVID-19 came along and forced all the SearchLove Conferences (London, San Diego, New York to postpone the in-person events rather than try to do the virtual conferences, which have drawbacks.
SearchPilot and How it Helps SEOs
The idea behind SearchPilot is to solve the biggest problems of the world’s biggest websites. From years of consulting work, what Will and his team found that kept coming up were two things:
1) The challenge of client actually implementing an agency’s recommendations, especially in the larger enterprise level and in spite of everyone seeming to be on board and agreeing the changes would be helpful.
2) Making SEO provable and testable, and something that can be reported with the same clarity as PPC reports (we spent this much money and made this much money). And so it has three main functionalities to do this:
Content Management: SearchPilot deploys kind of like a CDN (content delivery network), typically within the CDN layer. So it sits in the CDN layer of the webstack and it can make changes to anything about the HTML or HTTP response of any page on the website. For larger clients this means we can modify pages they can’t modify through their CMS for a variety of reasons.
SEO A/B Testing: This was being done by lots of agile tech giant sites like Etsy, TripAdvisor, Pinterest, Amazon, and so on, but is outside the capability of other in-house SEO teams. These are scientific controlled testing experiments to see if any particular SEO change is going to get the desired results. And it’s not the same as CRO (conversion rate optimization) testing, which is why it needed its own tool.
Full Funnel Testing: Now you can do both SEO and CRO testing at the same time—and this is a groundbreaking development because it lets you say that xyz SEO change will have this much of an SEO impact, this much of a CRO impact, and there this kind of net impact. This is important because you can see situations where something is good for search engines but bad for users and vice versa. This leads to interesting strategic decision making. You might find a change will have a net positive impact because of how good it is for SEO/traffic but has a negative CRO impact. You can implement the change to get that net lift, and then go in and figure out how to fix the CRO issue.
All this testing is absolutely essential because it’s the only way to figure out Google’s “black box” of an algorithm that’s increasingly too complex to articulate or understand in a way to know what to do for SEO. It’s the only way to see what works and what doesn’t and to therefore align what Google likes with what users like to optimally satisfy both!
15 Years of Search and SEO
Will published a blog article on Moz, Fifteen Years Is a Long Time in SEO, in which he walked through the co-evolution of Search and SEO during those years. In the early days (the early 2000s), it was all about links as a ranking signal, but of course SEOs then exploited that notion to the hilt, so then Google had to find ways to stop the exploitation of the link graph and look at other things as well. And this is the cycle that keeps repeating itself: Google adds in a new quality or ranking signal, and people find ways to exploit it or twist it into something Google didn’t intent in its quest to present users with the best information possible for their queries. And the whole links as a signal is still being worked with even today and tweaked to make it better.
In the mid-2000s Google was really exploding in terms of its immense capacity to crawl and index content. They were also incentivizing the publishing of long-tail content, especially through AdSense, which set up this virtuous cycle of AdSense getting more long-tail content published, which is what Google was best at indexing, which just sent them skyrocketing upward. People were starting to put in long-tail queries and Google was giving good answers because they had essentially funded the creation of that content!
This fueled the growth of the content farms and so forth because it was so easy to do with AdSense. There was a lot of useful stuff that came out of this because companies could put entire databases up and have it be indexed and then long-tail queries started producing links to content that simply wasn’t there before, which helped make companies like Zoopla in the UK (their equivalent to Zillow in the US). This took place in all kinds of industries beyond real estate, such as travel, jobs, ecommerce, any kind of long-tail business.
But there was also a lot of terrible content coming out of the content farms and published on sites like eHow and then it would get spun 25 different ways, and some of it was on topics you would want good information on, not just any information. So this is what gave rise to new standards around YMYL (your money, your life) and E-A-T (expertise, authoritativeness, trust) and so on to help correct for those content abuses that were ranking just because no one else had written much about it. The Panda update started to really address that, but it’s an ongoing cycle of creation-abuse-tweak.
But on the big-picture philosophical level, there’s always been tension between Google as information purveyor and the content publishers. Back in the early 2000s there were court cases There’s been tension since the early 2000s around Google’s use of published content. There was a court case in the early 2000s centered on whether or not Google was violating copyright laws of web content publishers by crawling, copying, and storing web content and then presenting search results based on that content. Google’s argument was that it was in the public interest for there to be this kind of search engine, and the only it way it can work is by making copies of the information. They also highlighted that publishers can opt-out through various mechanisms to prevent a page or site from being crawled and indexed. In copyright law, this wouldn’t normally fly (oh, well if you had told us not to copy your content, we wouldn’t have copied it), but it was allowed when it came to Search on the internet. But now with Google providing answers based on publishers’ content but without ever sending users to the original site because it can provide everything in rich-content SERPs, that tension between Google and publishers is higher than ever and is why antitrust moves are being made against Google’s monopoly power.
Enter the Machine Learning Era of Search and SEO
Fast-forward to the present and we’re into the machine learning (ML) era of Search and SEO where Google is trying to better understand content in terms of entities and concepts and ideas and relationships between those entities, along with who should be considered authorities in various content areas.
It’s important to understand that for a number of years, there were people within Google who resisted the idea of incorporating machine learning into the organic search algorithm, but those people have now all moved or were moved out and Google is now, without a doubt, all-in on machine learning. And even while it’s still in the early days of ML, the impacts have been immense. You can see it immediately in specialized areas, such as great improvements in Google Translate, Google Image Search and image classification, speech-to-text recognition, and so on. And that’s just in the early days! So we’re at this point where the humans have done their part. We’ve written the web by putting basically the sum total of human knowledge into machine readable format, so let’s see happens next when ML takes over.
Of course this leads some people to start worrying about all the machines-out-of-control-destroying-everything scenario. But frankly, there are a whole lot of steps that have to happen to get from machine learning to autonomous killer robots, so people can relax a bit on that front. However, there are two present areas of concern that are quite legitimate. First, if ML learns from humans then it learns human biases, and that’s a real concern. The most naïve thing of all is to think an algorithm is somehow free of bias. The algorithm will have all the biases of its creators and its training data set. Second, there will always be unexpected unintentional results. Just ask a few businesses who have been hugely negatively impacted by some of Google’s rich results and how they’re presented. Because at some point you can’t fix those uneven and unequal impacts if the algorithm is optimizing to some abstract stand of what’s “better” or “best.”
But the thing to keep in mind is all of this is the end user. Remember, Google is trying to provide the best information possible to the end user making a query. As a business company, your goal is to provide the right product or service that solves a problem for an end user (customer). If both sides of this equation (search engine and business company) are both keeping the end user in mind, it should be possible for both to achieve success because they converge on the same thing, which is user/customer experience. Keep that in mind and your website will be successful.
Connect with Will Critchlow and SearchPilot
Twitter: @willcritchlow (https://twitter.com/willcritchlow)
LinkedIn: https://www.linkedin.com/in/willcritchlow
SearchPilot Twitter: @SearchPilot (https://twitter.com/SearchPilot)
SearchPilot Website: https://www.searchpilot.com
Time to Check the Pulse of Your Digital Marketing ROI
Does it feel like your digital marketing ROI is flatlining? Maybe you have no idea at all where you’re at on digital marketing ROI. Either way, start with getting a baseline through a Site Strategics report examining your SEO, content, social media, and PPC. Visit https://edgeofthewebradio.com/roi to get 30% off a comprehensive review of your digital assets!
