For an in-depth review of the basic capabilities of Screaming Frog, review my previous blog post.
I’ll jump right into the good stuff here. Screaming Frog is my favorite SEO tool and with the amount of data it collects, you’re really only limited by your imagination and your technical chops. Fortunately, we’re here to help.
6 Advanced Screaming Frog Uses
- Auditing Meta Data
- Using the SERP Snippet Tool
- Auditing Backlinks for Site Disavowals
- Managing Site Migrations and Redirects
- Finding Schema Data and Verifying Google Analytics Code
- Connecting Google Analytics and Google Search Console API Integrations
1. Auditing Meta Data
Once you have crawled your site using the Screaming Frog tool (which you can download for free on their website), you can start to look for issues with different aspects of your site such as metadata. On the far right, there is a column that provides a quick overview of the crawl. In our example, we have highlighted the Page Titles section, where we are presented quantitative data on common page title issues such as missing, duplication, over 65 characters, and below 30 characters. From here, we are able to filter the results by issue, to present only the meta titles that are affected by the issue we have selected.
We can also use the export feature to export a list of affected meta titles, this feature helps greatly when you are looking to complete tasks found during a technical audit.
2. Using the SERP Snippet Tool
Screaming Frog provides a useful SERP snippet emulator in the lower window tab of the crawling tool, that allows you to edit page titles and meta descriptions directly in Screaming Frog. You will be given an emulated SERP snippet to see how your meta data might be displayed in a Google search results. Furthermore, you can save these changes directly in the tool for exporting to a client for approval or uploading on the site. Here’s an example:
SERP Snippet Tool – Devices
It has become increasingly important to understand how your metadata will be displayed on different devices (specifically mobile). Thankfully, Screaming Frog has us taken care of in this area as well with the ability to use the SERP Snippet Emulator in Desktop, Mobile, or Tablet views.
3. Auditing Backlinks for Site Disavowals
You can use Screaming Frog to assist you in tedious backlink audits, using the ‘list’ upload mode to audit masses of backlinks.
Step 1: Collect list of backlinks
Go to your website’s account on Google Search Console and navigate to the “Links to Your Site” page located in the “Search Traffic” menu. From here, click “Download more sample links” to acquire a list of your site’s backlinks to upload directly onto Screaming Frog for auditing. Save these links as .txt or .csv file.
Step 2: Upload the URL list to Screaming Frog
Step 3: Run Screaming Frog Tool and Test
Once the Screaming Frog Tool has finished crawling, we can look for issues in the backlinks we have uploaded. Firstly look for the backlinks that come up with errors in the response codes such as 404s. You can filter off these backlinks immediately for use on the disavow file in Google Search Console.
Step 4: Inspect Anchor Text of Backlinks
You can also look at the use of anchor text to validate the legitimacy, relevance, and authority of your outbound links. You can do this by exporting the list of outbound links through clicking on “All Out Links” in the “Bulk Export” Menu Tab. This will give you the URLs and anchor text for all links on those pages. Be sure to also use a filter on the “Destination” column that will be available in the .csv file to see if your site is linked and what anchor text has been included.
4. Managing Site Migrations and Redirects
Another incredibly helpful utilization of Screaming Frog is as a way to keep track of your SEO needs during a site migration.
Tracking and correcting crawl errors and redirects can be an arduous task in its own right and auditing them is no walk in the park either.
Thankfully, Screaming Frog comes to the rescue once again. You can upload a list of your old URLs from a previous site and crawl them to ensure that you have successfully implemented permanent 301 redirects to your new site. The beauty of this feature in Screaming Frog also comes in the ability to check for redirect chains which you can do by checking ‘always follow redirects’ in the ‘Advanced’ tab of the ‘Spider Configuration’ Menu. With this feature checked, Screaming Frog will crawl and follow any redirect chains until it reaches the final destination URL.
5. Finding Schema Data and Verifying Google Analytics Code
Screaming Frog can be used to find both schema data and Google Analytics code with the “Custom Filters” option present under the “Configuration Mode” in “Custom Search”.
To find pages on your site that contain Schema Markup, you must add a snippet of code to the custom filter (itemtype=”http://schema.org)
Different parts of your site may contain different types of schema data, to find more specific examples of schema data you have to use more specific examples of schema code in the custom filters.
You can choose the specific filter you wish to see when you have finished crawling, by clicking the ‘Custom’ tab.
For Google Analytics
The same process applies for verifying Google Analytics on your site. This time however, instead of adding the schema code we will add in the UA number and create two filters with the code. One custom filter will be for source code containing the UA number and the other filter will be for source code that does not contain the UA number. This also works for verifying Google Tag Manager on every page of your site, if that is the method that has been chosen for tracking your site’s performance.
6. Connecting Google Analytics and Google Search Console API Integrations
A very welcome new feature to the Screaming Frog tool has been the integration of Google Analytics and Google Search Console API data. A personal favorite of mine, this feature enables SEOs to tell a complete story on the organic performance of a website that is being audited with coverage of both cause and effect.
In one particular example, by using the Screaming Frog API integration, I was able to clearly discover that one of my accounts had been experiencing a low Click-Through-Rate in the face of strong organic impressions in the SERPS. Upon closer inspection, Screaming Frog showed me that those pages which suffered from low CTR had both missing meta descriptions and a lack of Schema markup.
To set up these API integrations in Screaming Frog, you need to look under the “Configuration” tab for “API Access” and then select either “Google Analytics” or “Google Search Console” respectively.
A connection must be made to the Google account that has access to Analytics or the Google Search Console account you wish to use by giving the app permission to the Screaming Frog tool to access your account and retrieve the data you desire.
From here, once you have successfully connected, you can choose the Google Analytics Account, Property, View, Segment, and Date Ranges you wish to analyze. When the crawl is completed, Google Analytics and Google Search Console data is presented in corresponding tabs in the screaming frog tool.
I hope you’ve enjoyed my two part tutorial on Screaming Frog. While you can find all of this information on other tools, Screaming Frog is unique because it compiles so much useful information in one place. Using all this data, it’s much easier to get a full understanding of a site’s success (or failure). Do you use Screaming Frog for some other inventive SEO data? Share in the comments!