This is a basic rule in digital and online marketing, one that you may have heard it or a version of it many times. While everyone agrees that SEO is the backbone for every business/company/individual’s online existence, they don’t often agree on the methods to implement better SEO strategies.
Sitemapping is one of many ways to boost SEO. It has been around for many years, and despite all the algorithmic changes happening in search engines, it is still an effective tool for improving your online presence. In this long-form piece, we will talk about sitemaps, its importance, types, creation, and optimization in great detail.
Contents
1- Introduction to Sitemaps and How They Work
Google Search Console showing the status of a website’s sitemapping. The bar graphs show the number of submitted and indexed web pages
If we have to define a sitemap in one line, we would say, “It is an outline of a website in the form of all the links of its pages that search engines use to crawl and index that website.” For a more detailed definition, we would refer you to the entity that is at the center of it all— Google. At the end of the day, every SEO effort is aimed at making it on the first page of Google when the relevant keywords.
So, who would be better than Google to define what a sitemap is? Let’s see what the biggest and most powerful search engine has to say about sitemaps.
“A sitemap is a file where you provide information about the pages, videos, and other files on your site, and the relationships between them. Search engines like Google read this file to more intelligently crawl your site.”
Many SEO experts explain the working of a sitemap in different ways. These dissimilar views get many users confused about how sitemaps work and what is their utility, especially if they have just forayed into SEO strategies that revolve around mapping. Google has also settled those varying accounts by giving a simple, short, and sweet explanation of how sitemaps work.
“A sitemap tells Google which pages and files you think are important in your site, and also provides valuable information about these files: for example, when the page was last updated, how often the page is changed, and any alternate language versions of a page.”
From the above explanation of Google regarding how sitemap works, it becomes clear that if done right, they can help a website to boost its SEO. Google takes the information relayed through sitemaps seriously, which also reflects via improved ranking on SERPs.
2- The Importance of Sitemaps for SEO in 2020
2.1- Not Everything Is Black and White
So, what we have gathered from the above section? Sitemapping allows search engines to index the content of a website, which consequently helps the given website to improve its SERP rank. In short, the importance of sitemapping is uncontested. However, one has to understand that the use and benefits of sitemapping are not in black and white.
This means there are caveats attached to the use of sitemaps. For instance, if you formulate the sitemap for a website that has a lot of duplicate content, it will become counterproductive at the time of indexing. As search algorithms are becoming more intuitive, search engines are able to detect duplicity on any website and can downgrade its ranking. Here, one report by SEMrush, a Boston-based marketing and analytics software company, is worth mentioning.
This above infographic indicates that nearly half of the websites that exist on the internet face duplication issues. For them, sitemapping might not yield the SEO results one expects as long as they are not giving due importance to the canonical URLs.
Canonical URLs are similar to the master copy of a videotape or a music record. If you have excessive duplicate content on your website, then you should assign some of its pages as canonical (the ones that feature original content and the ones you want to get indexed readily) and only put them in the sitemap instead of using all the pages filled with content that is more or less similar.
2.2- Instances When Sitemapping Becomes Important
While sitemapping is not the only solution to all your SEO woes (as discussed above), it is not a long-gone trend either, contrary to what many so-called SEO experts suggest. Sitemapping has always been relevant when it comes to SEO. Sitemaps for SEO in 2020 will be as important as they were in any previous year for the following reasons:
2.2.1- Sitemaps Can Take the Edge of Quickly Used Up Crawl Budget
Crawling is the means through which search bots make the acquisition of data about a website and its pages that help the search engine to index them. Crawling budget indicates the limit of pages of a website that Google bots crawl within the span of 24 hours. A large website (news websites, big online stores) has many pages, and hence, their crawl budget is used up really fast.
Such websites have many pages that don’t get crawled and consequently don’t get indexed. You can’t just pay Google to increase your site’s crawl budget. However, you can increase the crawl rate of your website somewhat by having it sitemappped thoroughly. Good sitemapping practices can help you in getting all the deeply buried webpages indexed in lesser time.
In the above snapshot, you can see the crawl rate panel on the Google Search Console. Google itself suggests website owners not to limit the maximum crawl rate. This indicates that the crawl budget is important for any website’s SEO, and needful sitemapping can help increase it. For instance, when you take care of your website’s XML sitemapping, you make it easy for search engine bots to make sense of where the mentioned links are leading. When bots take less time to crawl your website, the crawl rate automatically increases, and this affects your crawling budget eventually.
2.2.2- Sitemaps Help Websites with Bad Internal Linking Regimen
Search engines have a built-in ability to crawl pages that are interlinked. If a site doesn’t have a good internal linking strategy, it will struggle to get its pages indexed. Here too, a good sitemapping intervention can help to an extent. A page that has not been linked to any other page on the website can be picked up Google’s bots as long as it is mentioned in the XML sitemap of the website.
Having said that, we, at SEO Resellers Canada, would suggest users fix poor internal linking strategy as soon as possible to boost their website’s SEO.
2.2.3- Sitemaps Also Help Websites with Less or No Backlinking
Backlinking is a great way for websites to signal each other’s existence to search engines. Backlinking between two websites is like torrent seeding where every seeder needs the help of the other seeder to download a file. However, it is not feasible to have a backlinking on every page on your website, particularly if they are newly updated. A dynamic sitemapping strategy can help you in having those pages indexed faster.
2.2.4- New Website Owners Can Also Use Sitemapping for Head Start
It is an uphill battle for new websites to get online traction. While implementing other SEO strategies that SEO Resellers Canada can help them with, they should also have their website mapped so that Google bots can start crawling it, and its pages can appear in SERP on relevant keyword searches.
3- Types of Sitemaps
In the earlier days of Yahoo search, text sitemaps would be the norm. Now, we have images and video sitemaps. The branching out of sitemapping has come a long way in the last couple of years. However, there are three types of sitemaps, essentially:
- HTML sitemap
- XML sitemap
- News sitemap
3.1- HTML Sitemaps
HTML sitemap is actually a blueprint of any website that is available to users so they can easily navigate it. Look at the above image carefully; you have seen this and similar indexing on many websites where all the pages of the site are mentioned in a single window. This indexing for users’ convenience is called HTML sitemapping.
So, how do HTML sitemaps help with the SEO of a website since they don’t have much to do with crawling and indexing happening in the background? Before you become skeptical of HTML sitemapping and its effectiveness for boosting SEO, we want to tell you that it matters and sometimes even more than the all-important XML sitemapping.
When you make an HTML sitemap of your website, you increase its usability for the users. Users can readily find the content they are looking for when you offer them a comprehensive HTML sitemap. As the navigation of a website becomes easy, it decreases its bounce rate and also attracts more users. The intricate interlinking that happened in the HTML sitemaps also helps the search bots with the crawling.
Search engines also reward websites that take care of the user experience. HTML sitemaps improve the user experience that gives you the results (increased online traffic) that you only expect from pure SEO strategies. Google doesn’t clearly mention how it weighs HTML ranking, but it definitely hints at its importance by advising web owners to make one. Google says:
“Offer a site map to your users with links that point to the important parts of your site. If the site map has an extremely large number of links, you may want to break the site map into multiple pages.”
3.1.1- HTML Sitemap Best Practices
In order to ace HTML sitemapping take care of the following:
- Always use optimized (keyword-rich) anchor text for the sitemap
- Don’t use more than 100 links on a single sitemap; if you have a big website, breakdown the HTML sitemap in sections
- Always use a single static page instead of using an image or flash file for sitemapping
3.2- XML Sitemaps
XML sitemaps are actually the blueprint of the links to all the individual pages of your website that you want the search engine to index. XML stands for Extensible Markup Language. It is actually the format in which the links are sorted so they can be read by the machine (search bots). In the above diagram, you can see the snippet of the XML-format sitemap of our website.
As the pictorial content on the web has been increasing manifold, XML sitemapping now branches out into two subcategories: image sitemap and video sitemap.
Image Sitemap
Example of a sitemap of a page that features two images (the same XML template can be used for pages with the number of images up to 1000)
Image sitemaps are also created in XML format. Here, you need to put the links of images present on your website. By making image sitemaps and submitting them to Google, you can actually improve the Google Images search ranking of the pictorial content of your website. In image sitemapping, you can also tag the images with their locations. In this way, image sitemapping can help somewhere with localized SEO as well.
Video Sitemap
The required tags for video sitemapping
The regular structure of a video sitemap
The above image shows a video sitemap of a single page hosting a single video. It shows all the tags that Google bots use to crawl video content of websites. Video sitemap is the same as image sitemapping. Here, you will put video links of your website in XML format so they can be separately crawled and indexed. To find out how you can optimize the video content of your website for search engines, follow this link (the highlighted part in the first image).
A holistic XML sitemapping approach takes into account all the text, image, and video content of a website so that it can be indexed for better SEO separately.
3.2.1- News Sitemap
As the name suggests, these sitemaps are used for news-based websites, particularly. These websites are updated more rapidly and frequently than other domains. Therefore, Google has introduced a different sitemap format for news websites, which is slightly different from the regular XML map. If you want your news-based website to remain on top of Google News searches, you should create news sitemaps of your web pages instead of persisting with the typical XML sitemapping.
The news sitemap for Google’s indexing looks like that.
As you can see, the news sitemap has three major tags: publication, publication date, and title. The publication further divided into two sub-tags: name and language.
4- Creating XML Sitemap and Submitting It to Google
Creating an XML sitemap isn’t complex as long as you are following the right steps. The first thing you need to do is check if you already have an XML sitemap.
4.1- Checking Existing XML Sitemap of a Website
1) First, just type the address of your website
2) Then add the extension of “/sitemap.xml” to it and enter
3) You will get the XML website of your website
Showing a fragment of the XML sitemapping for SEO Resellers Canada’s website
If you don’t see a similar window opening by searching your website with a “/sitemap.xml” extension, you will need to generate your sitemap. You may need to get in touch with the developer of your website for that.
You can also create the XML sitemap of your website on your own. Before we delve into that detail, it is important to understand the difference between static and dynamic XML sitemapping because you have to pick between the two.
4.2- Static XML Sitemapping
As the name suggests, a static sitemap is an XML template with unchanged pages entries. It is easier to create since you have to put together all the website links only once. If the content on your website doesn’t require additions and updates (e.g., a history blog), then you can have good sitemapping in static XML. However, this is not the case with the majority of websites. They undergo constant updates and page additions. And as soon as you make any change to the existing mapped pages or add a new one, the static sitemap of the site renders useless.
4.3- Dynamic XML Sitemapping
Dynamic XML sitemapping is the answer to all those webpages that need constant upgrading and additions. Whether it is an online store that has to keep changing its product landing pages or a news website where the Homepage sees continuous alteration, you will need to have Dynamic XML sitemapping.
One way of having dynamic XML sitemapping is to develop it in PHP language. The PHP version of a sitemap can plug into your website, and from there, you can directly run it on your website and update it. You can also outsource the dynamic XML sitemapping management to any third-party tool. However, it is best to oversee the dynamic sitemapping on your own, because you are in the best position to make sure the important links and pages are added into the sitemap.
4.4- Creating Static XML Sitemaps for Free
If you are just starting out and trying to get the gist of how sitemaps work, then go for the DYI free sitemapping. XML-Sitemaps.com is a good option in this regard. It lets you make free static sitemaps. Their premium services include the installable version of PHP sitemap scripting and PRO sitemapping and submission.
Using XML-Sitemaps.com for creating a static sitemap is pretty simple.
1) Just open the link and type the URL of your custom-made website and click the “Start” button
2) It will start crawling your website
3) A complete crawl involves scanning of 224 pages and the addition of 167 new pages to the sitemap. If you want to see the complete sitemap in XML format, click “VIEW SITEMAP DETAILS”.
The full static XML sitemap
4.5- Creating Sitemap for CMS Websites
If a CMS (Content Management System) is hosting your website, then you may not have to go through a lot of hassle for sitemapping. Popular CMS like WordPress have their own plug-ins that will create dynamic XML sitemaps for you. For instance, Yoast SEO is the WordPress plug-in that lets you create sitemaps. The best thing about this automatic sitemapping is that it lets you set the criteria for real-time sitemapping (e.g., per-page override). This way, you can have a dynamic sitemap for your website.
The following image shows the sitemap generated by Yoast.
4.6- Validating the Sitemap Before Submission
Before submitting a sitemap to search engines, it is better to validate it. You can use a free online sitemap validator that will assess your sitemap for any formatting issues and let you know. Again XML-Sitemaps.com is a great free tool to validate your website’s sitemap. In a couple of simple steps, you can validate your sitemap.
1) Enter your website’s sitemap URL in the bar and click Validate Sitemap, e.g., https://seoresellerscanada.ca/sitemap.xml
2) Validation results are shown
Now, it is time to submit your sitemap to search engines. Here, we will discuss the submission to Google since it is the largest search engine, taking care of nearly 92% of all the search queries on the internet.
4.7- Submitting Sitemap to Google Webmaster (Search Console)
Previously known as Google Webmaster, the Search Console is the portal that allows web owners to have an overview of the indexing and take care of the optimization of their websites.
1) Sign in to Google Search Console
2) Select your website in the sidebar
3) Click on “Sitemaps”
If you do not see “Sitemaps” in the “Overview bar,” click on “Index” to expand the section further.
4) You can remove outdated and invalid sitemap, if any.
5) Enter the sitemap URL of your website in the “Add a new sitemap.” Enter the full sitemap URL in both bars.
6) Click Submit
5- Optimizing Sitemaps to Boost SEO
All the steps we have discussed above are about having a good, “optimized” sitemapping strategy. However, we will be more concise in this section on optimizing sitemaps to boost your website’s SEO.
5.1- Optimized Sitemap Means Submitted Sitemap
Creating an XML sitemap of your website reaps benefits when you submit it duly to search engines. You will have an optimized sitemap if there is no or very little difference between the pages you have submitted and the ones that have been indexed by the search engine. However, keep in mind that sitemap submission is not a sure-shot way to have every single page of your website indexed.
Nonetheless, submission of sitemap can boost your website’s SEO because:
- You can find out any errors in the indexing of your pages and rectify it
- You assist search engine bots in understanding the blueprint of your website
5.2- Prioritize Pages to Optimize the Sitemap
If your sitemap has hundreds and thousands of page entries, especially when the majority of pages have generic content, you actually feed search bots that your website might not be a good pick for most of the users. This redundancy in sitemapping can also affect the indexing of all those pages of your website that you really want to make visible in searches.
Therefore, prioritize and try to submit sitemaps that are full of high-quality pages, the ones that you want to get more impressions on. These are some signs of top-quality pages.
- They are keyword optimized
- Feature relatively unique content
- Have lots of pictorial content (both images and videos)
- Have an interactive nature (e.g., sections of user reviews and comments)
5.3- Incorporate the Indexing Feedback to Your Sitemapping
Index coverage report showing the status of submitted pages
As mentioned earlier, all the submitted pages don’t get indexed. Earlier, the process was frustrating because you couldn’t find out the reason behind their non-indexing. Now, Google has improved its index coverage report that points out the potential issues and also why some pages have not been indexed. It is important to pay heed to that feedback and incorporate it into your sitemapping. For instance, if you are noticing that Google is not indexing your pages that have generic, text-only content, try to add unique and pictorial content to them, and see how that pans out.
5.4- Don’t Add “Noindex” Pages to the Sitemap
Not every page of a website has to be indexed. For instance, you may not want to index pages that have the website policy or pages that thank costumers for making purchases. So, instead of using the “Noindex” tag with these pages in your sitemap, remove them altogether.
Earlier, we discussed the importance of the crawl budget. If you don’t want your website’s crawl budget to blow without giving you the needed indexing results, remove the noindex pages from the sitemap. It is simple to understand: if the page is not supposed to be indexed, it should not be in the sitemap.
5.4.1- Using Robots Tag
If you want to maintain the link juice/equity of a page and the other linked pages but don’t want it to show up in search results, use robots tag with that link. The robots tag addresses all the crawlers no matter what syntax they are following to not index and show the pages in search results.
Above is the example of how a robots tag can be inserted to a certain URL.
5.5- Your Sitemap Should Only Have Canonical URLs
As mentioned earlier, a Canonical URL is the master copy among many similar URLs, for example, different pages of the same product for its various sizes and colors. Search engine bots can read them as duplicate pages due to the similarity of the content. So, it is important that you assign one Canonical URL among them— the one you want search engines to index. The tag “link rel=canonical” with the given URL tells Google’s crawl bots that the corresponding page among similar pages is supposed to be indexed.
5.6- Don’t Try to Trick by Updating Modification Time
In the earlier years of SEO, people would succeed in tricking search bots into crawling and re-indexing their existing pages by only changing the modification date/time of the page. However, this trick doesn’t work anymore as crawler bots have become more intuitive. If you don’t want Google to remove date stamps from your pages, make sure you only update the modification times only when you have made considerable changes in the content.
The removal of data stamps will make your XML sitemaps weak when it comes to updated indexing.
5.7- Keep the Size of Sitemap in Check
Lastly, make sure your sitemap is not a bulky file. Google has increased the limit of the file size of a sitemap from 10MB to 50MB; still, it is still a good practice to keep your file as small as possible. Online stores find it tricky to keep the size of the sitemap in control. Many times, their URLs exceed the prescribed limit of 50,000 URLs. In that case, Google advises web owners to make more than one sitemap for their pages.
Google’s John Mueller talk about how to deal with the sitemapping of more than 50,000 URLs.
6- Conclusion
Good practices of sitemapping are key to better SEO. If you have an optimized sitemap, you can boost your website’s SEO. On the other hand, a badly managed sitemap can be counterproductive for your website’s ranking. Why sitemapping is important, and what websites need it the most? The following pointers will answer this succinctly.
- Sitemaps give valuable Metadata snippets to search engines (the frequency of page update and its time and how important are pages to each other on the website). This helps in better crawling and indexing of the webpage that eventually reflects from its improved visibility in relevant search results.
- Sitemaps are important for websites that feature a lot of archived content without any internal linking, backlinking, or have several pages that need to be indexed individually.
To get better SEO assistance for your website and other online platforms, you can contact SEO Resellers Canada anytime.
