In a catalog, only filter combinations with real demand and unique content should be indexed — for example, “men’s sneakers size 42.” All other combinations (color + price + brand + sort order at the same time) should not be indexed, because they display the same product list in different orders and drain crawl equity from the catalog’s actual pages. This issue is called index bloat: the index contains more “derived” URLs than the site itself.
Where Filter Pages Come From
On a catalog site, each filter attribute adds one parameter to the URL. If there are three filters — color, size, and brand — and each has 10 values, combinatorics can produce more than a thousand URLs. On top of that, sorting (sort=price_asc), view (view=grid), pagination (page=7), and session parameters are added. In practice, such a site may have only a few hundred products but offer tens of thousands of crawlable URLs.
Google calls this "faceted navigation" and provides guidance in a separate document: crawling-managing-faceted-navigation. The main idea is simple — such URLs can multiply indefinitely, so the site itself must decide which ones to keep open.
What Is the Harm?
First — crawl budget. The crawler spends the time allocated to the site not on new product pages, but on hundreds of variants of the same listing. New products are indexed later.
Second — cannibalization. The "sneaker" category, the "sneaker + black" filter, and the "sneaker + black + 42" filter compete for the same query. Google decides which one to show, and its choice is not stable.
Third — quality signal. Many filter pages contain only 2–3 products or no products at all. A page with empty results remaining in the index does not improve the site’s overall quality rating. This aspect is directly related to the indexing item in the SEO audit checklist.
Which filters are worth indexing
If all three conditions are met, the page should remain indexable:
- There is search demand. People search for "size 42 sneakers"; they don't search for "black, size 42, 300,000–400,000, cheapest to most expensive."
- The results are stable and sufficient. The list consistently contains dozens of products and does not become empty seasonally.
- The page has its own unique elements. Its own title/description, short text, and possibly its own H1.
| Page type | Index | Reason |
|---|---|---|
Category: /krossovka/ |
Yes | Primary demand |
Single attribute: /krossovka/42-razmer/ |
Yes | Searched for |
Brand + category: /krossovka/nike/ |
Yes | Demand exists |
| Combination of two or three attributes | Usually no | Low demand, duplicate content |
| Price range | No | Infinite variants |
| Sorting, view, page size | No | Same content |
| Empty filter result | No | No value to users |
Technical tools and their limits
There are four tools, and they are not interchangeable:
| Tool | What it does | What it does not do |
|---|---|---|
robots.txt Disallow |
Stops crawling | Does not remove from the index |
noindex meta tag |
Removes from the index | Does not stop crawling |
rel=canonical |
Points to the preferred version of duplicate content | It is a hint, not a command |
| Not linking at all | The URL is not discovered | It does not work if there is an external link |
The most common mistake is blocking a URL in robots.txt and adding noindex inside it. Because the robot cannot open the page, it does not see noindex; the URL may remain in the index without a description. Choose one of the two: if you need to remove the page from the index, add noindex and allow crawling; if the page is not already indexed and you simply need to save resources, use robots.txt.
Separate URL structure in advance
The most reliable solution is to put indexable filters into a clean path and move the rest into parameters. That way the rule fits on a single line:
# Indexed — path format
/krossovka/
/krossovka/nike/
/krossovka/42-razmer/
# Not indexed — parameter format
/krossovka/?color=qora&price=300000-400000
/krossovka/?sort=price_asc
# robots.txt
User-agent: *
Disallow: /*?sort=
Disallow: /*?view=
Disallow: /*?price=
Allow: /*?page=
Parameter order also matters: ?color=qora&size=42 and ?size=42&color=qora are two different URLs. In the backend, always output parameters in the same alphabetical order; otherwise, two versions of the same list can appear. Storing filter values in a # fragment is also not recommended — Google does not treat a fragment as a separate page, and the content may not be visible at all.
Don’t confuse pagination and sorting
Pagination and sorting are not the same. ?page=2 is a different product list, meaning genuinely new content; it should be left open for crawling, otherwise the product cards inside the list may not be discovered. ?sort=price_asc, on the other hand, simply displays the same products in a different order — there is no new content.
Therefore, canonicalizing page=2 to the first page is a mistake: the crawler may treat it as a duplicate and may not seriously follow the links inside it. The correct approach is for each pagination page to be self-canonical, while sorting variants should be canonicalized to the unfiltered main page. Do not rely on rel="next" and rel="prev" tags: Google officially announced in 2019 that it stopped using them as indexing signals.
Controlling Indexing Through Internal Links
Crawlers mainly discover pages through links. Therefore, simply not linking to filter combinations that should not be indexed is a powerful control. In practice: the filter panel should update results with JavaScript, but each checkbox should not exist in the HTML as an <a href>. Instead, indexable filter pages — such as "size 42" or "Nike" — should be linked from the category page with visible links.
This is how the pillar and cluster structure appears in a catalog: the category is the pillar, and selected filter pages are clusters. When building the Bisyor.uz catalog, the key decision was also made at this stage: which segments become separate pages, and the rest remain only interface elements.
How to check
- In the Search Console → Pages report, compare the number of indexed URLs with the actual number of pages on the site. If the difference is several times larger, index bloat is present.
- Open the "Duplicate without user-selected canonical" and "Crawled — currently not indexed" lists in the same report: filter URLs usually accumulate here.
- Manually check in search whether parameterized URLs have been indexed using a query like
site:sayt.uz inurl:sort=. - Check the server logs to see which addresses the crawler requested most often — this is the most reliable source.
- Before blocking, check traffic for each template: do not
noindexa filter page that is already driving traffic.
After making changes, the result will not appear immediately — for URLs to be removed from the index, the crawler needs to recrawl them, which can take weeks.
Frequently Asked Questions
Will I lose traffic if I block filter pages?
If the pages being blocked are not currently bringing traffic — no. So the order is: first check clicks for each template in Search Console, then block combinations with zero clicks and duplicate content. A filter page with traffic should instead be strengthened — add its own copy and a link from the category.
How many filter pages should be kept open?
There is no exact number; the criterion is demand. Usually a few dozen per category: slices by the most requested single attribute (size, brand, main type). A two-attribute combination is opened only if explicitly requested in search.
Can I do this myself or do I need a developer?
Configuring robots.txt and meta tags is a console task. But changing the URL structure (moving a parameter to the path, standardizing parameter order) is done on the backend. If the catalog is already live, 301 redirects from old URLs to new ones are needed — this part cannot be done without a developer.
Is index bloat considered a penalty?
No, it is not a separate penalty. The harm is indirect: crawling budget is wasted, pages compete with each other, and the share of low-quality pages on the site increases. No penalty is expected, but growth slows down.
If the indexing strategy for a catalog site is defined before the filter interface is built, you won’t later have to consolidate thousands of URLs. If you need to review this layer in your project — the workflow is described on the services page.