# Insightfulpipe Crawler MCP server

Source: https://glimind.com/server/mcp-registry%2Fcom.insightfulpipe%2Fcrawler
Registry id: `mcp-registry/com.insightfulpipe/crawler`
Status: auth-required (checked 2026-11-22T14:58:32.016Z)

## Measurements (30 days)

- Uptime: 100%
- Checks: 700
- Outages (90 days): 0

## Install

```json
{
  "mcpServers": {
    "insightfulpipe-crawler-mcp-server": {
      "type": "http",
      "url": "https://crawler.insightfulmcp.com/"
    }
  }
}
```

Claude Code:

```sh
claude mcp add --transport http insightfulpipe-crawler-mcp-server https://crawler.insightfulmcp.com/
```

## Tools (33)

- `ai-crawler-checker`: Check permissions for AI training bots and crawlers
- `amp-checker`: Verify AMP implementation quality and discoverability
- `brand-extractor`: Extract brand design details from a website (colors, fonts, buttons, logos)
- `canonical-checker`: Validate canonical URLs and conflicting hints
- `character-encoding-checker`: Validate declared charset and encoding consistency
- `content-optimizer-checker`: Analyze on-page content quality and keyword density
- `content-quality-checker`: Score readability, word count, and duplication risk
- `deprecated-html-checker`: Detect obsolete tags or attributes still in use
- `favicon-checker`: Confirm favicon/icon discovery across devices
- `headings-checker`: Review heading hierarchy (H1-H6) for structure gaps
- `hreflang-checker`: Review international hreflang annotations
- `image-optimization-checker`: Check alt text, formats, and compression levels
- `indexability-checker`: Review robots/meta directives affecting indexation
- `internal-linking-checker`: Map internal links and surface opportunities
- `language-checker`: Verify html[lang] attributes and language targeting
- `link-analysis-checker`: Detect broken or low-quality links on page
- `link-extractor`: Extract all links from a page and categorize them (internal and external)
- `llms-txt-checker`: Parse llms.txt instructions for AI crawlers
- `meta-tags-checker`: Inspect meta title/description tags across the page
- `mobile-responsive-checker`: Audit responsive layout rules and breakpoints
- `mobile-viewport-checker`: Ensure viewport meta tags and responsive meta configs are in place
- `page-load-checker`: Measure page weight and render-blocking assets
- `page-speed-checker`: Measure load times and highlight slow resources
- `robots-txt-checker`: Parse robots.txt directives and highlight blocking issues
- `robots-txt-generator`: Generate robots.txt content with optional AI bot restrictions
- `security-headers-checker`: Check CSP/HSTS and other security headers
- `seo-audit-checker`: Comprehensive SEO audit covering technical and content issues
- `sitemap-generator`: Crawl the target domain and produce an XML sitemap
- `social-media-checker`: Validate Open Graph and Twitter Card metadata
- `ssl-certificate-checker`: Inspect certificate chain, expiry, and HTTPS enforcement
- `status-code-checker`: Trace HTTP status/redirect chains for a URL
- `structured-data-checker`: Validate Schema.org markup and structured data coverage
- `url-structure-checker`: Evaluate keyword usage and cleanliness of URLs

Measured by Glimind (https://glimind.com): MCP handshake + tool list only, no tool is ever called. CC BY 4.0.
