Seven x2y tools remain free. Code Leak Detector is $29 as a one-time purchase.
View productsProfessional Sitemap Generator & SEO Crawler
A professional-grade desktop application for web developers, SEO specialists, and digital marketers. Built as a high-performance crawling engine, it automates website structure discovery to generate search engine-compliant XML, HTML, JSON, and TXT sitemaps.
Free — no account, no cloud processing, no telemetry. Every crawl runs entirely on your machine.
At a glance
Crawls, renders and audits run on your machine. Results export as sitemaps and CSV — nothing is sent to a vendor.
Core features
Most crawlers see only static HTML. SiteDirective uses a real browser rendering engine to execute JavaScript, wait for dynamic content to load, and discover URLs that only appear after client-side hydration. Single-page applications, React, Vue, Angular — all fully indexed.
Export sitemaps in every format search engines accept. XML for Google and Bing submission, HTML for human-readable site maps, JSON for API-driven workflows, and plain TXT for lightweight indexing. All generated locally from live crawl data.
Build, preview and validate robots.txt files with a visual rule editor. Define user-agent directives, allow/disallow patterns, crawl-delay values and sitemap references. Test rules against your crawled URL set before deploying.
Set exactly how deep the crawler goes — from a single landing page (depth 1) to a full site traversal (depth 10). Combine with URL pattern filters to scope crawls precisely. Throttle request rate to respect server resources and avoid rate limiting.
SEO analysis
Every internal and external link is checked during the crawl. Dead links are flagged with their source page, anchor text and HTTP status code. Export the full report to prioritise fixes before they hurt your search rankings.
Scan every crawled page for missing, duplicate or truncated title tags and meta descriptions. Identify pages with no H1 heading, multiple H1s, or missing canonical URLs. Colour-coded severity makes triage fast.
Pages with excessive response times, oversized payloads or slow server responses are flagged automatically. Identify the pages dragging down your Core Web Vitals before Google does it for you.
Every crawl runs on your machine. Crawled data, sitemaps, audit reports and robots.txt files are stored locally and never transmitted. No analytics, no crash reports, no cloud relay. Verify with any network monitor.
Specifications
| Product | SiteDirective |
|---|---|
| Version | v2.0.0 |
| Platform | Windows |
| Rendering | Full JavaScript engine (SPA-aware) |
| Sitemap formats | XML, HTML, JSON, TXT |
| Crawl depth | 1–10 levels configurable |
| Robots.txt | Visual architect with rule testing |
| Broken links | Internal + external 404 detection |
| Metadata audit | Titles, descriptions, H1, canonicals |
| Performance | Response time and payload flagging |
| Rate control | Configurable request throttle |
| URL filtering | Include/exclude patterns |
| Export | Sitemaps, CSV audit reports |
| Processing | 100% local — no cloud relay |
| Telemetry | 0 bytes — verified continuously |
| Account required | None — ever |
| Price | Free |
| Developer | x2y Devs Tools Ltd, Nairobi, Kenya |
Quick start
Download from Microsoft Store or itch.io. Run the installer — no administrator privileges required for standard installs. No account creation, no activation key.
Enter the root URL. Set crawl depth (1–10), enable or disable JS rendering, configure request throttle and add URL include/exclude patterns to scope the crawl.
SiteDirective discovers URLs, renders JavaScript, checks every link and audits metadata. Review the results table — status codes, response times, titles, SEO flags — all in real time.
Generate sitemaps in XML, HTML, JSON or TXT. Build a robots.txt with the visual architect. Export audit reports as CSV. Everything saves locally — upload to your server when ready.
Downloads
Changelog
Major release. Full JavaScript rendering engine for SPA-aware crawling. Robots.txt Architect with visual rule editor and testing. Multi-format sitemap export (XML, HTML, JSON, TXT). Broken link detection for internal and external URLs. Metadata audit covering titles, descriptions, H1 and canonicals. Performance flagging for heavy pages. Crawl depth control (1–10 levels). Configurable request throttling and URL pattern filtering.
Initial release. Basic sitemap generation from static HTML crawling. XML export. Simple URL discovery without JavaScript rendering.
Security model
SiteDirective makes outbound HTTP requests to the websites you choose to crawl — that is its function. It makes zero other network connections. No analytics endpoint, no crash reporter, no licence verification server, no "improve this product" telemetry. The crawled data, generated sitemaps, audit reports and robots.txt files are written to your local filesystem and exist nowhere else. Verify with Wireshark, Fiddler, GlassWire or your firewall logs.
| Telemetry | 0 bytes collected |
|---|---|
| Account | None required — ever |
| Crawl data | Stored locally only |
| Sitemaps | Saved to filesystem, never transmitted |
| Cloud processing | None — engine runs on your hardware |
| Outbound traffic | Only to crawled target sites |
| Verification | Any network monitor |
Crawling properly
A static fetch sees the HTML shell of a client-rendered application: nav links that exist only after hydration, routes that never appear in markup, and menus built by a router. Crawl that and you conclude a site is a fraction the size it is.
SiteDirective drives a full JavaScript engine, so it sees what a browser sees — including content injected after load — and records the status code and payload that a real visitor would. Depth control (1 to 10 levels) and include/exclude URL patterns keep large sites tractable without giving up rendering.
A crawler without a throttle is a denial-of-service test on your own staging box. Request throttling, depth limits and pattern filters exist so a crawl is sized to what the server can take, and so the results describe your site rather than the artefacts of a hammered environment.
This is also the right way to treat robots.txt: read it, respect it, and use the built-in architect to author rules you have actually tested before you deploy them.
XML is what search engines consume, so it is the default export. HTML sitemaps serve people and give crawlers a link graph that does not depend on a file path. JSON suits programmatic pipelines and static-site generators; TXT is the quick inventory for a review or a handover.
Because they come from the same crawl, the four cannot drift out of sync with each other — the failure mode of hand-maintained sitemap files. That single-source behaviour is most of the value of the tool.
Broken-link detection covers internal and external 404s; the metadata audit flags missing or duplicate titles and descriptions, absent H1s and canonical mismatches; performance flagging identifies heavy pages by response time and payload.
These are the same checks we run against this site before a release. An offline crawler is not a Search Console replacement — it has no idea what a search engine decided to index — but it is the cheapest way to catch a broken template, a missing canonical or an orphaned route before anybody else does.
Crawled URLs, response bodies, audit findings and exports stay on the machine that ran the crawl. There is no vendor cloud receiving your pre-launch site, your staging credentials or your client's structure.
For agencies, that is a contractual point as much as a privacy one: the crawl of a client's unreleased site is confidential data, and a hosted SEO platform stores it somewhere you did not choose.
How it works
Set the start address, choose a crawl depth between 1 and 10 and add include or exclude patterns so a marketing site is not swallowed by its own documentation subfolder.
The engine fetches each URL, renders JavaScript, records status codes and response times, and builds the link graph. Throttle to protect the target while it works.
Review broken internal and external links, metadata gaps, canonical mismatches and heavy pages from the same crawl — no second pass, no re-fetch.
Write the XML, HTML, JSON or TXT sitemap plus a CSV audit report. Validate against a live search-console submission once deployed; a local crawl proves structure, not index coverage.
| Rendering | Full JavaScript engine, SPA routes included |
|---|---|
| Crawl scope | Depth 1–10, include and exclude patterns |
| Etiquette | Configurable request throttling |
| robots.txt | Visual architect with rule testing |
| Audits | Links, metadata, canonicals, performance |
| Exports | XML · HTML · JSON · TXT sitemaps, CSV reports |
noindex and a restrictive robots.txt, confirm the crawl respects both, and only then point it at production. A crawler that cannot be tamed by those files will annoy every server it ever touches.Who it is for
Audit a client site before and after a migration, and hand over the CSV report without the crawl itself living on a vendor's server.
Catch orphaned routes, broken internal links and missing metadata in a SPA build before it ships.
Generate the XML and HTML sitemaps your host expects, from a local crawl, with no third-party account in the loop.
Large doc trees are exactly where depth limits and URL filters pay for themselves — and where hand-maintained sitemaps drift.
Verify canonicals, titles, H1s and status codes on staging, while the site is still nobody's problem but yours.
Heavy-page flagging and response times from the same crawl point at the templates worth optimising, without installing an agent.
Prerequisites
| Requirement | Minimum | Recommended |
|---|---|---|
| Operating system | Windows 10 | Windows 11 |
| Target access | A URL the machine can reach | Include staging hosts that resolve only on your network or VPN |
| Crawl budget | Depth 1–2 for a quick inventory | Depth 3–5 with include and exclude patterns for a real audit |
| Throttling | Any value above the default concurrency | Throttle down for staging, small VPS or shared servers |
| Rendering | On for static HTML sites too, to confirm parity | Always on for SPAs and any client-rendered route |
| Output | Local disk for sitemaps and CSV exports | Version the generated XML in your repository so it is reviewable |
| Privileges | Standard user account | No service, scheduler or background agent is installed |
robots.txt, and never crawl a production site you do not own without permission. The tool gives you the controls; the judgement stays with the operator.Context
| Concern | SiteDirective | Hosted SEO platform | Ad-hoc script |
|---|---|---|---|
| Where crawl data sits | Your machine | Vendor cloud, per retention policy | Your terminal scrollback |
| Cost | Free | Monthly per-project subscription | Engineering hours |
| JavaScript rendering | Full engine, SPA-aware | Usually included in higher tiers | You build it |
| Works against staging or offline | Yes, including localhost | Often blocked by plan tier or public-URL requirement | Yes, if you write it |
| robots.txt authoring | Built-in architect with rule testing | Report-only in most tools | You parse it |
| Scheduling | On demand from the desktop app | Recurring crawls | You own the cron |
| Historical trend dashboards | Not the purpose; export and diff CSVs | Built in | Not the purpose |
Questions
href harvester. A full JavaScript engine executes the page, so client-rendered routes and content appear in the crawl the same way a browser would find them.localhost and hosts that resolve only on your network or VPN. Nothing needs to be publicly accessible first, which is what makes pre-launch auditing possible.Ready to crawl
No cloud processing. No account. No telemetry. Just a high-performance crawler that renders JavaScript, finds broken links and generates compliant sitemaps — all on your machine.
| Version | v2.0.0 |
|---|---|
| Platform | Windows |
| Price | Free |
| Telemetry | 0 bytes |