Technical SEO Audits with AI: A Practical Checklist

A crawler can find thousands of issues without explaining why your most important page disappeared from search. An audit becomes useful when it connects a symptom to a cause you can verify.

Start with a small set of pages that matter: a revenue page losing visits, a new section that has not appeared in search, or a template affected by a release. Diagnose those pages before expanding the crawl. A broken template deserves a different response from one obsolete URL.

Use Claude or ChatGPT with RankParse to gather scoped public-page evidence and turn it into reproducible issues. Supply crawl and Search Console evidence separately where needed. This technical SEO audit checklist follows the path from discovery to a usable page. It helps you answer a specific question at each stage and collect enough evidence to assign a fix. For deciding which confirmed problems deserve budget, use the broader SEO audit prioritization framework.

Give the agent a small, evidence-based audit

Connect RankParse through the Claude or ChatGPT MCP setup. The secure OAuth connector provides read-only tools for public-page checks. This workflow does not require the agent to change your site.

Supply a list of representative URLs, their intended indexability, and the symptom you are investigating. Add a crawler export and relevant Search Console observations if you have them. Do not tell the agent to “audit everything” without defining the scope.

RankParse's get_page_seo can inspect a public URL's fetched title, headings, canonical, robots directives, and link counts. get_site_health adds site-level checks such as redirects and robots.txt; get_sitemap supplies sitemap evidence. Use get_page_performance on representative templates when performance is relevant. These checks are not a comprehensive site crawl, a Google indexing verdict, or proof of Google's rendered view.

Copy and adapt this prompt:

Audit only the attached priority URLs using RankParse and the attached
crawl export. Our symptom is: [describe it]. The inventory states which
URLs should be indexable. Check page SEO for each URL; use sitemap and
site health checks to investigate discovery or access inconsistencies.
Check mobile performance on one representative URL per template.

Return: affected URL or template, observed evidence with source and
collection date, likely cause, missing evidence, proposed fix, and an
acceptance check. Separate observations from hypotheses. Do not infer
Google's selected canonical, rendered content, or index status from
public-page checks. List the manual Search Console/browser checks needed.
Do not change the site or expand to additional URLs without asking.

Review the underlying results for the highest-priority finding. A short description or multiple headings may not explain the symptom. Use the checklist below to decide which observations justify a reproducible issue.

Run the connector in three passes

Prepare an audit packet before collecting results. For a first pass, use six URLs: three affected pages, two healthy pages using the same template, and one page using another template. This is a diagnostic sample, not a sitewide assurance.

Keep unknown values blank and label them unknown. A missing Search Console inspection is not an “excluded” result. Preserve full URLs, including meaningful query parameters; treating every variant as equivalent can erase the bug you are trying to find.

Pass 1: collect facts without assigning fixes

Ask Claude or ChatGPT to run get_page_seo with each exact URL, then retain these Page SEO fields:

Use this follow-up after the opening prompt:

Before writing recommendations, return one evidence row per supplied URL.
Include the requested URL, status_code, final_url, canonical.url,
robots.raw, robots.index, title.text, and collection time. Preserve errors
and nulls. Compare affected pages with controls. Cite the exact returned
field behind every discrepancy. Keep HTTP-header and Google-indexing
questions in a separate evidence-needed column.

The page checker parses server-returned HTML; it does not execute client-side JavaScript. A sparse heading list therefore triggers a browser/rendering check before it triggers a content rewrite.

Pass 2: investigate only the shared discrepancy

If the sample suggests a site access problem, call get_site_health for the bare domain. Its checks array reports probes of the homepage, robots.txt, and sitemap.xml. Inspect null or failed probes before interpreting summary booleans. In particular, robots_txt.disallow_all is a heuristic that can flag a subpath disallow. Read the actual robots.txt rules and applicable user-agent group before filing a sitewide blocking incident.

For a discovery or canonical mismatch, use get_sitemap with the domain and a stated limit. Save sitemap_url, truncated, and child_sitemaps_fetched with the returned URLs. A URL absent from a limited response is not proven absent from all sitemap files. If coverage is incomplete, supply the relevant sitemap file or CMS export instead of asking the agent to infer the missing inventory.

Investigate the shared discrepancy in the evidence table. Choose site
health or sitemap evidence only if it can distinguish our hypotheses.
State that question first. Separate confirmed sample behavior from the
number of potentially affected inventory URLs. If coverage is incomplete,
name the exact file, header, browser check, or Google inspection needed.

Pass 3: create the smallest reproducible ticket

Have the agent turn one confirmed discrepancy into a ticket with the exact URL, reproduction date, expected behavior, observed fields, affected sample, possible scope, owner, and acceptance checks. Reject tickets whose evidence is only a title-length flag or a generic score.

For a solo operator, one table and one ticket are enough to start. On a larger site, group tickets by template and release rather than creating hundreds of identical URL issues. Keep one healthy control in each group. Expand the sample when affected and healthy URLs disagree; that disagreement may expose a conditional rule instead of a shared template defect.

If the connector cannot fetch a page, retain the error code and stop drawing conclusions about that page. An authenticated staging page needs a supplied response or developer inspection. For a public page returning a bot challenge, compare a normal browser response and server logs. A successful audit of the challenge's HTML is not a successful audit of the intended content.

Define what should happen before checking what does

Build an inventory of intended search landing pages. Include the preferred URL, page type, owner, business purpose, and expected indexability. Product pages, category pages, and useful guides might belong in this inventory. Internal search results and account screens usually do not.

Select representative URLs from each important template, including both healthy and affected pages. Add recent redirects, newly published pages, and pages reported as excluded in Search Console. Sampling helps isolate patterns; it does not establish that every other URL is healthy.

Gather four kinds of evidence:

• A crawl for links, response codes, directives, and template patterns.
• Search Console for Google's reported discovery, crawling, indexing, and selected canonical information.
• Browser and rendered-page inspection for what loads and what visitors can use.
• Server logs, where available, for requests actually received, their timing, and response codes.

Record the collection date. A live page, a cached crawl, and Google's last indexed version can describe different moments without any tool being wrong.

1. Can Google discover the important URL?

Find at least one ordinary, crawlable internal link to each priority page. The link should use an anchor element with an href pointing to a resolvable URL. A button that only navigates after a JavaScript event is a weaker discovery path.

Check whether relevant category pages, guides, or navigation expose the page. A URL can be accessible when pasted into a browser and still be absent from your site's link structure.

Compare the crawl against your CMS inventory and XML sitemap. A crawl alone cannot reveal every orphan page because it follows links from its starting points. A page absent from the crawl is a candidate for investigation, not automatic proof that Google has never found it.

Sitemaps should emphasize canonical URLs you want indexed. Remove redirected, missing, and intentionally excluded URLs from that set. A sitemap helps discovery; submission does not guarantee crawling or indexing.

Pass evidence: intended landing pages have relevant crawlable links, and sitemap entries match the preferred URLs. If discovery is the gap, improve the internal linking strategy before requesting repeated indexing checks.

2. Can the crawler fetch the page reliably?

Check the actual HTTP response and redirect destination. A page displaying normally in your browser may behave differently for unauthenticated requests, different locations, or a bot encountering a firewall challenge.

For priority pages, investigate:

• Unexpected 4xx responses, authentication prompts, and access challenges.
• Repeated 5xx responses or timeouts, especially after releases or during load spikes.
• Redirect loops, long chains, and redirects to irrelevant destinations.
• Pages returning 200 while showing an error or almost no useful content.

A genuine missing page should usually return 404 or 410. Redirect a retired URL when a relevant replacement exists; sending every deleted product to the homepage can confuse visitors and may be treated as a soft 404.

Read robots.txt alongside these checks. A disallow rule limits crawling of matching paths. It is not a reliable instruction to remove a URL from search: Google can sometimes know and display a blocked URL without fetching its content.

If a fetch fails only in one testing tool, compare request behavior and logs before changing production rules. Conversely, one successful fetch does not rule out intermittent failures. Google's documentation on HTTP status codes and network errors provides the reference for interpreting responses.

Pass evidence: important URLs return the intended response consistently, and crawling rules allow the access needed for the intended search behavior.

3. Do indexing directives match the page's purpose?

Inspect both the HTML robots meta tag and the HTTP X-Robots-Tag header. The header matters for HTML and for files such as PDFs. Look for directives introduced by staging settings, shared templates, middleware, or document delivery rules.

A noindex directive tells a supporting search engine not to index the resource, but the crawler must be able to access the directive to process it. Blocking a page in robots.txt can prevent Google from seeing a newly added noindex.

Do not remove exclusions merely because an audit tool marks them red. An account page excluded intentionally is behaving correctly. An important category page excluded by a template mistake is not.

When urgent search removal is necessary, Search Console's removal tool can temporarily hide a result. It does not replace a durable removal method. Confidential content needs proper access control; neither robots rules nor indexing directives secure it.

Pass evidence: the fetched directives match the inventory's intended indexability. Save the response or source excerpt in the issue so the developer can reproduce it.

4. Are canonical signals pointing to the right page?

Canonicalization addresses duplicate or very similar content. A canonical annotation identifies the preferred version, but Google can select a different canonical based on its signals. It is not an unconditional command.

For a suspicious page, compare:

Investigate parameter URLs, HTTP and HTTPS variants, trailing slashes, and duplicate category paths. Where alternate URLs serve a valid user purpose, a canonical can help consolidate signals. Where an obsolete URL has been replaced permanently, a relevant redirect may be more appropriate.

Do not canonicalize distinct, useful pages to a broad category simply to reduce the number of indexed URLs. Likewise, later pagination pages often contain different items; canonicalizing them all to page one can obscure their intended role.

Pass evidence: signals agree on the preferred version, and Google's reported choice is understood. A changed canonical may require recrawling and reprocessing before the report changes.

5. Does the rendered page contain the essential content?

Compare the initial HTML with the rendered page. Focus on the main text, product information, navigation, and links required to understand or discover the page.

JavaScript can work in search, but blocked resources, failed requests, delayed rendering, or interactions required to reveal content can create gaps. Essential text that appears only after clicking a button deserves closer inspection. So does a product listing that depends on an API request failing for unauthenticated visitors.

Use Search Console URL Inspection to compare indexed information with a live test. The live test helps diagnose current access and rendering; it does not prove that the URL is indexed or will be selected for indexing. Check the screenshot and available rendered HTML, not only the headline test status.

Test a failing page and a healthy page from the same template. If both contain the main content but only one is excluded, investigate duplication, page value, and canonical selection before blaming JavaScript.

Pass evidence: essential content and links appear in the rendered result without user interaction, and failures have a reproducible cause.

6. Is page experience obstructing the visit?

Once access and indexing intent are correct, examine performance on representative templates. Use field data to understand real visitor experience where enough data exists. Use laboratory tests to reproduce problems and investigate causes. Their numbers need not match.

Look at Largest Contentful Paint for loading, Interaction to Next Paint for responsiveness, and Cumulative Layout Shift for visual stability. Identify which templates share the problem and which changes address it: image delivery, excessive script work, unstable embedded content, or slow server responses.

Do not treat a perfect audit score as the finish line. A fast page can still answer the wrong question, and good Core Web Vitals do not guarantee rankings. Prioritize improvements that remove a real obstacle, especially on pages where visitors complete important tasks.

Pass evidence: a specific cause is improved in repeatable diagnostics, followed by field monitoring when available. Field metrics can take time to reflect a release.

Read the performance response before assigning a speed fix

For the selected template, ask for get_page_performance with the exact url and strategy: mobile. Save cached_at and strategy beside performance_score and the core_web_vitals object. Despite that object's name, these returned values are Lighthouse lab measurements. crux_data: true indicates that field data exists; the response does not contain those field values. A null inp_ms is unavailable evidence, not a zero or a passing result.

Read this RankParse performance response as a lab result. Keep its
strategy and cached_at visible. Identify what it can justify investigating,
then list the diagnostic trace or field report needed to choose a fix.
Do not infer real-user INP from null or from crux_data. Do not name a slow
image or script unless a supplied diagnostic identifies that resource.

Results are cached for 24 hours. An immediate repeat can return the same pre-deployment measurement and still be billed. Use a fresh browser diagnostic for immediate release verification, then compare a later RankParse result with the same URL and strategy. If the tool returns a daily quota error, save the unfinished check for after the reset instead of retrying it in a loop. The performance reference explains caching and quota behavior.

Turn findings into reproducible fixes

Consider this illustrative example, not a customer result: an equipment supplier publishes 180 replacement-part pages. Search Console reports many as excluded, and the team assumes it needs more content.

In this invented packet, get_page_seo returns status_code: 200, the expected final_url, a self-referencing canonical.url, and robots.raw: "noindex,follow" for three affected pages. Healthy controls return robots.raw: "index,follow". Those fields establish a difference in the fetched meta directives. They do not establish its source in the application or Google's current index state.

A developer traces the directive to a preview-derived template. Separately supplied robots.txt and browser checks confirm that crawling is allowed and the part information renders. The CMS inventory identifies 180 pages on that template. The agent can now draft a template ticket without presenting 180 pages as individually tested.

The useful issue is precise:

Affected group: replacement-part pages using the preview-derived template. Evidence: three sampled production responses contain noindex; the template inventory identifies 180 potentially affected URLs. Change: remove the production exclusion from that template while retaining preview access controls. Acceptance check: recrawl the affected inventory and confirm the intended directive state; inspect representative live pages; then monitor Google's indexed reports after recrawling.

Have the agent draft the issue from the sampled responses, with the untested URLs explicitly marked as potentially affected. The team should verify the full affected inventory rather than assuming every page shares the sampled behavior. Removing the directive restores eligibility. It does not guarantee indexing of every part page.

Use the same issue structure elsewhere: affected group, observation, likely cause, reproduction steps, owner, proposed change, acceptance check, and follow-up date. Separate a confirmed cause from a hypothesis that still needs testing.

Close the audit when the evidence changes

After deployment, repeat the checks that demonstrated the defect. Confirm the right URLs changed and inspect a healthy control group for regressions. Keep deployment verification separate from later search outcomes.

For indexing fixes, monitor selected examples and the relevant Search Console groups after Google has had time to revisit them. For performance fixes, compare suitable field periods. For traffic outcomes, account for demand, seasonality, and other releases using a decision-focused SEO report.

Mark the first ticket fixed when its technical acceptance checks pass. Keep unresolved hypotheses visible and report the subsequent search response separately.