A GEO monitoring program should help the firm answer three questions: Are we appearing in relevant situations? Are we being described accurately? Are the recommendations appropriate for the prospective client? Each question requires evidence that the team can inspect and compare over time.
Chapter 5 established a baseline. This chapter turns that initial audit into a recurring operating process: a stable set of questions, a collection schedule, defined measures, validated tools, and a reporting routine with named owners.
The objective is accurate, appropriate consideration of the firm. A favorable recommendation for a service the firm does not provide should trigger a review. It should not count as a successful outcome merely because the name appeared.
The sampling rules and metric definitions below are proposed operating conventions for this playbook, not universal GEO standards. All examples and sample results are hypothetical.
6.1 Design a Stable Prompt Panel Covering Priority Practices, Clients, and Markets
A prompt panel is the defined set of cases that you repeat to track change. Draw it from the priority practices and markets in Chapter 2, the question library in Chapter 3, and the findings from the baseline audit. Select questions because they matter to the firm’s intended clients and referrers.
Keep a core panel for recurring comparisons and an exploratory set for new questions, unusual results, and investigations. Report them separately. Otherwise, adding several difficult discovery questions can make performance appear to deteriorate even when answers to the original questions have improved.
Define cases precisely
A case includes the exact prompt and any prescribed preceding conversation or supplied material. Give it an identifier and version. Record the intended audience, buying task, practice, market, language, and suitability requirements. Identify whether the question came from an actual inquiry or is a practitioner hypothesis.
Assign a primary question type: unbranded discovery, named-firm evaluation, named-professional evaluation, or comparison. Use additional tags for service boundaries, geography, industry, and follow-up conversations. Keep discovery results separate from questions that already supply the firm’s name.
Consulting example. A practice helping investors assess manufacturing operations might track: “Which consulting firms provide operational due diligence for midmarket manufacturing acquisitions in the UK?” A separate named-firm case asks whether the practice handles post-acquisition implementation. Another case tests a service it explicitly excludes. Those cases examine discovery, understanding, and suitability.
Define what happens when an assistant asks for clarification. In a first-turn case, the clarifying question is the result. In a conversational case, specify the next response in advance. An operator who supplies different budgets, locations, or hints on each run is changing the case.
Choose coverage the team can maintain
Map cases against the few dimensions that drive provider selection: service, client type, industry, geography, jurisdiction, or engagement size. Identify material gaps. You do not need every possible combination, but you should know which priority situations the panel covers and which it leaves untested.
Retain cases where the firm is unsuitable. A law firm should be able to distinguish a correct statement of its jurisdictional limits from an unjustified exclusion. An accounting practice should detect recommendations for an assurance service it does not offer. These are tests of appropriate representation.
For a first recurring program, 24 well-chosen cases can be a workable planning assumption. Reduce that number if the firm cannot review the results. A deliberately selected panel measures performance on those questions; it does not establish the proportion of all prospective clients who encounter the firm.
Review coverage quarterly and after a material change in strategy. Additions and removals require a reason, a new panel version, and an effective date. Keep new cases outside the historical trend until you can show a comparable series.
6.2 Set Sampling Frequency, Repeated Runs, and Rules for Changing the Panel
Set the schedule before looking at the results. Anthropic’s evaluation guidance distinguishes a defined task from repeated trials because model outputs vary, and emphasizes clear grading and human calibration. These principles are useful here, although they do not prescribe a GEO sample size or reporting frequency.
A practical starting schedule is a monthly full-panel collection, with three planned runs per case and experience spread across scheduled collection blocks. Add a small weekly check for priority credentials, affiliations, or service boundaries. Keep those additional checks identifiable so they do not change the weighting of the monthly panel.
For 24 cases across three specified experiences, three repetitions produce 216 planned observations per monthly cycle. A second geographic condition doubles that workload. This is a capacity estimate, not a research-established minimum. Start smaller when review time is the constraint.
Use the general planning formula: cases × experiences × geographic or language conditions × repetitions × collection cycles. Count only conditions you actually intend to test. Additional diagnostic checks and technical retries need their own allowance.
A tool’s daily collection schedule does not dictate the management review schedule. If it collects more frequently, specify in advance which observations feed each reported measure. Keep additional monitoring identifiable and avoid selecting favorable days after seeing the answers.
Separate outcomes from collection problems
Retain every scheduled run slot and assign its outcome: substantive answer, clarifying question, genuine decline, no AI feature displayed, technical or capture failure, or not attempted. A successfully observed Google results page without an AI Overview is a valid observation of feature absence. It is different from a failed capture.
Declare a limited retry policy for technical failures. Preserve the failed attempt and link its recovery to the same scheduled slot. A recovered slot supplies one usable observation, not two independent successes. An unfavorable answer, clarification, or decline does not justify replacing that observation with another attempt.
Before reporting, check planned versus completed collection, missing source evidence, unresolved entity matches, and the human-review backlog. State how much of the intended panel is usable. If an entire experience failed to collect, its absence must remain visible in the report.
Preserve comparability over time
Record changes to prompts, comparison firms, location settings, interfaces, collection methods, classifiers, and metric definitions. A revised classifier can change the score of an unchanged answer. Recalculate prior records consistently where practical; otherwise annotate a break in the series.
When replacing cases, use an overlap period where feasible. Report the common cases as the comparable trend and new cases as a separate group. Do not splice a redesigned panel onto the old series without identifying the change.
Repeated runs describe variation under the conditions observed. They do not automatically create independent observations or a representative survey of buyers. Avoid attaching a conventional survey margin of error to a deliberately selected prompt panel without a justified statistical model.
Set frequency according to decisions the firm can make. A practice changing its service scope may need more frequent checks during the transition. A stable practice may learn enough from monthly reporting. Review a potentially serious false credential or affiliation promptly, even if it appears only once. Questions about whether an intervention caused a change belong in Chapter 17.
6.3 Define Mention, Citation, Recommendation, Accuracy, Favorability, and Sample-Based Share Metrics
Write a metric dictionary before accepting a dashboard. For each measure, specify what counts, the denominator, applicable cases, treatment of unknowns, aggregation method, and reviewer. Keep the dictionary version with every report.
For mention, owned-site citation, and recommendation rates, the basic denominator is completed AI text responses in the stated group, including genuine clarifying questions and declines. Count an event if it occurs; otherwise count no occurrence. Technical failures and absent AI features are reported separately. Recommendation analysis uses cases designated in advance as asking about provider selection or fit.
Do not turn incomplete evidence into a zero. If a citation panel was not captured, citation status is unknown. If the name could refer to another organization, resolve the identity before finalizing its classification. Until required classifications are complete, show known counts and unresolved records instead of presenting a final rate.
Measure | Operational definition |
|---|---|
Firm mention rate | Responses identifying the firm ÷ completed responses. Count the firm once per response. Track individual professionals separately, including their stated affiliations. |
Owned-site citation rate | Responses displaying at least one citation or source link to the approved owned-domain set ÷ completed responses. Count once per response; inspect whether the cited page supports the relevant claim. |
Recommendation rate | Responses offering the firm as a provider to consider ÷ completed responses in eligible cases. Distinguish explicit and conditional recommendations. Merely naming or rejecting the firm does not qualify. |
Accuracy assessment | Count material factual claims about the tracked firm or professional as supported, contradicted, or unverified against the reference evidence. Count the same assertion once within an answer; recurrence in another answer is another observation. |
Confirmed material-error incidence | Reviewed responses with at least one verified material factual error about the tracked firm or professional ÷ reviewed responses containing material factual claims about that entity, within the stated review set. Show unverified claims and unreviewed records alongside it. |
Favorability distribution | Favorable, neutral, unfavorable, or mixed characterizations among reviewed responses that characterize the firm. No characterization is N/A. Unreviewed is a separate status. |
Sample share of firm appearances | Own firm’s appearances ÷ all appearances for a fixed, disclosed comparison set, with any case weighting stated. Count each tracked firm once per response; include your firm. |
For a named professional, calculate mention rate within that person’s defined case group. Record whether the stated affiliation is current, outdated, absent, or unresolved. A correct mention of an individual does not automatically establish a correct mention of the current firm.
Pair recommendations with suitability
For each assessed recommendation, record whether it fits known requirements, is conditional on a stated requirement, fails a verified requirement, or has insufficient information to establish fit. Current availability and client conflicts may remain unknown. Keep those unknowns explicit.
An assistant can be favorable, factually wrong, and unsuitable in the same answer. Conversely, an accurate statement that a firm does not handle a particular matter can be useful. Negative subject matter, such as litigation or financial distress, does not by itself make the description of the firm unfavorable.
For accuracy, keep claim counts and answer counts distinct. One answer containing five supported statements and one false professional credential still contains a material error. An unverified claim is neither confirmed false nor confirmed accurate. A response with no relevant factual claim has no applicable accuracy assessment.
Use a worked example to check the definitions
An accounting practice tests two unbranded provider-selection cases four times each in one specified experience. All eight runs produce completed responses: six substantive answers, one clarification, and one decline. The firm appears in four, its website is cited in two, and it is recommended in three.
The mention rate is 4/8 = 50%; the owned-site citation rate is 2/8 = 25%; the recommendation rate is 3/8 = 37.5%. Two recommendations fit known requirements. The third incorrectly attributes an excluded audit service to the practice. That third recommendation belongs in the raw recommendation count and in the unsuitable-recommendation record.
Across the same eight answers, suppose Competitor B appears in five and Competitor C in three. With those three firms as the fixed comparison set, there are 4 + 5 + 3 = 12 firm–response pairs. The practice’s sample share is 4/12 = 33.3%. Its mention rate remains 50%. The measures have different denominators.
That share describes the collected answers and named comparison set. It is not market share, search demand, unique users, leads, or revenue. Keep other observed providers in the evidence, but do not silently add them to the comparison denominator.
Make aggregation explicit
For the main visibility trend, calculate each case’s rate within a defined experience and segment, then average those case rates equally. Alternatively, declare fixed weights reflecting business priorities. Describe them as business-priority weights unless actual demand evidence supports another interpretation.
Suppose Case A produces two mentions in two responses and Case B none in eight. Pooling runs gives 2/10 = 20%; averaging case rates gives (100% + 0%)/2 = 50%. Both calculations are correct. Equal case weighting prevents extra runs from silently increasing one question’s importance.
For weighted share, calculate each tracked firm’s per-case mention rate, multiply by the case weight, and sum across the same cases. Divide your firm’s sum by the sum for all firms in the fixed comparison set. If the denominator is zero, report N/A.
A case with no usable responses has no observed rate. Report it as missing and compare an explicitly identified common set, or defer the aggregate. Avoid filling gaps with zero or carrying forward the previous score.
For AI Overviews, show feature appearance alongside conditional mention rates. If 20 observed search pages produce 12 overviews and six mention the firm, overview appearance is 12/20 = 60%; mention among overviews is 6/12 = 50%; mention per observed search page is 6/20 = 30%. Reporting only 50% would conceal feature absence.
6.4 Keep Consumer Interfaces, APIs, Platforms, and Search Modes Distinguishable
Name the experience precisely in every series: product, interface, search behavior, and collection method. “ChatGPT visibility” is an incomplete label if some records come from a consumer browser session and others from a configured API request.
A useful record distinguishes a consumer application, a vendor’s collection of that application, and an API configuration. It also distinguishes Google AI Mode from AI Overviews and Gemini Apps. Keep each selected experience identifiable before considering any combined management view.
Record the visible model or model identifier where available, date and time zone, language, location method, account or session conditions, memory settings, supplied context, and whether search was requested or observed. Mark unavailable details as unknown. Opening a new conversation does not by itself establish that all personalization has been removed: OpenAI documents that memory can influence search-query rewriting and that location can affect local results.
APIs can provide controlled, repeatable configurations for testing. Their settings still matter. OpenAI’s web-search API, for example, supports domain filtering and approximate location settings for supported uses. Its retrieved-source list is distinct from the inline citations displayed in an answer. Record those distinctions rather than treating every retrieved URL as a visible citation.
Do not assume that an API result reproduces a client-facing product merely because both use the same provider or model name. Equally, browser collection observes an interface under particular conditions; it cannot establish what every account, location, or conversation receives. Ask the supplier to document the conditions it controls and those it cannot observe.
Law firm example. A test asks for counsel on an employment dispute in Ontario. A location phrase in the question, an account located in London, and an API configured for Toronto are three separate pieces of context. Record them. If the firm changes those conditions, compare the resulting series under their own labels.
Use occasional manual checks in the named consumer experience to investigate collection behavior. Look for systematic differences in prompt wording, search use, location, answer completeness, or citation capture. Exact answer identity is not a sensible universal acceptance test for a variable system.
Apply the version and comparability rules in Section 6.2 whenever a provider or supplier changes the collection method.
6.5 Select and Validate Tools for Answer Monitoring, Source Analysis, Crawl Diagnostics, and Referral Tracking
Build a measurement stack around four kinds of evidence: answers to your test questions, provider reports about your site’s visibility, crawler and access records, and attributed website visits. Each answers a different question. Keep its unit and coverage visible.
Answer monitoring and source analysis
A small program can begin with scheduled manual collection and a structured register. Software becomes useful when repeat collection, source inspection, classification, and reporting exceed the team’s capacity. Five representative offerings illustrate the choices; these are descriptions of documented capabilities, not endorsements or evidence of improved outcomes.
Profound offers custom-prompt monitoring of brand presence, sentiment, competitors, citations, and claims flagged as inaccurate. It says it collects from consumer browser experiences. Validate the specific collection conditions and the evidence behind its scores.
Peec AI focuses on AI-search visibility, position, sentiment, and source analysis, with daily prompt tracking and suggested actions. Its documentation describes the UI collection. Check the meaning of each metric and the coverage relevant to the firm’s markets.
OtterlyAI tracks mentions and citation links, collects tracked prompts daily, and permits inspection of individual answers. Its documentation describes public-interface collection while explicitly identifying Claude tracking as API-based—an example of why collection methods must be checked by experience.
Semrush Prompt Tracking monitors a custom prompt set, with answer snapshots, citation analysis, and comparisons alongside traditional search. Inspect its position definitions: citation placement or result-element placement does not automatically represent the strength of a provider recommendation.
AirOps connects daily prompt monitoring, answer and citation inspection, and answer exports with content work. Its Opportunities feature proposes creation, refresh, outreach, and community activities. Evaluate the monitoring evidence and the firm’s review controls for those workflows separately.
Use broad vendor prompt databases to explore topics and competitors. Require the firm’s recurring cases to remain identifiable. A large database does not, by itself, establish coverage of a specialized practice’s buyers. For cited sources, inspect the underlying page, its date, the passage relevant to the claim, and whether it actually supports the answer.
Provider reports
As of October 2026, Google Search Console’s Generative AI performance report for Search reports impressions for AI Overviews and AI Mode, with dimensions including page, country, date, and device. Those data are also included in overall Web performance data; do not add the two totals. The documented report does not supply the original answer text, a favorability assessment, or a separate click metric. Recent data may be preliminary, and unavailable values can become zeros in exports, so preserve the distinction between missing data and observed zero.
Bing Webmaster Tools AI Performance reports citation activity across Microsoft Copilot, Bing AI summaries, and selected partner integrations. Its grounding queries are retrieval phrases associated with cited material, not necessarily the user’s original question.
Bing’s expanded preview includes Intents, Topics, Citation Share, and Compare. Its Citation Share measures relative citation presence for a grounding query. Keep that provider-defined measure distinct from the sample share of firm appearances defined in this chapter; it is not a ranking, traffic share, or content-quality score.
Access records and website visits
Server and content-delivery-network logs help establish which URLs were requested and whether access succeeded. Bot purposes also differ: OpenAI distinguishes search crawling from crawling for possible model training, while noting that a crawl can serve both uses when both are allowed. A logged request alone does not prove citation, recommendation, or inclusion in training. Chapter 10 addresses access configuration.
Google Analytics now documents an AI Assistant default channel for recognized assistant referrals. Google AI Overviews and AI Mode remain included under Organic Search. Check the actual source, medium, channel definitions, and reporting dates in the firm’s property before constructing a trend.
These analytics identify attributed visits, not everyone influenced by an AI answer. Google describes direct/none traffic as lacking a clear referral source; do not automatically assign it to AI. Retain those limits when connecting monitoring to the commercial analysis in Chapter 18.
6.6 Evaluate Vendor Methods, Human Review, Exports, Privacy, Costs, Dashboards, and Alerts
Buy against a written monitoring specification. Require a trial with the firm’s cases and a sample export. A demonstration should show whether the proposed arrangement can collect, explain, and preserve the evidence the program needs.
Run an acceptance test
- Load meaningful cases. Include discovery, named-firm and named-professional questions, a geographic or jurisdictional condition, a service exclusion, and a name that could be confused with another entity.
- Inspect the collection. Confirm exact prompts and any added persona, location, or brand instructions. Check the full answer, source links, timestamp, experience, search behavior, and handling of incomplete results and retries.
- Check the scoring. Have a practitioner review entity matching, recommendation status, factual support, suitability, and favorability. Test an unfavorable result and an omission as well as a favorable example. Record disagreements and the correction process.
- Export and reconstruct. Recalculate at least one headline measure from exported records. Check case and run IDs, panel version, full answer text, citation URLs, conditions, statuses, scoring version, and review decisions. Verify that filters and date ranges select the intended records.
- Test continuity. Add a prompt version or revise a firm alias and examine the effect on history. Establish what remains available after cancellation and how changes in collection or scoring methods are disclosed.
Use the same evidence standard for an agency’s report. A PDF dashboard may be useful for management, but the team also needs access to the records behind the numbers. If a supplier cannot disclose a method, describe the resulting limitation and decide whether the measure is usable.
Keep practitioner review in the process
Automated classification can help process volume. Calibrate it against human judgments on the firm’s actual cases, especially ambiguous names, professional affiliations, service exclusions, and conditional recommendations. Anthropic’s evaluation guidance explicitly identifies human calibration as necessary for model-based grading.
Choose a routine review sample before inspecting the outcomes, with coverage across experiences and question types. Review potential material errors additionally. Keep rates from the routine sample separate from the extra cases selected because a problem was suspected. Otherwise, the review process itself changes the apparent error rate.
Record reviewer, date, evidence, decision, and unresolved questions. Have a second practitioner resolve material disagreements. A marketing operator can flag a possible false credential; the appropriate professional should verify it before the firm treats it as established.
Evaluate the whole operating cost
Ask what a quoted prompt, check, or credit buys. Determine how additional experiences, markets, repetitions, retries, history, exports, users, and content workflows affect cost. Budget practitioner review and administration as well as software. More collected answers are useful only when the team can interpret them.
Apply Chapter 4’s information rules to the vendor arrangement: permitted inputs, retention, access, subprocessors, deletion, and export rights. Use approved public or hypothetical cases. Confirm contractual treatment of submitted material and how collection respects each platform’s access rules and usage limits. Monitoring approval does not automatically authorize a connected tool to publish content or contact outside parties.
Design reporting around decisions
Give the dashboard four views: collection health with missing and unreviewed records; visibility by case group and experience; accuracy and suitability with evidence-backed issues; and an action register with owner, next step, and due date. Show panel version, period, denominators, review coverage, and method changes wherever a trend is presented.
Use three alert classes. Collection failures go to the operator. Sustained movement on a comparable panel goes to the program owner for investigation. Potentially material misrepresentation goes promptly to the responsible practitioner. Set movement thresholds after observing ordinary variation, and require inspection of the saved answers before escalating a numerical alert.
The monthly review should decide what needs verification, what warrants an improvement project, and what should simply continue to be observed. Record the evidence and uncertainty behind each decision. Chapter 7 uses those findings to prioritize the work.
Practical Output: Approve the Monitoring Specification
Finish this chapter with four working documents. They can be tabs in one register rather than separate systems.
- Monitoring specification. Record the core cases and version, selected experiences and conditions, schedule, repetitions, retry policy, and owners. A starting design might use 24 cases, three experiences, and three monthly repetitions, plus separately reported weekly priority checks.
- Metric dictionary. Adopt or modify the definitions in Section 6.3. State eligible cases, denominators, unknown statuses, comparison firms, weighting, review coverage, and version-change rules.
- Tool evaluation. Record the acceptance-test results, sample-export findings, unresolved limitations, privacy terms, total operating cost, and approval decision for the chosen arrangement.
- Dashboard and alert design. Specify the four reporting views, who reviews each, which events trigger investigation, and who owns the next action. Schedule the first monthly decision meeting.
The completion test is straightforward: another competent person should be able to run the prescribed cases, reproduce a reported measure from the saved evidence, identify what remains unknown, and route a material finding to its owner. That is the foundation of a monitoring program the firm can use.
