Competitor Change Monitor With a Human Review Queue

By Simeon Matheka, Founder & Creative Director · Published 2026-10-01 · Updated 2026-10-01 · 16 min read

Most competitor tracking dies in a bookmarks folder. Build a small monitor that fetches a short list of public pages, diffs them with plain code, lets a language model describe the change, and parks every alert in a queue a person clears.

Someone on the team bookmarks a rival’s pricing page in March. In September a client mentions the rival now offers a free tier, and nobody knew. The information was public the whole time. The failure was that checking it depended on a person remembering to look.

A change monitor fixes the remembering part, and only that part. It does not decide strategy. It tells you a page changed, shows you what changed, and waits for a person. If your team has already written down how it speaks and what it may claim, put that in a shared context pack first. The monitor then has something to compare against.

The whole pipeline on one page

flowchart TD
    A["Schedule: weekly"] --> B["Fetch watchlist pages"]
    B --> C["Clean main content"]
    C --> D{"Changed since last snapshot?"}
    D -->|No| E["Store snapshot, stop"]
    D -->|Yes| F["Text diff (code)"]
    F --> G["Severity rules (code)"]
    G --> H["LLM describes the diff"]
    H --> I["Review queue"]
    I -->|Approve| J["Notify team"]
    I -->|Dismiss| K["Log and learn"]

Count the model steps. One. Everything else is boring code, and that is on purpose. The less you ask the model to do, the less there is to check.

Step 1: write a watchlist you can defend

The watchlist is a table, not a vibe. Each row is one public URL with a reason.

Step 2: fetch politely and cheaply

A weekly fetch of a handful of pages is light, and you can make it lighter. HTTP lets a client send a stored entity tag with the If-None-Match header, and a server that supports it can answer 304 (Not Modified) when nothing changed, which saves both sides the transfer. Many sites do not send usable tags, so treat it as an optimization and still hash the cleaned text yourself. Sourced: RFC 9110, section 13.1.2 (checked 1 October 2026).

• Identify yourself: Send a clear User-Agent with a contact address so a site owner can reach you.
• Respect robots.txt: If a path is disallowed for crawlers, skip it. That is courtesy and good risk hygiene.
• Stay public: No logins, no paywall workarounds, no accounts you made under a false name.
• Rate limit: One request per page per run, with a pause between hosts.
• Keep the snapshot: Store the cleaned text and the fetch date. You need the old version to diff against and to prove what the page said.

Why robots.txt is courtesy and not a guarantee: the Robots Exclusion Protocol (RFC 9309) states it is not a substitute for valid content security measures, and that listed paths are publicly discoverable. Read the site’s terms too. Sourced: RFC 9309, section 3 (checked 1 October 2026).

Step 3: diff with code, then rank by severity

Compare the new cleaned text to the last snapshot, line by line. Any standard diff library does this. Then apply plain rules to decide how loudly to alert. These are rules, not model opinions, so you can read them and change them.

// diffLines: array of added/removed lines from your diff step.
const HIGH = /(\$|€|£|\d+\s?%|free|per month|per user|limit|discontinu)/i;
const MED  = /(new|launch|now includes|beta|integration|plan)/i;

function severity(diffLines, watchFor) {
  const text = diffLines.join(' ');
  if (HIGH.test(text)) return 'high';
  if (watchFor.some((w) => text.toLowerCase().includes(w))) return 'medium';
  if (MED.test(text)) return 'medium';
  return diffLines.length > 40 ? 'medium' : 'low';
}

{
  "alert_id": "2026-w40-rival-pricing",
  "url": "https://rival.example/pricing",
  "fetched_on": "2026-10-01",
  "severity": "high",
  "diff": ["- Starter: $29 / month", "+ Starter: free, limited to 3 projects"],
  "llm_summary": "",
  "status": "pending"
}

Notice that the alert record carries the raw diff. The reviewer always sees the real change next to the model’s sentence. If the two disagree, the diff wins.

Step 4: let the model describe, not decide

Pass the model the diff and the watch_for field. Ask for one to three sentences that describe what changed in plain words. Do not ask what it means for you, whether to respond, or what the competitor is thinking. Those questions invite invention.

You describe a change to a public web page. You do not advise.
You do not guess motives. You do not add facts that are not in the diff.

Page purpose: {{watch_for}}
Diff (removed lines start with -, added with +):
{{diff}}

Write 1 to 3 plain sentences describing exactly what changed.
If the diff is only layout, wording, or formatting, write: "No substantive change."
Return JSON: { "summary": "", "substantive": true|false }

The page text is fetched from someone else’s server, so treat it as untrusted. The Open Worldwide Application Security Project (OWASP) describes prompt injection as inputs that alter a model’s behavior, even inputs invisible to a human reader. This monitor is safer than most because the model has no tools. It cannot send, post, or write. Keep it that way. Sourced: OWASP LLM01:2025 Prompt Injection (checked 1 October 2026). Validate the JSON before it enters the queue, as described in evaluating an AI workflow before production.

Step 5: the human review queue

This is the human-in-the-loop (HITL) pattern from human-in-the-loop review for n8n and LLM jobs, applied to a read-only job. Every alert is a row. A person opens it, reads the diff, and either approves the notification or dismisses it.

Set a service level. High alerts get opened within a working day. Medium and low are batched into one weekly pass. An unreviewed queue is worse than no monitor, because it looks like coverage.

Kill switch and cost cap

• One pause flag: A single setting stops new runs. You should be able to flip it at lunch.
• Run cap: A hard limit on pages fetched and model calls per run. A selector bug that makes a diff 10,000 lines long should hit the cap, not your bill.
• Diff size limit: If a diff passes a line threshold, skip the model, mark it “large change,” and send it straight to a human.
• Failure alert: If a fetch fails twice, alert the owner. Silent failure looks exactly like “no changes.”

Filled hypothetical: a small studio, five rivals

Hypothetical, not a client. A studio watches five rivals, two pages each, ten rows total. Week one, the monitor flags eight changes. Seven are a rotating testimonial and a cookie banner. The studio tightens the selectors and removes the testimonial block. Week two, it flags one change: a rival renamed a package and dropped a line item. A reviewer approves it, and the owner schedules a pricing conversation.

The first week looked like failure. It was tuning. Expect the first two or three runs to teach you what is noise on each page.

What this does not prove

A monitor tells you a public page changed. It does not tell you why, whether the change is real for customers, or what to do. Pages can change without the offer changing, and offers can change without the page changing. It also sees only public pages, so it is silent about sales calls, private discounts, and product behavior.

Print the competitor monitor launch card and tick it before the first scheduled run. When the summaries start feeding briefs, add the source-required research rules. If you want help wiring the queue or the severity rules, start a conversation.

Tags: competitive intelligence, competitor monitoring, change detection, human in the loop, n8n, LLM, workflow design