Kaburu Wiki

# Divi Builder Bot — SEO Automation Pipeline

Status: ✅ Core components built, testing in progress

Goal: Automate content-to-page workflow for client sites with image generation, SEO meta, and ranking tracking.

—

## Architecture

``` ┌─────────────────────────────────────────────────────────────┐ │ Content Source │ │ (Client brief, keywords, SEO strategy, Matomo data) │ └─────────────────────────────────────────────────────────────┘

                          │
                          ▼

┌─────────────────────────────────────────────────────────────┐ │ SEO Planning Layer │ │ • Keyword research (RankMath searchIntent) │ │ • SEO scoring (RankMath seo-score) │ │ • SEO strategy (robots.txt 3 modes, sitemap) │ │ • SEO score report (Telegram weekly) │ └─────────────────────────────────────────────────────────────┘

                          │
                          ▼

┌─────────────────────────────────────────────────────────────┐ │ Image Generation Layer │ │ • ComfyUI (Juggernaut XL v9, photorealistic) │ │ • Fallback: OpenAI DALL-E 3 │ │ • Script: scripts/comfyui-generate.py │ │ • Output: attachment_id + URL → WP CLI │ └─────────────────────────────────────────────────────────────┘

                          │
                          ▼

┌─────────────────────────────────────────────────────────────┐ │ Page Build Layer │ │ • Page content (markdown-for-agents bot crawl) │ │ • Hero image (ComfyUI/DALL-E) │ │ • SEO meta (RankMath set-meta, set-schema) │ │ • Page publish → WP CLI │ └─────────────────────────────────────────────────────────────┘

                          │
                          ▼

┌─────────────────────────────────────────────────────────────┐ │ SEO Monitoring Layer │ │ • Matomo analytics (AI bot crawls, keywords) │ │ • RankWatch tracking (rankings, movers) │ │ • SEO score report (Telegram weekly) │ └─────────────────────────────────────────────────────────────┘ ```

—

## Components

### 1. ComfyUI Image Generation

Location: `/home/kaburu/.hermes/skills/infrastructure/divi-5-builder/scripts/comfyui-generate.py`

Features: - Photorealistic images via Juggernaut XL v9 (6.7GB, SDXL-based) - Fallback to OpenAI DALL-E 3 when ComfyUI fails - Returns attachment_id + URL for WP CLI - Handles prompt engineering, size, style, seed - Error handling and logging

Status: - ✅ Juggernaut XL v9 downloaded to `/models/comfyui/checkpoints/juggernautXL_v9Rdphoto.safetensors` - ✅ SDXL VAE downloaded to `/models/comfyui/vae/sdxl_vae.safetensors` - ✅ Tested with prompt: “professional water hygiene engineer testing water quality in a modern building, wearing safety equipment” - ✅ Image generated successfully at `/tmp/test-hero-image.png` - ✅ Vision verified: photorealistic quality suitable for hero images

Pipeline Issue: - ❌ SCP upload blocked: cannot route directly from kaburuaibox to Hetzner (MCP limitation) - 🔧 Fix needed: use MCP routing instead of direct SSH

—

### 2. RankMath SEO Automation

Location: `/home/kaburu/.hermes/skills/infrastructure/divi-5-builder/scripts/rankmath-seo.py`

Commands: ```bash # Set meta for a page (focus keyword, title, description, OG tags) python rankmath-seo.py set-meta \

1. -site l8.kaburu.co.uk \
2. -slug test-water-quality \
3. -focus-keyword “water hygiene testing” \
4. -title “Professional Water Hygiene Testing | L8 Water Hygiene” \
5. -description “Expert water quality testing services for buildings and industrial facilities.”

# Keyword research (search intent via RankMath API) python rankmath-seo.py keyword-research \

1. -site l8.kaburu.co.uk \
2. -focus-keyword “legionella risk assessment”

# Set structured data (Article, FAQ, Service, LocalBusiness) python rankmath-seo.py set-schema \

1. -site l8.kaburu.co.uk \
2. -slug test-water-quality \
3. -type service

# SEO score (pull RankMath score for a page) python rankmath-seo.py seo-score \

1. -site l8.kaburu.co.uk \
2. -slug test-water-quality

# Orphans (find pages with no internal links) python rankmath-seo.py orphans \

1. -site l8.kaburu.co.uk

# Robots.txt (3 modes: allow-all, allow-ai-search, block-training) python rankmath-seo.py robots \

1. -site l8.kaburu.co.uk \
2. -mode ai-search

# Full SEO setup (after page build) python rankmath-seo.py full-seo \

1. -site l8.kaburu.co.uk \
2. -slug new-page \
3. -focus-keyword “water hygiene testing” \
4. -title “Professional Water Hygiene Testing | L8 Water Hygiene” \
5. -description “Expert water quality testing services for buildings and industrial facilities.”

```

Status: - ✅ RankMath installed on `l8.kaburu.co.uk` (WP-CLI, v1.5.0) - ✅ RankMath installed on `kaburu.co` - ✅ Application passwords created: `kaburu:qPAxzp9fegmvRE3yKRDO6LNq` - ✅ RankMath REST API confirmed working:

1. `searchIntent` endpoint returns “Informational” for “water hygiene testing”
2. API authentication working

- ✅ SEO automation script built (497 lines)

Pending: - ⚠️ RankMath Pro on l8.kaburu.co.uk — free version installed, Pro needs manual download from RankWatch account - ⚠️ Full SEO pipeline testing with real page builds

—

### 3. RankWatch API Integration

Location: `/home/kaburu/.hermes/skills/infrastructure/divi-5-builder/scripts/rankwatch.py`

Features: - List all tracked projects - Show keyword rankings (desktop/mobile) - Add new projects to track - Add keywords to track - Web stats history - Weekly ranking report (Telegram)

API Details: - Base URL: `https://apiv2.rankwatch.com/{path}/{json}/{params}/` - Auth: Basic HTTP auth (`token:password`) - Token: `rw-v2-82758-17299` - Password: `MD4g$sL(SWd#`

Usage: ```bash # List all projects python rankwatch.py projects

# Show rankings for l8.kaburu.co.uk python rankwatch.py rankings –site l8waterhygiene.co.uk python rankwatch.py rankings –site l8waterhygiene.co.uk –mobile

# Add a new keyword to track python rankwatch.py add-keyword –site l8waterhygiene.co.uk –keyword “legionella risk assessment”

# Add a new project (onboarding new client) python rankwatch.py add-project –domain newsite.co.uk –name “New Client” –se 2.00851,2.00861

# Weekly ranking report (all sites) python rankwatch.py report –all ```

Status: - ✅ API authentication working - ✅ Project list fetched successfully - ✅ Rankings API confirmed working: `project/detail/json/p_id/{pid}/s_id/{se}` - ✅ Script built (16,326 bytes, 497 lines) - ✅ Tested rankings for l8waterhygiene.co.uk

Pending: - ⚠️ Full weekly report automation (Telegram delivery) - ⚠️ Integration with page publish flow (add keyword after page creation)

—

### 4. Markdown for Agents

Status: - ✅ Installed on `l8.kaburu.co.uk` (WP-CLI, v1.5.0) - ✅ Already active on `rainbowvapes.co.uk` (v1.5.0) - ✅ Scripted robots.txt strategies in rankmath-seo.py (3 modes):

1. `allow-all`: All crawlers allowed
2. `ai-search`: Only AI search/crawlers allowed, training blocked
3. `block-training`: Only Google/ Bing allowed, AI training blocked

Rainbowvapes Current Strategy: - Cloudflare Managed Content blocking ALL AI crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, meta-externalagent)

Pending: - ⚠️ Deploy robots.txt strategy to rainbowvapes (shift from block-all to ai-search) - ⚠️ Deploy to test sites (myretonmarquees, jafricasafari.com)

—

### 5. Matomo Analytics

Status: - ✅ Matomo MCP connected (18 sites tracked) - ✅ BotTracking module available (AI bot crawls, AI agents, AI assistants) - ✅ All report types available (VisitsSummary, Actions, Referrers, Goals, etc.) - ❌ Zero visits in last 30 days for all sites — tracking may be dead

Likely Causes: 1. Matomo archive cron (`core:archive`) is stale 2. Tracking codes not firing (Cloudflare or LiteSpeed caching stripping JS) 3. Sites not actually tracked (tracking code missing from newer pages)

What This Enables: - AI Bot Visibility Report (weekly Telegram report)

1. Which AI bots crawled each site
2. Which pages they accessed most
3. Compare to robots.txt strategy compliance

- Keyword Performance Feed

1. Actual search keywords from Matomo → feed into RankMath focus keywords

- Page Performance → Content Decisions

1. High bounce rate / low time-on-page → flag for content refresh
2. Good engagement → use as templates for new builds

- Cross-Site Dashboard

1. Compare all client sites in one report for monthly retainer work

Pending: - ⚠️ Fix Matomo tracking codes / archive cron - ⚠️ Add `l8.kaburu.co.uk` to Matomo (not in tracked sites yet)

—

## Scripts

—

## Sites

—

## Next Steps (Tomorrow)

1. Fix SCP upload pipeline — use MCP routing instead of direct SSH from kaburuaibox to Hetzner 2. Fix Matomo tracking — check archive cron and tracking codes 3. Add l8.kaburu.co.uk to Matomo — not in tracked sites yet 4. Deploy markdown-for-agents robots.txt — shift rainbowvapes from block-all to ai-search 5. Test full content-to-page pipeline — copy → image → SEO meta → page build 6. Build keyword research helper — RankMath searchIntent + web search 7. Implement bot crawling encouragement — robots.txt, sitemap, markdown-for-agents testing 8. Build weekly Telegram SEO report — Matomo AI crawls + RankWatch movers + RankMath scores

—

## Credentials

- ComfyUI API: `https://kaburuaibox:8188` - RankMath API: `kaburu:qPAxzp9fegmvRE3yKRDO6LNq` - RankWatch API: `rw-v2-82758-17299` : `MD4g$sL(SWd#` - WP-CLI paths:

1. `/home/kaburu.co/public_html` (kaburu.co)
2. `/home/l8.kaburu.co.uk/public_html` (l8.kaburu.co.uk)

- SSH keys:

1. rainbowvapes: `~/.ssh/rainbowvapes` (on hetzner server)

—

## Dependencies

- ComfyUI: v0.24.0, Juggernaut XL v9, SDXL VAE - RankMath: v1.5.0 (Pro pending) - RankWatch: L or above plan (API access confirmed) - Matomo: MCP connected (tracking issue) - Markdown for Agents: v1.5.0

—

## Related Wiki Pages

- `/home/kaburu/wiki/wiki/infrastructure/comfyui.md` — ComfyUI setup - `/home/kaburu/wiki/wiki/projects/divi-5-builder-bot.md` — This page - `/home/kaburu/wiki/wiki/infrastructure/hermes.md` — Hermes config - `/home/kaburu/wiki/wiki/clients/rainbowvapes.md` — rainbowvapes client context - `/home/kaburu/wiki/wiki/clients/rainbowvapes-seo.md` — rainbowvapes SEO context - `/home/kaburu/wiki/wiki/services/matomo.md` — Matomo setup