Self-Healing Batch Processing Explained

Server environments vary dramatically. A shared hosting plan on a budget provider handles load very differently than a dedicated VPS with 8 cores and 512MB+ of PHP memory. Migration tools that use fixed batch sizes inevitably fail in one of two ways: they're too aggressive for weak servers (causing timeouts) or too conservative for strong servers (wasting capacity). The v1.5.0 release introduced self-optimizing batch processing to solve this.

Adaptive Batch Sizing

Ecwid2Woo starts with a default batch size of 100 items per batch for both products and categories. As batches complete, the plugin monitors for errors and timeouts. If timeouts or HTTP errors occur (Cloudflare 524, Gateway 504, Request 408, or jQuery timeouts), the batch size is automatically halved.

The Self-Healing Part

What makes this bidirectional is the recovery mechanism. After the batch size decreases due to errors, the system doesn't stay at the reduced size forever. After 5 consecutive successful batches, the batch size increases by 50%. For example: 100 → [timeout] → 50 → [5 successes] → 75 → [5 successes] → 100. The system fully recovers to optimal throughput once conditions stabilize.

This means the system adapts to temporary load spikes — another plugin running a cron job, a traffic surge on the site, or a brief hosting provider issue — without permanently degrading performance. When conditions improve, throughput recovers automatically.

Server Intelligence

The plugin includes auto server detection that identifies your server tier based on available memory. High-end servers (512MB+) get aggressive defaults. Medium servers (256-512MB) get balanced settings. Low-resource servers (<256MB) get conservative settings. This tier system is fully automatic.

Progressive Cooldowns

When server errors do occur, the plugin uses progressive cooldown periods. Recovery waits increase with consecutive timeouts: 5 seconds after the first, 10 seconds after the second, up to 30 seconds. For severe server errors (Cloudflare 520-530 codes), extended cooldowns of 30-120 seconds are applied. Up to 5 retry attempts are made with batch size reduction before the operation gives up on that batch.

Memory-Aware Processing

The batch processor is memory-aware. If PHP detects less than 128MB of free memory, the batch size is automatically reduced. Combined with the processing-in-discrete-batches approach that releases memory between batches, this prevents PHP memory limit errors even when processing thousands of products with large image galleries.

All sync pages — Full Sync, Product Sync, and Category Sync — use identical adaptive logic. The current batch size is displayed in the status area so you can see the system adjusting in real time. Zero configuration is required.