Handling API Rate Limits in n8n: Retry, Queue, and Batch

Quick Answer

A workflow that hits an API rate limit and isn’t explicitly checking for it can end up continuing as if the call succeeded — particularly if error handling on that node is set to continue rather than stop, or if the downstream logic doesn’t distinguish a 429 response from any other. The result is a workflow that reports success while the actual operation (a charge, an email send, an inventory update) never happened. The fix has three parts: exponential backoff retry logic, request queuing to control throughput, and batching to reduce total call volume in the first place.

Why This Happens at Volume, Not in Testing

A workflow tested with a few dozen sample runs rarely comes close to a real API’s rate limit. The problem shows up when real traffic scales past what testing exercised — an order-processing workflow calling four or five APIs per transaction can go from comfortably under a rate limit at low volume to well over it once request volume multiplies. Every API has a different limit (Stripe, Mailchimp, and most SaaS APIs all publish their own), and a workflow calling several of them per execution can hit the lowest one first without that being obvious from testing alone.

Concurrent executions compound the problem. If your workflow instance runs many executions in parallel, each making several API calls, the aggregate request rate can exceed a limit that any single execution would never approach on its own.

Three Strategies

Strategy 1: Exponential Backoff Retry

A Function node after an HTTP Request node that detects a 429 and retries with increasing delay rather than either failing immediately or silently continuing:

// Check for rate limit response
if ($node["HTTP Request"].json.status === 429 || 
    $input.all()[0].json.error?.type === 'rate_limit_exceeded') {
  
  const maxRetries = 5;
  const baseDelay = 1000; // 1 second
  const retryCount = $executionData.contextData?.retryCount || 0;
  
  if (retryCount < maxRetries) {
    const delay = baseDelay * Math.pow(2, retryCount);
    
    // Wait and retry
    await new Promise(resolve => setTimeout(resolve, delay));
    
    // Increment retry counter
    $executionData.contextData = { 
      ...($executionData.contextData || {}), 
      retryCount: retryCount + 1 
    };
    
    return { retry: true, delay: delay };
  } else {
    throw new Error(`Rate limit exceeded after ${maxRetries} retries`);
  }
}

return $input.all()[0].json;

The important part isn’t just the retry — it’s that a 429 which exhausts its retries throws rather than passing through as if it succeeded. Make sure whatever error handling is configured on the surrounding nodes doesn’t swallow that thrown error and continue anyway.

Strategy 2: Request Queue

Rather than firing calls as fast as the workflow can generate them, queue requests and process them at a controlled rate:

// Queue Manager Function Node
const Redis = require('redis');
const redis = Redis.createClient({ url: 'redis://localhost:6379' });

async function queueRequest(apiName, requestData, priority = 0) {
  const queueKey = `api_queue:${apiName}`;
  const payload = JSON.stringify({
    data: requestData,
    timestamp: Date.now(),
    workflowId: $workflow.id,
    executionId: $execution.id
  });
  
  await redis.zadd(queueKey, priority, payload);
}

// Rate-controlled processor (separate workflow)
async function processQueue(apiName, rateLimit) {
  const queueKey = `api_queue:${apiName}`;
  const rateLimitKey = `rate_limit:${apiName}`;
  
  const currentCount = await redis.get(rateLimitKey) || 0;
  
  if (currentCount < rateLimit) {
    const request = await redis.zpopmin(queueKey, 1);
    if (request.length > 0) {
      await redis.incr(rateLimitKey);
      await redis.expire(rateLimitKey, 60); // Reset every minute
      
      return JSON.parse(request[1]);
    }
  }
  
  return null;
}

Set rateLimit comfortably under the API’s actual published limit, not right at it — a small margin absorbs timing jitter between your queue processor and the API’s own rate-limit window.

Strategy 3: Batching

Where the API supports it, replace many single-item calls with fewer batch calls — this reduces total request count directly rather than just spacing requests out:

// Batch Stripe Charges
const charges = [];
for (let i = 0; i < $input.all().length; i += 100) {
  const batch = $input.all().slice(i, i + 100);
  
  const batchPayload = {
    charges: batch.map(item => ({
      amount: item.json.amount,
      currency: item.json.currency,
      customer: item.json.customer_id,
      description: `Batch charge ${i/100 + 1}`
    }))
  };
  
  charges.push(batchPayload);
}

// Process 1 batch every 2 seconds
for (let batch of charges) {
  await $http.request({
    method: 'POST',
    url: 'https://api.stripe.com/v1/charges/batch',
    headers: { 'Authorization': `Bearer ${$vars.stripeKey}` },
    body: batch
  });
  
  await new Promise(resolve => setTimeout(resolve, 2000));
}

Check the specific API’s documentation for its actual batch endpoint and format before using this pattern — batch support and payload shape vary significantly between providers.

The Real Risk Is Silent Success

The most expensive failure mode here isn’t the rate limit itself — it’s a workflow that hits one and reports success anyway, because nothing downstream checks for it. Before optimizing for throughput, confirm that a rate-limited call actually surfaces as a failure in your workflow’s execution status and in whatever monitoring or alerting you have on top of it. A workflow that’s slow because it’s queuing and retrying correctly is a much better problem to have than one that’s fast because it’s silently dropping failed operations.

Related Reading

See Automation Governance for the broader pattern this fits — exception handling as a first-class design decision, not an afterthought.