Blog SEO Automation in 2026: Finishing Initial Search Awareness with IndexNow Scripts

2026 Blog SEO Automation: Finishing Initial Search Awareness with an IndexNow Script

I’ve been thinking a lot about how to automate the initial search engine awareness process when publishing new posts or when a large number of posts have accumulated on an existing blog. To boost a blog's initial SEO performance, it’s crucial to leverage the IndexNow API by submitting sitemaps to search engines. However, manually entering URLs one by one is incredibly inefficient.

Attempts and Pitfalls

I started by fetching all published blog post slugs from the database to create a complete URL list. Then, I needed to write a Python script to ping this list to the IndexNow API in batches. I decided to use asyncio for asynchronous processing and to send URLs in chunks of 100 for efficiency.

# riel_backend/scripts/indexnow_seed.py (partial)
import asyncio
from typing import List
from riel_backend.services import indexnow_service
from riel_backend.database import session_scope, Blog

async def ping_urls_in_chunks(urls: List[str], chunk_size: int = 100):
    tasks = []
    for i in range(0, len(urls), chunk_size):
        chunk = urls[i:i + chunk_size]
        tasks.append(indexnow_service.ping_urls(chunk))
    await asyncio.gather(*tasks)

async def main():
    async with session_scope() as session:
        slugs = await Blog.get_published_slugs(session)
        urls = [f"https://{os.environ['INDEXNOW_HOST']}/blog/{slug}" for slug in slugs]
        await ping_urls_in_chunks(urls)

if __name__ == "__main__":
    asyncio.run(main())

Initially, I tried sending all URLs at once, but the API responses were too slow, and I occasionally encountered timeouts. I was unsure if it was an issue with asynchronous processing or simply too many requests. After about 3 hours of struggling, I realized that I wasn't correctly using asyncio.gather, which prevented parallel execution.

The Cause

Ultimately, the problem stemmed from not properly managing the tasks list when calling the ping_urls function for each chunk, even though I was using asyncio.gather. I should have called a separate asyncio.gather for each chunk, but I failed to consolidate them into a single asyncio.gather call.

The Solution

I modified the riel_backend/scripts/indexnow_seed.py script to query for the slugs of blog posts with a published status from the database and then generated a list of URLs in the format https://{INDEXNOW_HOST}/blog/{slug}. I changed the code to divide the generated URL list into chunks of 100 and asynchronously call the services.indexnow_service.ping_urls function to ping the IndexNow API. The revised code now prints the result of each chunk processing, summarizes the total number of successful chunks, and ensures the database connection is closed after the operation.

# riel_backend/scripts/indexnow_seed.py (modified)
import asyncio
import os
from typing import List
from riel_backend.services import indexnow_service
from riel_backend.database import session_scope, Blog

async def ping_urls_in_chunks(urls: List[str], chunk_size: int = 100):
    total_chunks = (len(urls) + chunk_size - 1) // chunk_size
    successful_chunks = 0
    for i in range(0, len(urls), chunk_size):
        chunk = urls[i:i + chunk_size]
        try:
            await indexnow_service.ping_urls(chunk)
            print(f"Successfully pinged chunk {i//chunk_size + 1}/{total_chunks}")
            successful_chunks += 1
        except Exception as e:
            print(f"Error pinging chunk {i//chunk_size + 1}/{total_chunks}: {e}")
    return successful_chunks

async def main():
    async with session_scope() as session:
        slugs = await Blog.get_published_slugs(session)
        urls = [f"https://{os.environ['INDEXNOW_HOST']}/blog/{slug}" for slug in slugs]
        print(f"Found {len(urls)} URLs to ping.")
        successful_chunks = await ping_urls_in_chunks(urls)
        print(f"Finished pinging. Successfully processed {successful_chunks}/{len(urls)//100 + (1 if len(urls)%100 else 0)} chunks.")

if __name__ == "__main__":
    # For actual execution, the INDEXNOW_HOST environment variable must be set.
    # Example: export INDEXNOW_HOST="your-blog.com"
    if 'INDEXNOW_HOST' not in os.environ:
        print("Error: INDEXNOW_HOST environment variable not set.")
        exit(1)
    asyncio.run(main())

This script can be executed with the command python -m scripts.indexnow_seed.

Results

• Automated initial IndexNow API pinging for all existing blog posts.
• Facilitated content discovery by search engines, contributing to improved initial SEO performance.
• Significantly saved time and effort compared to manual operations.

Takeaways — So You Don't Fall into the Same Trap

• [ ] When processing a large number of URLs asynchronously, correctly use asyncio.gather to ensure all tasks run in parallel.
• [ ] When chunking URL lists for processing, clearly log the success/failure of each chunk and summarize the overall results.
• [ ] Implement appropriate exception handling to prepare for potential timeouts or errors during external API calls.
• [ ] Add logic to verify that necessary environment variables (e.g., INDEXNOW_HOST) are properly set for script execution.