TL;DR
Manually requesting indexing in Google Search Console is fine when you manage one website with a handful of pages. The moment you are managing ten sites with hundreds of unindexed URLs each, it becomes a full-time job on its own. I built a free local tool that automates the entire process using Google's own official APIs — it scans your sitemaps, checks which URLs are not indexed, submits them to Google and Bing daily, and runs on a schedule without you touching anything. This post explains exactly how it works, what the real limits are, and how you can build or use the same thing.
The Part Nobody Talks About When They Say “Just Request Indexing”
If you have spent any real time in Google Search Console, you know the drill. Open GSC, pick a property, go to URL Inspection, paste a URL, click Request Indexing, wait, go back, do it again. Google lets you do this for roughly 10 to 15 URLs per site per day through the manual interface.
That sounds manageable until you are running an agency with 10 client websites, each with 200 to 300 pages sitting unindexed. Suddenly you are looking at weeks of manual clicking just to get through the backlog. And while you are working through that backlog, new content keeps going live and joining the queue.
This was the exact situation I was dealing with. 1,778 URLs across multiple sites. No realistic way to handle it manually. No tool on the market that solved it the right way without costing a fortune or doing something shady.
So I built one.
Why Existing Tools Did Not Work
Before building anything, I looked at what was already available. The honest summary: most paid indexing tools either charge $50 to $200 per site per month, use methods that bend or break Google’s guidelines (link spam, fake crawl signals), spread their API quota across thousands of customers so you have no idea how much you are actually getting, or give you no visibility into what is happening under the hood.
None of that worked for client work. Clients need transparency. Agencies need control. And nobody needs another monthly subscription that delivers unclear results.
The only clean approach was to use Google’s own APIs directly. Free, official, documented, and fully within Google’s guidelines.
The Five APIs That Power This Tool
Before getting into what the tool does, it helps to understand what each API actually is. None of these require you to be a developer to understand. Think of each one as a different conversation you can have with Google’s infrastructure.
Google OAuth 2.0 is how the tool logs into Google on your behalf. You click Sign In with Google in your browser, approve access once, and your login token is saved locally. You never have to log in again unless you explicitly log out.
The Google Search Console API pulls the list of all websites connected to your Google account automatically. You do not type domain names in manually. The tool fetches everything at once. This matters especially when managing multiple client sites across different Gmail accounts.
The GSC URL Inspection API is the most powerful piece. Think of it as having a Google employee tell you whether a specific page is in their database, without you having to open Search Console and check manually. For each URL, it returns whether it is indexed or not, when Google last crawled it, whether a noindex tag is blocking it, and whether robots.txt is preventing crawling. The daily limit is 2,000 checks across all your sites combined.
The Google Indexing API is what actually submits URLs to Google for crawling. The daily limit is 200 submissions per site per day. One honest caveat worth mentioning: Google officially designed this API for pages with job posting or livestream structured data. The broader SEO community has used it for all page types for years and it works in practice, but Google could restrict this at any time. It is the best tool available right now within Google’s ecosystem, but it is not a permanent guarantee.
IndexNow is Bing’s free open protocol for instant URL submission. No hard daily limit. You can batch up to 10,000 URLs in a single request and Bing often indexes within hours. The only setup is hosting a small verification text file at the root of your website, which can be tricky for client sites where you do not have direct server access.
The Daily Limits You Need to Plan Around
| API | Daily Limit | Important Note |
|---|---|---|
| GSC URL Inspection | 2,000 checks/day | Shared across ALL your sites |
| Google Indexing API | 200 submissions/day | Per site, not shared |
| Bing IndexNow | No hard limit | Batch-friendly, up to 10,000/request |
| GSC Sites List | 1,000/day | More than enough for any setup |
The URL Inspection limit is the one that actually shapes how you use this tool. With 10 sites at 200 submissions each, you can push 2,000 URLs to Google per day. For a fresh setup with a large backlog, expect to spend the first two to three days working through everything. Once the backlog clears, daily new content volumes are almost always well within the limits.
What the Tool Actually Does
The full workflow from start to finish looks like this:
tool-flow.txt
You open the tool → http://localhost:5000
|
v
Sign in with Google (one click, saved after first login)
|
v
Tool fetches all your GSC properties automatically
|
v
For each site:
|
+-- Reads sitemap XML → discovers all URLs
|
+-- URL Inspection API → checks indexed vs not indexed
|
+-- Queues all unindexed URLs for submission
|
+-- Google Indexing API → submits up to 200/day per site
|
+-- IndexNow / Bing → submits simultaneously
|
v
Dashboard shows live status, quota usage, and full logsThe “Run Everything” button handles the entire sequence in one click. A progress bar shows which step is running. For large sites it can take 30 to 60 minutes. You leave the tab open and walk away.
The Features That Actually Matter for Agency Use
Per-site on/off toggles. If a client is mid-redesign and you do not want their URLs being pushed to Google right now, you flip a toggle and that site is completely skipped in every scan, inspection, and submission. No workarounds. No worrying about accidentally submitting broken pages.
Multiple Google accounts. You connect additional Gmail accounts through a single button. All their GSC properties appear in the tool automatically, each associated with the correct account credentials. This is the feature that makes it genuinely usable for agency work rather than just personal sites.
URL Explorer. A filterable table of every URL in the database. Filter by site, by status (indexed, not indexed, submitted, pending), or search by URL text. Useful for spotting patterns in what is and is not getting picked up by Google.
Full logs. Every single API call is recorded: the site, the URL, the action, the result, and the exact API response. When something goes wrong, you know exactly what happened and when.
Daily scheduler. Four jobs run automatically each morning while the app is open: sitemap scan at 6 AM, URL inspection at 7 AM, Google submission at 8 AM, Bing submission at 8:05 AM. Set it up once and let it run.
The Project Structure
project-structure.txt
indexing-tool/
├── app.py ← Main app and all web routes
├── auth.py ← Google login, token storage, multi-account
├── database.py ← All database operations
├── gsc.py ← GSC URL Inspection API calls
├── indexing.py ← Google Indexing API submission
├── indexnow.py ← Bing IndexNow submission
├── sitemap.py ← Sitemap parser (handles index files)
├── scheduler.py ← Daily automation jobs
├── config.py ← Loads settings from .env
├── templates/ ← HTML pages (dashboard, sites, URLs, logs)
├── static/ ← CSS and JavaScript
├── .env ← Your secrets — never share or commit this
├── credentials.json ← Downloaded from Google Cloud Console
├── token.json ← Auto-created after first login
└── indexing.db ← Your local database (auto-created)Everything runs on your own computer. Nothing gets sent to any external server. All data lives in a local SQLite file called indexing.db. For client data privacy, that matters.
The One Technical Step You Cannot Skip
The Google Cloud setup is the only part of this that requires some patience. It takes about 10 minutes and you only do it once.
google-cloud-setup.txt
Step 1: Go to console.cloud.google.com
Create a new project → name it "Indexing Tool"
Step 2: APIs & Services → Library
Enable: "Google Search Console API"
Enable: "Web Search Indexing API"
Step 3: APIs & Services → Credentials
Create OAuth 2.0 Client ID → Web Application
Set redirect URI: http://localhost:5000/oauth2callback
Download the JSON → rename it credentials.json
Step 4: Configure OAuth consent screen
Set to External
Add your Google account as a Test User
Add these scopes (use the full URLs):
https://www.googleapis.com/auth/webmasters.readonly
https://www.googleapis.com/auth/indexing
Step 5: Drop credentials.json into your indexing-tool folderWhen adding scopes, do not type the short names like webmasters.readonly. Use the full URL versions shown above, or simply check the checkboxes next to the entries in the scopes table. Either works. Typing short names does not.
How to Install and Run It
Once the Google Cloud setup is done, getting the tool running takes under two minutes.
install-and-run.txt
# Prerequisites: Python 3.10+ installed on your computer
# Step 1: Open terminal inside the indexing-tool folder
# (Right-click the folder → Open in Terminal on Windows 11)
# Step 2: Install dependencies
pip install -r requirements.txt
# Step 3: Start the tool
python app.py
# Step 4: Open your browser and go to:
http://localhost:5000
# Step 5: Click "Sign in with Google" and approve access
# That is it. The tool is running.Keep the terminal window open while using the tool. If you close it, the tool stops. The scheduler also only runs while the app is active, so plan to leave it running during working hours or overnight if you want the automated daily jobs to fire.
What Happened When We Actually Ran It
On the first real run across all connected sites, the sitemap scan completed in about two minutes and found 1,778 URLs. The URL Inspection step found 77 unindexed URLs from just the first 100 inspected. With the 2,000 per day inspection limit, the full first-time scan across everything took two days to complete.
The most time-consuming part of the entire setup was the Google Cloud configuration, not the tool itself. Once that was done, everything else ran cleanly.
After the first two days: full sitemap coverage, all unindexed URLs identified, daily submission running automatically. No manual GSC clicks. No spreadsheet tracking. No repetitive workflow.
What This Tool Cannot Do
Being straight about the limitations matters more than overselling what this does.
Submitting a URL is a crawl request, not a guarantee. Google decides whether to actually index a page based on content quality, relevance, site authority, and signals that no tool can influence. A page with thin content or duplicate issues will not get indexed just because you submitted it 200 times.
The 200 per day and 2,000 per day API limits are hard Google limits. The tool does not bypass them. If you have 5,000 unindexed URLs across 10 sites, you are working through them over multiple days. That is just how Google’s API works.
The Google Indexing API was officially built for job postings and livestreams. It works for general content right now, and has worked for years. But it is not a permanent guarantee and Google could tighten restrictions. Worth knowing before building your entire workflow around it.
And it does not work without the credentials.json file from Google Cloud. There is no way around that step.
The Bigger Point About AI-Built Tools
This entire tool, including the planning, the code, the debugging, and the feature additions, was built through a conversation with an AI. No developer was hired. No agency was briefed. No months of back and forth.
The barrier to building custom SEO tooling has genuinely never been lower. If you can describe the problem clearly and work through the iterations, you can have something functional and production-ready without a technical background. The hard part is not the code. It is knowing what the tool needs to do and being specific enough about it.
That applies to indexing tools, reporting dashboards, rank trackers, content workflows, client onboarding systems, and anything else that currently lives in a spreadsheet or happens manually. If it is repetitive and rule-based, it can probably be automated. And it probably does not need a six-month development project to get there.
Want This Built for Your Site or Your Agency?
If you manage multiple client websites and the indexing workflow is a real operational drain, this is something that can be built and configured for your specific setup. Every agency has different site structures, different GSC configurations, and different volume requirements. The tool above is a solid foundation and it can be adapted.
Beyond indexing automation, if you want your content to show up in both Google search results and AI-generated answers, that sits at the core of what I work on with agencies and business owners. AEO (Answer Engine Optimization) and custom SEO strategy are not separate from operational work like this. Getting indexed properly is step one. Being found and cited is what comes after.
If you want to talk through your indexing situation, your search visibility, or whether an automation like this makes sense for your setup, book a call. No pitch. Just a real conversation.
Dhruv is an SEO and AEO consultant working with business owners, founders, and agencies. If search visibility is a problem for your business, this is the right place to start.
Dhruv The SEO Guy
I do SEO for agencies, founders, and business owners. No fixed packages. No fluff. Just technical, revenue-focused strategies that scale your organic presence securely.
Ready to dominate search?
Stop reading about algorithms and start ranking. Book a quick 1-on-1 strategy call below.
Book a Strategy Call →
