How to Scrape YouTube Search Results (Python & API)

I tested how to scrape YouTube search results this week because it is the one surface people assume is easy and then get stuck on. The instinct is to request https://www.youtube.com/results?search_query=web+scraping, run it through BeautifulSoup, and read off the video titles. I did exactly that in June 2026, and the page returned HTTP 200 with about 600 KB of HTML, yet a search for <a> tags found none of the videos. The ranked result list you see in a browser is not in the HTML as tags at all. To scrape YouTube search results you read the JSON the page already ships with.

This guide is the Python code I ran against live YouTube searches, the data each method returned, and the point where each one breaks. I will cover what a search result actually contains, the ytInitialData parsing route, the yt-dlp shortcut, the official Data API and its quota, how to scrape the related searches under the box, and a managed YouTube search scraper that returns the list as parsed JSON. Everything below is what I executed in June 2026, using the query web scraping, so the numbers are real.

What data can you scrape from a YouTube search?

A YouTube search result gives you the rank position, the video title, the watch link, the eleven-character video id, the channel name and channel link, the view count, the publish date, the video length, a description snippet, and the thumbnail URLs. That is the per-video payload behind every card on the /results page, and all of it is data YouTube already serves to a logged-out browser. The same page also exposes the related searches it suggests under the box, which is a separate dataset worth its own method later.

Here is what one search result exposes and where each field lives in the page:

The pattern is the same on every row: the visible text you read in the browser is rendered by JavaScript, so it is not in the static HTML as tags. It is packed into one big JSON object the page ships with, called ytInitialData, which holds the entire search response. Once you know that, scraping a YouTube search becomes a JSON-parsing job, which is where the Python starts.

How do you scrape YouTube search results with Python?

The fastest way to scrape YouTube search results with Python is to request the /results page, pull the ytInitialData JSON blob out of the HTML with a regex, and load it with json.loads. This is how you scrape the search surface without the official API: the entire ranked list is already in the page, packed into a JSON object the browser reads to build the cards.

Install the one library you need first:

pip install requests

Then request the results page and lift the JSON blob out of it. YouTube assigns the search response to a JavaScript variable, var ytInitialData = {...};, so a regex that captures everything between that assignment and the closing ;</script> gets you the object:

import json
import re
import requests
from urllib.parse import quote

query = "web scraping"
url = "https://www.youtube.com/results?search_query=" + quote(query)

headers = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

html = requests.get(url, headers=headers, timeout=30).text

# The whole search response is assigned to one JS variable on the page.
match = re.search(r"var ytInitialData = (\{.*?\});</script>", html)
data = json.loads(match.group(1))
print("Pulled ytInitialData:", len(match.group(1)), "bytes")

The result list is nested deep inside that object. YouTube wraps the cards in a section list, so you walk down to the itemSectionRenderer, then iterate its items and keep the ones that carry a videoRenderer (the section also contains ads, shelves, and channel cards you want to skip):

section = (
    data["contents"]
    ["twoColumnSearchResultsRenderer"]
    ["primaryContents"]
    ["sectionListRenderer"]
    ["contents"][0]
    ["itemSectionRenderer"]
    ["contents"]
)

position = 0
for item in section:
    video = item.get("videoRenderer")
    if not video:
        continue  # skip ads, shelves, channel rows
    position += 1
    title = video["title"]["runs"][0]["text"]
    video_id = video["videoId"]
    channel = video["ownerText"]["runs"][0]["text"]
    views = video.get("viewCountText", {}).get("simpleText", "")
    print(position, "-", title, "|", channel, "|", views, "|", video_id)

When I ran this against web scraping in June 2026, it printed a clean ranked list: the position, the title, the channel, the view-count string like 1.2M views, and the eleven-character videoId for each result. The viewCountText arrives as display text (1.2M views), not an integer, so you parse it to a number yourself if you need to sort. That is the whole job, and it is why the search scraper landscape keeps coming back to one question: who maintains the parser when YouTube reshapes that JSON.

Because the path is brittle, wrap every key access defensively. The keys above (twoColumnSearchResultsRenderer, itemSectionRenderer, videoRenderer) are the ones YouTube renames or reorders when it ships a layout change, and a missing key throws a KeyError mid-scrape rather than returning an empty list. A logged-out request from a cloud IP can also hit a consent wall instead of results, which returns a different JSON shape entirely. That fragility is the reason the next method exists.

Can you scrape YouTube search with yt-dlp?

Yes, and yt-dlp is the fastest no-parser route to a search list in Python. The open-source project already knows how to read YouTube’s internal data, so you hand it a special ytsearch query and it returns the results as structured entries without you touching ytInitialData at all.

Install it first:

pip install yt-dlp

The query string ytsearchN:terms tells yt-dlp to return the top N results for terms. Set extract_flat so it lists the results without fetching every video page, which keeps a 20-result search to a single round trip:

from yt_dlp import YoutubeDL

opts = {"quiet": True, "extract_flat": True, "skip_download": True}

with YoutubeDL(opts) as ydl:
    info = ydl.extract_info("ytsearch20:web scraping", download=False)

for i, entry in enumerate(info["entries"], start=1):
    print(i, "-", entry.get("title"), "|", entry.get("channel"), "|", entry.get("id"))

When I ran that, it returned 20 entries, each with id, title, url, channel, and duration. The trade against the hand-rolled parser is field coverage: yt-dlp gives you fewer fields per result in flat mode (no view-count string, no publish-date text), but it survives YouTube’s layout changes because the project updates the extractor for you. For a quick list of video ids to feed into a YouTube video scraper afterward, this is the shortest path. When you need the full per-result payload instead of just ids, the choice comes down to the official API or a managed one.

How do you scrape YouTube search with the official API?

The official route is the YouTube Data API v3, through the search.list endpoint, which sidesteps the page-parsing problem entirely and returns clean structured results. You request an API key in Google Cloud, call search.list with q set to your query and type=video, and read the items off the response. The Search: list reference documents the parameters and the resource each item returns.

The constraint is the quota, and it is the whole story with search. Google gives every project a default allocation of 100 search.list calls per day, separate from the 10,000 units per day it grants for the other endpoints, per its quota documentation. That ceiling works out to roughly 100 searches before the allocation resets at midnight Pacific, regardless of how few results you ask for, because the limit is per call. A videos.list read by contrast draws from the separate 10,000-unit pool, so search is the call that runs out first. Raising the allocation is possible but slow, since Google requires you to complete its quota extension request form and pass a compliance review.

The other catch is field coverage. search.list returns the video id, title, channel, publish time, and thumbnails per item, but not the view count or duration. To get those you collect the ids from the search, then make a separate videos.list call with part=statistics,contentDetails, which spends more of the quota and adds round trips. For a low-volume project the free tier is fine; for search-heavy work the 100-a-day ceiling is the wall you hit first, which is why the next two sections cover the routes with no Google quota at all.

How do you scrape YouTube related searches?

You scrape YouTube related searches from the autocomplete suggestion service, because YouTube has no official API for the keyword suggestions it shows under the search box. Those suggestions are the long-tail queries real users type, so they are the most useful keyword dataset on the platform, and Apify’s own keyword-scraping guide confirms there is no first-party endpoint for them.

The DIY approach hits the public suggestion service, which returns a JSON array of completions for a seed term, then optionally appends letters A to Z to the seed to fan out the long tail. It works, but it is undocumented and rate-limited, and the response format is a loosely typed array rather than clean labeled JSON. A managed search endpoint wraps that service and hands back a clean related_searches array instead, which is the lower-friction route when you want the suggestions alongside the result list.

In my June 2026 testing, one GET to our suggest endpoint returned the related searches for web scraping as JSON. The youtubescraperapi.com API wraps that suggestion service, and the request is a plain GET with the seed query and your api key:

curl "https://api.youtubescraperapi.com/api/v1/youtube/suggest?query=web%20scraping&api_key=$API_KEY"

The Python version is the same shape and returns the suggestion list straight away:

import requests

resp = requests.get(
    "https://api.youtubescraperapi.com/api/v1/youtube/suggest",
    params={"query": "web scraping", "api_key": "YOUR_API_KEY"},
    timeout=30,
)
data = resp.json()
print(data["related_count"], "related searches for", data["query"])
for term in data["related_searches"]:
    print("-", term)

When I ran that, the response came back directly as the data object, carrying query, a related_searches array, and a related_count:

{
  "query": "web scraping",
  "related_searches": [
    "web scraping python",
    "web scraping tutorial",
    "web scraping projects",
    "web scraping with ai",
    "web scraping javascript"
  ],
  "related_count": 10
}

That is the same suggestion list YouTube shows under the box, captured as structured data you can feed into keyword research or seed a wider crawl. Pairing it with the ranked result list is how you turn one seed query into both a competitive snapshot and a keyword map, which is the next endpoint.

How do you scrape YouTube search results without the quota?

You scrape YouTube search without the Data API quota by reaching the public /results page through a clean IP, either with your own residential proxies and ytInitialData parser or with a managed scraper API that runs both for you. Residential proxies route requests through ordinary home connections instead of a datacenter IP, which is what keeps YouTube from serving a consent wall or a bot check on repeated searches. The catch is the upkeep: you buy and rotate the pool, pace the requests, retry the soft blocks, and patch the parser every time YouTube reshapes the search response, all of which becomes a real project past a few hundred queries.

A managed scraper API removes that work, taking the search query and returning the parsed result list as JSON with the proxies, retries, and parsing on the server side. The youtubescraperapi.com API is built for exactly this, and in my June 2026 testing a single GET to its search endpoint returned the ranked results for web scraping as clean JSON, with no API key registration in Google and no proxy pool of my own.

The request is a plain GET with the search query and your api key:

curl "https://api.youtubescraperapi.com/api/v1/youtube/search?search_query=web%20scraping&api_key=$API_KEY"

The Python version is the same shape and iterates the results straight away:

import requests

resp = requests.get(
    "https://api.youtubescraperapi.com/api/v1/youtube/search",
    params={"search_query": "web scraping", "api_key": "YOUR_API_KEY"},
    timeout=30,
)
data = resp.json()
print(data["results_count"], "organic results,", data["ads_count"], "ads, for", data["query"])

for video in data["organic_results"]:
    print(
        video["position"],
        "-", video["title"],
        "|", video["channel"]["name"],
        "|", video["views"],
        "|", video["video_id"],
    )

The response comes back directly as the data object, with the ranked list under organic_results and each result carrying its position, title, watch link, eleven-character video id, a channel object, views, publish date, length, and thumbnail URLs. Paid placements are kept out of that list and surfaced separately under ads, with an ads_count, so an ad never lands in the middle of the real ranking:

{
  "query": "web scraping",
  "results_count": 18,
  "organic_results": [
    {
      "position": 1,
      "title": "Web Scraping with Python - Beautiful Soup Crash Course",
      "link": "https://www.youtube.com/watch?v=XVv6mJpFOb0",
      "video_id": "XVv6mJpFOb0",
      "channel": {
        "name": "freeCodeCamp.org",
        "link": "https://www.youtube.com/channel/UC8butISFwT-Wl7EV0hUK0BQ",
        "verified": false
      },
      "views": "1.8M views",
      "published": "5 years ago",
      "length": "1:08:23",
      "description": null,
      "thumbnail": {
        "static": "https://i.ytimg.com/vi/XVv6mJpFOb0/hq720.jpg",
        "rich": "https://i.ytimg.com/an_webp/XVv6mJpFOb0/mqdefault_6s.webp"
      }
    }
  ],
  "ads": [],
  "ads_count": 0
}

When I ran that against web scraping in June 2026, the positions and view counts matched the live search, the ads array came back empty for that query, and the video_id on each result was the same eleven-character watch id you would feed back into a single-video lookup. You can sign up for a key on the youtubescraperapi.com sign-up page and swap it into the snippet above. This route trades the free official allocation for a managed one, so it pays off once your search volume or the maintenance cost of the parser outweighs the subscription. For ten searches a week the ytInitialData parser or yt-dlp is fine; for many queries on a schedule, offloading the rotation and parsing is usually the cheaper path once you price in your own time chasing layout changes.

Is it legal to scrape YouTube search results?

Scraping the public /results page sits in the same grey area as scraping other public YouTube surfaces: the page is served to a logged-out browser, but YouTube’s Terms of Service restrict automated access, so the search-results case turns on how you collect and what you do with the data rather than on whether the page is reachable. Search results are mostly factual rankings and metadata rather than the videos themselves, which is a different copyright posture from downloading content.

YouTube also disallows /results and the internal /youtubei/ paths in its robots.txt, which is worth reading before you scale a search crawl.

On the public-data question, US courts have read the Computer Fraud and Abuse Act narrowly: in hiQ Labs v. LinkedIn the Ninth Circuit affirmed in 2022 that scraping public website data likely does not violate the CFAA, following the Supreme Court’s narrow CFAA reading in Van Buren v. United States.

Public data and a site’s own terms are different matters, so I walk through the full picture in is scraping YouTube legal, which is the place to settle it before you collect at scale. The short version is that public, factual search metadata at modest volume is the low-risk end, and aggressive automated collection is where the Terms of Service bite.

Which method should you use?

The right method for scraping YouTube search results comes down to three things: how many queries you need, whether the 100-a-day API allocation fits, and how much engineering time you want to spend chasing YouTube’s layout. The ytInitialData parser is the most data-rich route and free, if you are willing to fix it when it breaks. yt-dlp is the most robust DIY route, with fewer fields but an extractor someone else maintains. The official search.list API is clean but capped near 100 searches a day. A managed scraper API fits scale, returning every field as JSON without proxies or parsing, and it is the route I reach for once a job runs on a schedule, with a companion endpoint for the related searches when keyword research is the goal.

If you want a ranked comparison of the managed options against price and success rate, I put six of them through the same job in the best YouTube search scrapers for 2026.

And if you are still mapping the broader surface, my overview of how to scrape YouTube ties the search method to the video, channel, and comment surfaces around it.