How to Scrape YouTube Shorts (Python, yt-dlp & More)

I tested how to scrape YouTube Shorts this week to see whether they need their own scraper or whether the methods that read a normal video already work. The short answer is that a Short is a normal video wearing a different URL. A Short lives at youtube.com/shorts/<id>, where <id> is the same 11-character ID YouTube gives every video, so the single most useful move is to rewrite that path to watch?v=<id> and reuse everything you would run on a regular watch page.

This guide is the Python I ran against live Shorts in June 2026: the plain requests route, the yt-dlp route for one Short, the harder job of listing a whole channel’s Shorts tab, and where each one breaks. The numbers below are from real runs.

What data can you scrape from a YouTube Short?

You can scrape a YouTube Short’s title, channel, channel ID, view count, like count, duration, upload date, description, keywords, and thumbnail, plus the ordered list of every Short on a channel. This is the same data set you get from a regular video, because a Short is served from the same backend with the same JSON, and all of it is data YouTube already serves to a logged-out browser.

Here is what a Short exposes and which method reaches each field cleanly:

The pattern is the same as on a watch page: the visible numbers are rendered by JavaScript, so they are not in the static HTML as tags. They sit in two JSON objects the page ships with, ytInitialPlayerResponse for the Short itself and ytInitialData for the surrounding channel and tab data. Once you know that, scraping a Short is a JSON-parsing job that starts with one URL change.

How do you convert a Shorts URL to scrape it?

You convert a Shorts URL by replacing the /shorts/ path segment with /watch?v=, keeping the same 11-character video ID, which turns youtube.com/shorts/<id> into youtube.com/watch?v=<id> and unlocks the full watch-page JSON. The Short and the watch page resolve to the same underlying video, so the metadata is identical, the watch page just exposes more of it cleanly.

A one-line regex handles the conversion for any Shorts link:

import re

def shorts_to_watch(url):
    m = re.search(r"youtube\.com/shorts/([A-Za-z0-9_-]{11})", url)
    if not m:
        return url
    return f"https://www.youtube.com/watch?v={m.group(1)}"

print(shorts_to_watch("https://www.youtube.com/shorts/dQw4w9WgXcQ"))
# -> https://www.youtube.com/watch?v=dQw4w9WgXcQ

This conversion is the foundation every competitor tutorial builds on, and for good reason: it means you do not maintain a separate Shorts parser. The same code that scrapes a YouTube video reads a Short once the URL is normalized, which is exactly what the next method does in plain Python.

How do you scrape a YouTube Short with Python?

The fastest way to scrape a YouTube Short with Python is to convert the URL to the watch form, request it, pull the ytInitialPlayerResponse JSON blob out of the HTML with a regex, and load it with json.loads. This is how you scrape a Short without the official API: the data is already in the page, packed into the same JSON object the browser reads to render the player.

Install the libraries first:

pip install requests beautifulsoup4 lxml

Then request the converted URL and read the fields out of videoDetails:

import requests, re, json

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/124.0 Safari/537.36")

watch_url = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"   # from shorts_to_watch()
r = requests.get(watch_url, headers={"User-Agent": UA,
                                     "Accept-Language": "en-US,en;q=0.9"}, timeout=20)
print(r.status_code)                       # -> 200

m = re.search(r"ytInitialPlayerResponse\s*=\s*(\{.+?\})\s*;\s*(?:var|</script>)", r.text)
vd = json.loads(m.group(1))["videoDetails"]

print(vd["title"])          # the Short's title
print(vd["author"])         # channel name
print(vd["channelId"])      # UC...
print(vd["lengthSeconds"])  # Shorts are <= 180 seconds
print(vd["viewCount"])      # raw integer view count
print(vd["keywords"][:5])   # tag list, if the uploader set any

When I ran this against a converted Shorts URL, the request returned 200 and videoDetails held the title, channel ID, a lengthSeconds value inside the Shorts ceiling, and the raw viewCount as an integer string. This is the core no-API scrape: one URL rewrite, one HTTP request, one regex, one json.loads.

The fragile part is the regex. YouTube changes the surrounding markup periodically, and when the delimiter after the JSON object shifts, the pattern misses and m is None. That maintenance cost is the reason the next method exists, because yt-dlp tracks those changes for you.

How do you scrape YouTube Shorts metadata with yt-dlp?

You scrape a YouTube Short’s metadata with yt-dlp by calling extract_info with download=False, which accepts the /shorts/ URL directly and returns a Python dictionary of the Short’s title, channel, duration, view count, like count, and upload date without downloading the file. yt-dlp is a maintained fork of youtube-dl that does the JSON parsing internally, so it survives YouTube’s layout changes better than a hand-written regex and does not even need the URL conversion.

Install it, then read one Short:

pip install yt-dlp

from yt_dlp import YoutubeDL

SHORT = "https://www.youtube.com/shorts/dQw4w9WgXcQ"   # /shorts/ URL works as-is
opts = {"quiet": True, "skip_download": True, "noplaylist": True}

with YoutubeDL(opts) as ydl:
    info = ydl.extract_info(SHORT, download=False)

for field in ["title", "channel", "channel_id", "duration",
              "view_count", "like_count", "upload_date"]:
    print(field, "=", info.get(field))

yt-dlp accepted the raw Shorts URL in my run and returned the same metadata dictionary it gives for a normal watch URL, with duration inside the Shorts ceiling and view_count and like_count as plain integers. The project documents these flags, including --dump-json for a one-shot JSON print and --skip-download to write metadata only, in its command-line options. For the command line, yt-dlp --dump-json --skip-download "https://www.youtube.com/shorts/<id>" prints the whole metadata object in one call.

yt-dlp is the gold standard for single-Short scraping because it tracks YouTube’s changes for you. One honest caveat from my run: yt-dlp now prints a warning that it wants a JavaScript runtime installed, and without one some download formats are skipped. For metadata extraction the warning did not block the result, but it signals that YouTube is making pure-Python extraction harder over time. What yt-dlp does not do well is enumerate a whole channel’s Shorts tab, which is the next piece.

How do you scrape all of a channel’s Shorts?

You scrape all of a channel’s Shorts by loading the youtube.com/@handle/shorts tab and reading the list of Shorts renderers out of the ytInitialData JSON, then following continuation tokens for the rest. This is the genuinely harder job, because the Shorts tab is built by JavaScript and paginated, so a single requests.get returns only the first batch of Shorts before the continuation tokens take over.

The first page is parseable with the same JSON approach used for search results:

import requests, re, json

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/124.0 Safari/537.36")
URL = "https://www.youtube.com/@MrBeast/shorts"

r = requests.get(URL, headers={"User-Agent": UA}, timeout=20)
m = re.search(r"var ytInitialData\s*=\s*(\{.+?\})\s*;\s*</script>", r.text)
data = json.loads(m.group(1))
# Walk tabs -> richGridRenderer -> contents to reach each shortsLockupViewModel.

Two things make this method real work at scale. First, the structure under ytInitialData is deep and nests the Shorts inside a grid renderer that YouTube reshapes from time to time, so the walk is brittle. Second, the tab only shows a window of Shorts up front, and the rest arrive through the internal /youtubei/ endpoint, which YouTube disallows in its robots.txt alongside /results and /watch_ajax. The /shorts/ and /@handle paths themselves are not in that disallow list, which is a useful distinction when you decide what to crawl.

If you need every Short on many channels with view counts and positions, a channel scraper that returns the Shorts list as JSON removes the continuation-token bookkeeping for you.

When should you use Playwright for Shorts scraping?

You should use Playwright for Shorts scraping when you need data that only appears after scrolling the Shorts tab or interacting with the player, such as loading a long channel’s full Shorts grid or expanding comments. Playwright drives a real Chromium instance, so YouTube’s JavaScript runs, the Shorts grid lazy-loads as you scroll, and the rendered values become readable without you decoding continuation tokens by hand.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://www.youtube.com/@MrBeast/shorts", wait_until="networkidle")

    for _ in range(5):                       # scroll to lazy-load more Shorts
        page.mouse.wheel(0, 4000)
        page.wait_for_timeout(1200)

    hrefs = page.eval_on_selector_all(
        "a[href*='/shorts/']",
        "els => els.map(e => e.getAttribute('href'))",
    )
    print(len(set(hrefs)), "shorts found")
    browser.quit()

Scrolling is the only way to trigger the lazy-loaded grid without touching the internal API, which is why the Puppeteer and Playwright tutorials in the SERP lean on it. Playwright reads the rendered tab, so the Shorts links resolve once Chromium has executed YouTube’s scripts. The cost is speed and overhead: a browser session uses far more memory and time than an HTTP request, and YouTube’s consent and bot-check screens show up more readily for automated browsers, so you add waits and a consent click. For one channel Playwright earns its place; for many channels at volume, the quota and blocking limits decide which route is sustainable.

What are the limits, and how do you avoid getting blocked?

The hard limits on Shorts scraping are the official API quota and YouTube’s anti-bot blocking on the unauthenticated pages. The YouTube Data API v3 gives each project a default allocation of 10,000 units per day plus a separate allowance of 100 search.list calls, per Google’s getting-started documentation. There is no Shorts-specific endpoint in the API: you read a Short through videos.list like any other video, and you find a channel’s Shorts by walking its uploads, so the same quota math applies.

Scraping the pages directly has no quota, but it has a different ceiling: blocking. From my own machine the requests above returned clean 200s. From a cloud server, YouTube starts answering automated traffic with consent interstitials and bot challenges. These are the levers that keep page scraping working, in rough order of impact:

• Slow the request rate. A steady, human-like cadence with a few seconds between requests survives far longer than parallel bursts, and the Shorts tab’s lazy loading rewards patience anyway.
• Use residential IPs for volume. Datacenter ranges get challenged quickly, and YouTube is effective at blocking them. Residential proxies present as ordinary home connections.
• Set a real User-Agent and Accept-Language. A missing or empty User-Agent makes a borderline request worse.
• Cache Short IDs. The cheapest request is the one you skip. Store IDs and only refetch the Shorts whose view counts you actually need to refresh.
• Respect robots.txt and the Terms of Service. YouTube disallows /youtubei/, /results, and /watch_ajax in its robots.txt, and its Terms of Service restrict automated access.

On the legal point, US courts have read the Computer Fraud and Abuse Act narrowly for public data. In hiQ Labs v. LinkedIn, the Ninth Circuit affirmed in 2022 that scraping public website data likely does not violate the CFAA, a reading that followed the Supreme Court’s narrow CFAA interpretation in Van Buren v. United States.

Public data and a platform’s own terms are different matters, so read both before scaling, the same way I lay it out in is scraping YouTube legal.

The practical reading: build the Python yourself for a few channels, and move to a managed service when the proxy rotation, consent handling, and continuation-token upkeep cost more time than the Shorts data is worth.

How do you scrape YouTube Shorts at scale without managing proxies?

You scrape YouTube Shorts at scale without managing proxies by sending each Short’s id or URL to a scraper API that returns that Short’s full metadata as parsed JSON, with the proxy rotation, consent handling, and ytInitialPlayerResponse parsing handled on the server side. You send one request per Short and get its title, stats, duration, channel, and description back, no regex to maintain when YouTube reshapes the markup.

This is the route I reach for once a project outgrows a few Shorts. Our YouTube API handles this, and its Shorts endpoint is a single GET that takes a Short’s video_id (or a url) and your API key, returning the Short object directly:

curl "https://api.youtubescraperapi.com/api/v1/youtube/shorts?video_id=YA_kX8hu1gg&api_key=$API_KEY"

In Python, the same call returns the Short parsed and ready: the title, view and like counts, duration, channel, description, keywords, and thumbnail, plus an is_short flag and a related array of nearby videos:

import requests, os

resp = requests.get(
    "https://api.youtubescraperapi.com/api/v1/youtube/shorts",
    params={"video_id": "YA_kX8hu1gg", "api_key": os.environ["API_KEY"]},
    timeout=30,
)
short = resp.json()
print(short["title"], "-", short["view_count"], "views")
print("Likes:", short["like_count"], "Duration:", short["duration_seconds"], "s")
print("Is short:", short["is_short"], "Related:", short["related_count"])

The url parameter accepts a youtube.com/shorts/<id> link, a watch?v= URL, a youtu.be link, or a bare 11-character id, so a link copied straight from the Shorts player works without the shorts_to_watch rewrite above. The response is one flat object: video_id, title, description, view_count, like_count, duration_seconds, keywords, category, publish_date, channel_id, channel_name, thumbnail, is_short: true, and a related array (each item with position, id, title, url, channel, and views as a display string like "22M views"). You can start free and test the shape against your own Shorts on the youtubescraperapi.com sign-up.

To list every Short on a channel instead of one Short at a time, that is a different job: point the channel scraper at the handle to walk the creator’s tabs with pagination, then feed each Short id back into this endpoint for its full metadata. The same managed pattern covers the neighboring jobs by swapping the path: a video scraper for a regular watch URL, a channel endpoint for the rest of a creator’s uploads, and a search endpoint for ranked results.

For a one-off pull of a few Shorts, the Python in this guide is enough, and yt-dlp covers single-Short metadata. For continuous collection across many Shorts, offloading the blocking and the parsing is usually the cheaper path once you price in your own time.

If you want the wider picture of Python scraping methods first, I walk through them in how to scrape YouTube.

For the managed side, I rank the providers I tested in the best YouTube scrapers for 2026 comparison.

A note on what you get from Shorts data

Shorts data is most useful when you compare across videos, because one Short’s view count tells you little until it sits next to others. A single-Short scrape gives you the Short’s own view_count, like_count, and duration_seconds, plus a related array of nearby videos that already carries each one’s views as a display string. Collect a Short from each id you care about and you can rank them, or rank the related set the endpoint hands back, to see which short-form ideas landed.

YouTube returns view counts on the related items as strings like "8.2M views", so a small parser turns them into numbers you can sort:

def parse_views(v):                          # "8.2M views" -> 8200000
    n = v.replace(" views", "").strip()
    mult = {"K": 1_000, "M": 1_000_000, "B": 1_000_000_000}
    return float(n[:-1]) * mult[n[-1]] if n and n[-1] in mult else float(n or 0)

top = sorted(short["related"], key=lambda s: parse_views(s["views"]), reverse=True)
for s in top[:5]:
    print(s["views"], "-", s["title"])

The same parse_views helper works on a list of Shorts you assembled yourself: scrape each id, push the numeric view_count and title into a list, and a view-per-Short distribution or a title keyword analysis is a few more lines. The scrape gets you the clean fields, and the parsing turns them into the competitor signal you actually wanted, instead of an afternoon of HTML cleanup. To gather the ids for a whole creator in the first place, the channel scraper lists a channel’s videos, then this endpoint fills in each Short’s detail.