HN Explorer

Ask HN: Why is YouTube adding “&pp=sAQA” to video URLs?

109 points by pdkl95 ↗ HN
YouTube started adding a new parameter "pp=sAQA" to video URLs on most index style pages (e.g. /feed/subscriptions, search results, the /videos page on any channel). The actual video pages (/watch) strip the pp= parameter, and it doesn't appear to be added to the URLs for the "recommended" videos.

Does anybody know what this parameter does and/or why it was added? It's really annoying; using YouTube URLs in the shell now requires quoting due to the "&".

73 comments

[ 1.5 ms ] story [ 153 ms ] thread
EE84M3i
Sounds to me like a side effect of an A/B test.
20after4
Maybe it's A/B testing? if so, I'm in the same "sAQA" bucket ...
have_faith
An analyst somewhere who isn't aware that people are literally playing with the test buckets and jumping between them to figure out what the query parameters do is going to write a confident report on why A outperformed B.
quickthrower2
Weird they need URL params when they know who you are anyway from the browser thumbprint :-)
lmilcin
Sometimes the intention is to detect the param on the way back, regardless of whom it came back from.

For example, the param may point to specific time on the video.

Or the param may point that the video is part of some playlist, so that when somebody clicks he is brought not just to the same video but also to same playlist.

As we don't know anything about this particular param we don't know if it is the case.

bserge
They're 80% assumptions and fiction in any case, so it's OK.
sltkr
What fraction of the YouTube userbase do you imagine manually edits URLs?
lmilcin
I always strip youtube links of an ampersand and everything that follows when I give them to anybody.
Agent766
If I forget to do that I like to say "pardon my url"
EForEndeavour
I manually edit "utm_" URL parameters to at least feel like I'm sending harmless easter eggs to analysts somewhere. utm_campaign=helpimabot
[deleted]
(comment deleted)
tsywke44
Congrats, you’re in the 0.0001% of youtube users who actually does that
ricardo81
I'm seeing them on the home page, channels, explore, subscriptions but not on searches and 'related videos' while viewing one.

/edited for fullness

sen
I'm seeing it on all pages... Home, Explore, Subscriptions, even History.
themaninthedark
I see it on the front page, but not on left sidebar when watching a video or in the overlay that shows other videos once the one you watched is finished.

I see it in Explore and in all Subscriptions categories(not Subscriptions feed/subscriptions).

It is also not appended to mixes/playlists for me.

Does show up in Library and History, not in Watch Later and Liked Videos.

Shows up in: Movies & Shows, Gaming, Live, Fashion & Beauty, Learning and Sports.

cesarb
It's fairly annoying for other reasons than just shell quoting. For instance, I use the browser history to know whether I've already watched some video; this extra parameter makes all these video links appear to be unvisited to the browser (since back when I visited them, that useless extra parameter didn't exist).
Zealotux
It's not as good, but the history is pretty useful: https://www.youtube.com/feed/history
Nowado
The absolute future of a proprietary service-specific history.
cesarb
That link shows empty to me, since I've never logged into YouTube, and most of the time I'm not even logged into the Google account.

(I don't know if it's still the case, but I've heard that if you log into YouTube, and Google for some reason decides your real name is not your real name, you could lose access to your whole Google account; and at least to me, the Google Talk account was more important than being able to comment on YouTube videos.)

Kye
I don't think that's been a thing for a long time.

https://en.wikipedia.org/wiki/Nymwars

elric
Last I checked, it was impossible to create a YT/Google account without providing a (working) phone number. Which is even worse than requiring "real names" (which can at least be spoofed).
ravedev
That depends. I’ve found that, while creating an account on an iOS device, providing a phone number isn’t required, but while creating an account via a desktop browser, it is.

It will probably depend on other factors as well such as IP, country, …

rtsil
It was still possible as of six months or so ago, who knows what factors Google takes into account to decide whether to require a phone number or a simple email.
jannes
That would require to be logged in to a Google account
rvba
Does it work when a video was deleted?
horlux
how do you use the history exactly?
rasz
I inject this into YT

    // make VISITED links distinct
    //subscriptions/lists
    css += 'a#video-title {color: #167ac6!important;}';
    css += 'a:visited#video-title {color: #141761!important;}';
    //commenter channel name
    css += 'a#author-text {color: #167ac6!important;}';
    css += 'a:visited#author-text {color: #141761!important;}';
    //sidebar
    css += 'a #video-title {color: #167ac6!important;}';
    css += 'a:visited #video-title {color: #141761!important;}';
    
    css += '#content-text a, #description a {color: #167ac6!important;}';
    css += '#content-text a:visited, #description a:visited{color: #141761!important;}';
makes pretty clear which links are visited
rasz
Yesterday after discovering &pp=sAQA breaking by History tracking (got a custom CSS injected to make sure its distinct) I added this

    document.querySelectorAll('a[href*=\\&pp\\=]').forEach(el => {el.href = decodeURIComponent("/watch?v="+el.href.match(/v=(.*?)(&|$)/)[1])})
to my userscript injected into YT. I still need to pinpoint reliable DOMSubtreeModified event to make it work without manually reloading the page.

Edit: and just now discovered its being added server side. Fetched https://www.youtube.com/youtubei/v1/browse?key=xxx already has &pp=sAQA, makes patching even easier, just donna hijack fetch and strip them in the handler... aaand done:

    const origFetch = unsafeWindow.fetch
    unsafeWindow.fetch = async (url, init) => {
      if (url.url.startsWith('https://www.youtube.com/youtubei/v1/browse?key=')) {
        let response = await (await origFetch(url)).text()
        response = response.replace(/(\&pp\=.*?)",/g,"\",")
        return new Response(response)
      }
      else {
        return origFetch(url)
      }
    }

    window.addEventListener('DOMContentLoaded', ()=> {
      if (document.location.pathname == "/feed/subscriptions") {
        document.querySelectorAll('a[href*=\\&pp\\=]').forEach(el => {el.href = decodeURIComponent("/watch?v="+el.href.match(/v=(.*?)(&|$)/)[1])})
      }
    })
Sephr
Regular expression replaces on content are very fragile. Unfortunately, browsers do not yet provide general request regulation capabilities. Until then, Transcend Consent Manager's API can solve this much more easily with a more robust request-override based approach:

    self.airgap = {
      overrides: [{
        matcher: '//www.youtube.com/',
        override(request) {
          request.urls.forEach((input, i) => {
            const url = new URL(input, location.href);
            if (url.searchParams.has('pp')) {
              url.searchParams.delete('pp');
              request.urls[i] = url.href;
            }
          });
        }
      }]
    };
rasz
Its true URLSearchParams is a proper blessed way of working with url parameters, but its just my tiny userscript working on a serialized JSON, so either I grep and be done with it, or be forced to actually parse it, traverse huge Object looking for 100 individual URL fields to modify "properly", and stringlify once again. Im good with regex thank you :)

>Transcend Consent Manager's API

what is that?

>request-override based approach

the idea is to modify those URLs before they get rendered on the page, this way they get applied :visited style in the first place.

Sephr
> what is that?

A new consent management tool for website owners. We're currently in closed beta. Learn more here: https://transcend.io/blog/defeating-cookie-banners

> the idea is to modify those URLs before they get rendered on the page

These overrides are applied before the URLs are rendered on the page or trigger any requests.

rasz
>DOM nodes by directly patching the prototypes, methods, and accessors of all global element constructors and DOM tree construction utilities. We can write a patcher which overrides the constructor prototype, which allows us to check if the URL is allowed before the DOM mutation is performed.

Now I understand. Your product is aimed at corporations seeking more mature? compliance with privacy laws. Sounds like tough market.

qwerty456127
I actually find all kinds of browser history pollution a very serious problem rendering it almost useless (only useful the way "recycle bin" is - to recover an address of that useful page you have seen yesterday but forgotten).

I wish there were browser extensions capable to smartly sanitize the history by removing duplicate URLs pointing to the same actual page visited multiple times during the day and/or by via slightly different URLs, removing the specific parameters which can be removed without changing what does the link point to, rearranging the remaining parameters alphabetically, removing GMail and alike useless URLs, also normalizing the entry titles by removing everything but the actual article title (e.g. removing website name or moving it to an uniform place - always the beginning or always the end) etc.

pmoriarty
I'd prefer that the browser be taken out of the equation completely.

The web browser is becoming just a shitty operating system, largely controlled by unaccountable megacorps who are primarily interested not in empowering users but in tracking them and serving them ads.

folli
You can use curl if you don't like browsers. It just needs a lot of imagination to get an idea on how the site is supposed to look like.
qwerty456127
Whatever, pmoriarty is right. Today a browser indeed is an operating system with its own GUI toolkit and a decentralized repository of apps (we call the web, most of the sites actually are apps today).

Perhaps if the web servers served something like QML/XAML/whatever (while other web servers serving just APIs for them to get/post the content) and there was an operating system command to run such an app by its URL, both the developers and the users might have a better experience. Or perhaps worse.

This way we might even have more than one standard competing: QML, XAML and whatever else would emerge (introducing new would be more viable) vs just HTML.

codegladiator
atob("sAQA") = "°\u0004\u0000"

Although not happening for me

q3k
Shooting from the hip here, but this ("\xb0\x04\x00") could be interpreted as a protobuf with this IDL:

    message Foo {
      optional int32 bar = 70;
    }
that decodes to the following value:

    bar: 0
(or bar could be bool of value false, or enum at value 0/UNSPECIFIED, or int of any other size)

But with such a short message and without other samples it's difficult to prove or disprove that this is indeed a protobuf :).

One could try to set &pp=sAQB (bar = true/1), &pp=sAQP (bar = 15), ... and observe what happens.

mentat
This seems the most like case to me. Google loves their query PBs.
[dead] hollander
Just remove it. The video plays.
scubbo
What an uncurious attitude.
mikewarot
That gives a plausible cause to my suspicion that I'm seeing a bunch of stuff I already watched showing up in suggestions again.
carnitine
Why would this affect YouTube’s internal recommendation system?
ttctciyf
YouTube has never been shy of recommending already-watched videos, the issue is whether you can see that you watched them. I think.
HKH2
So YouTube is setting environment variables for people who don't do shell quoting. Next they can add "&reboot" to all their URLs.
hobos_delight
This is how you append query params to a url - nothing to do with the shell.
tomerv
The GP complained about shell quoting.
judge2020
I doubt it is YouTube's intention to require people start quote URLs in shell, though.
PaulDavisThe1st
....&foo=BAR

sets a shell variable, not an environment variable. The value of $foo will not be available to shell-forked processes.

coldpie
What? Look at the URL in your browser right now. It has a bunch of x=y parameters. That's how URLs work.
Sophira
The parent comment is talking about what would happen if, for example, you typed this in bash to try to download a video:

$ youtube-dl https://www.youtube.com/watch?v=dQw4w9WgXcQ&pp=sAQA

In this case, bash would interpret it as "run youtube-dl in the background with the parameter 'https://www.youtube.com/watch?v=dQw4w9WgXcQ', then execute the command 'pp=sAQA' to set the 'pp' variable".

The comment is joking that if YouTube decided to replace "pp=sAWA" with "reboot", they could get a lot of people to accidentally reboot their machines instead because they didn't quote the URL.

chrisseaton
That was always a problem if you just copy-and-pasted URLs into your terminal. You need to quote them. It's not new.
villgax
It used to work in bash but doesn't in zsh
__henil
Not sure if its something related, I got a wikipedia link from someone which included `&wprov=sfti1` as query string as well.
Brian_K_White
Stripping is nice an all but I want a plugin or greasemonkey script to rewrite that to pp=random on every request.
smw
Why don’t we have better magic in shells for quoting pasted urls?
[deleted]
(comment deleted)
[deleted]
(comment deleted)
glow1nthedark
Seems like it's being appended to video links shown on youtube that are not from either search results or related videos.

I suspect this is a way for them to exclude click metrics, etc. from videos that were not shown as a result of machine learning/lexical analysis/whatever.

I'm sure they've been identifying how videos propagate since forever, but my guess is that they recently decided it would be efficient to add some cryptic-looking parameter (that probably just corresponds to a boolean) to all video links shown on channels, the home page, etc.