Here's the secret sauce: "Using machine learning, the algorithm serves videos to users based on their proximity to other clusters of users and content that they like." It's impressive how they arranged to be transparent for everything else, but kept this important part opaque.
Rumor is that the actual secret sauce here is human curation; people hand-select videos with high appeal and label them "viral", "popular", etc. in order to astroturf eyeballs and clicks. I suppose that admitting this directly would contrast sharply with the Chinese-harmonious-technocracy veneer that they work to project.
If this is true... then imagine if you had an unofficial business relationship with these curators to tag your videos over other similar videos. Obviously you don’t want them to be too obvious but when many with good viral quality just need a bump that could be enough to generate lots of revenue to split.
if that rumor holds any water, that would be an impressive team of curators who have a pretty extraordinary ability to spot* globally favorable trends. I remember reading when FB tried to employ human curators for their news section, bias creeped in and was scrapped away. wonder what TikTok did differently.
* although with popularity of TikTok, one needs to wonder if that team have become THE trendsetters themselves...
Your recollection is a little bit faulty on Facebook.
They had teams of editors for Trending. Then, in 2016, conservatives complained that there was a liberal bias (which I don't really think there was, except obvious lies were filtered out). The editors were then removed, with the result that Pope Francis endorsed Trump a few days before the 2016 election.
It mentions that it also serves you from time to time a video from a different cluster and analyzes your reaction so you are not stuck in one cluster without hope to get out.
Probably YT algorithm fall into a local maximum where it detected that X is enraging enough, then creators notice that the X is popular and they start putting more X, then the algorithm is putting more weight behind X and you are now stuck here.
Can you tell me a case where humans curators would help?
IMO YT should put more humans to answer support issues, like you have an youtuber with 10 years of experience and a large number of viewers and you just stike him without a human looking? If the strike/block has a large impact then have a human check.
It is not possible to have a very large pool of curators that could 'hand-select' the 'right' content that could hook up people.
Tictok and Toutiao in China often push irrelevant content or new content by new creator to users to give them potentially interesting things outside their usual sweet spot and also to give new creators chances to win followers. They are exceptionally good at this.
"We're a 2-year-old company operating with the expectations of a 10-year-old company," said Michael Beckerman, TikTok's vice president in charge of U.S. public policy. "We didn't have the opportunity to grow up in the golden years of the internet, when tech companies could do no wrong. We grew up in the techlash age ,where there's a lot of skepticism of platforms, how they moderate content and how their algorithms work.""
Well, yes, Compete or die. It's the same as a tobacco company start up.
I meant by copy/pasting that quote that TikTok's excuse of "we're just 2yrs old" is not applicable because the Tech world is entering an era of regulation, which put TikTok on the front page of whom to target first, same as any other business that enters the marketplace. Using Juul as a reference to the regulation that they have to abide by (now) and how they innovated to become known/verb.
Though, you can very much argue that Juul is both Tech & Tobacco. But, imagine the same statement from Juul.
I don't see any details about its algorithm, just that they use a recommendation engine? Things like this aren't proprietary info, they're just how recommendation engines work:
"Once TikTok collects enough data about the user, the app is able to map a user's preferences in relation to similar users and group them into "clusters." Simultaneously, it also groups videos into "clusters" based on similar themes, like "basketball" or "bunnies.""
Although I do wonder, and maybe someone else with more experience could shed some light here, whether or not it is likely that TikTok has some fundamentally super advanced algorithm, or if they just do a better job of collecting data/training & evaluating their models?
More data beats smarter algorithms any day, and TikTok gets a lot of data because its videos are so short and interaction rate so high. There are tons of signals it can use as inputs: How much of a video did you let play before swiping next? How many times did you let the video loop? Did you like? Did you comment? Did you like a comment? Did you click through to the profile? Did you view other videos from the profile?
I never thought of it like this. Just had one of those mind exploding moments as I realized the frequency of interactions on TikTok vs other platforms and the virtuous cycle it creates (order of magnitude more data -> better recommendations -> even more interactions). Thanks for this.
I don't buy this argument. YouTube ostensibly has a metric ton of information like this and even if tiktok has more training data now, I'm fairly sure their training data in principle was smaller than what YouTube historically had given their decades long presence and their ubiquity in the internet.
This is on top of the documented effort by YouTube to perfect their recommendation algorithm using the best ML minds they got [1] only to polarizing response from its users.
Clearly tiktok has other advantages (homogeneity in some content characteristics, viz. Extremely short videos which probably correlates to their simplicity) and has clearly tuned a fundamentally better recommendation algorithm that even the minds at Google brain couldn't figure out.
YouTube has the problem that all Google products have: they put you in a filter bubble which you can never get out of. The algorithm also optimises for more "long form" content and it's pretty well known that the optimum video length is around 10 mins.
YouTube's recommendations are arguably just as good as TikTok's in my opinion. The only difference is that YouTube places the burden of choice on you while TikTok makes every choice for you.
If you don't quickly find something you'd like to watch on YouTube you're very likely to leave and find something else to entertain you. Meanwhile TikTok is automatically feeding you an infinite source of quick and easy to digest content, all of which you'll probably like to some degree.
YouTube could do something similar and give users an automatic continuous feed (ala a TV channel), but I think YouTube's content is much too longform to work well in this format. This burden of choice problem affects Netflix in the same way which also has superb recommendations.
I also agree that YouTube's recommendations are pretty good. For example, I like to watch Linux distro reviews which are fairly niche and usually only get a few hundred views but my YouTube homepage is filled with them. In contrast, when I was testing out TikTok it mostly recommended me things with broad appeal like comedy skits and generic science videos that I think 90%+ of the population would find at least somewhat enjoyable.
> I also agree that YouTube's recommendations are pretty good. For example, I like to watch Linux distro reviews which are fairly niche and usually only get a few hundred views but my YouTube homepage is filled with them
This is exactly its main weakness. Once it finds one or two things you like, you're hardly ever going to see anything diversity, and if you ever do, it won't be long before that thing completely fills your feed.
I have two Google accounts, and I switch between them for non-YouTube-related reasons. The one is currently all Periodic Videos all the time, the other is all History Matters all the time. It's not as if I lose all interest in chemistry if I switch accounts. If I have a few minutes to kill, I'll end up thinking "History Matters isn't exactly what I want to watch, but I don't want to spend ten minutes trying to find something else", so I watch it, and the algorithm pats itself on the back and cues up another one of the same.
I just watched the sumptuous Gandhi movie and thought "Nice, now I'll search for some movies about the real Gandhi". After typing "Gand" in the remote control, there was Gandhi in the 5th, after searches about a more important historical figure: Gandalf.
The first video was titled in big letters "Was Gandhi a racist who spanked women?".
I’d say it’s more likely they have super advanced/clever ways of doing the latter. The algorithm could be a simple dot product and the result could be great or terrible depending on how good the feature extraction is.
Pulling useful features out of videos is no small task. The fact that everyone raves about how good the recommendations are indicates to me that this is where their innovation lies.
There's so much good meta-data (likes, comments, duration, sound used, views, like/view ratio, skips, loops, subscribes, etc.) that I'd be surprised if they were digging into the contents of the video at all right now.
They could also be digging only into audio, doing speech recognition on it, then clustering the text. Augment that with the text users have put into the video directly using the in-app editor and you have some pretty solid data.
If that were true, it'd be interesting to see if they push out support for close-captioning. It's an accessibility push, but also would leverage a lot of the same capabilities...
Would this have any advantage over just using video embeddings (or a sequence of frame embeddings?) which in theory should capture those things in vectorized form.
It can be a) very expensive b) also very difficult to implement.
Video understanding is an active field of research and I'm not sure state of the art is there yet for capturing nuance like engagement potential, categories etc.
Google was able to build a very useful search engine that ran for decades relying on the significance of links and keywords, without much understanding of the meaning of page content. You can get very far with the readily available data, before you need to delve into the fancy stuff to make it a few percent better.
That would be the "sound used". The music in the video is specified/labeled before upload so there's no need to actually process the sound of the video.
Maybe I am just strange, but I find no appeal in TikTok, and would not like something that just shows me more of the same things. But I can see how it might appear to people who just want to be stimulated with stuff.
I thought this would be the case for me, but after trying it I've been really impressed with the creativity and content quality. The algorithm quickly figured out I didn't care about teenagers dancing and started feeding me cool crafts and comedy, and my faith in Gen Z has grown dramatically.
Agreed. TikTok has very good built-in video tools, but the creativity and sense of storytelling in short form videos on the part of these kids is very impressive.
exactly, Hacker News is full of great and informative content - some of which I have actually benefitted from. however, I think me and anyone who uses this site must admit much of what is consumed goes in one ear and out the other just like any other media platform like reddit or YouTube. the difference I think is that since so much of the content is "smart", users here would like to think they are spending their time better than others
TikTok's homepage is like trying to figure out what Youtube is by scrolling through Youtube's trending tab. If you didn't know better, you'd think "well, I'm never checking out Youtube again, what the hell was that?"
I thought it was just preteens dancing until I realized my girlfriend uses it for some sort of daily exercise routine, food prep / cooking, and even some sort of podcast-like thing where a conversation is broken up into a bunch of autoplaying short videos.
I still couldn't be bothered to figure it out and use it (though I feel like I should as a self-respecting technologist and app developer who should know what the good people like), but there seems to be more there than the first impression reveals.
Things I've been sent by TikTok in the last few days:
* Cockpit walkthroughs by professional airline pilots
* A first-person plane crash and water evacuation that I had somehow missed from 2014 (and was very interested to see)
* Music theory videos explaining how current pop songs work
* Baking videos explaining how to make things I'd never thought of before
* Dermatology video showing laser skin retexturing in action (its amazing to see, not what I expected); helped me discover a new podcast that I now really enjoy
* Linguistic analysis of English and how certain grammar rules came to be
* A PSA about melatonin dosage (I already knew but discovered recently that most supplements are dosed too high)
* HIIT cardio routines you can do at home
* Planet Money TikTok (actually very solid and educational - e.g. economics of horror movie creation)
I think the site you're talking about is YouTube, which does in fact repeatedly show nonsense and never figures out how wrong it is.
It's a (seemingly) pretty simple matter of keeping track of how long someone looks at a video, and optimizing to show them videos they'll spend a long time looking at. It's pretty powerful too, I spend much more time looking at TikTok than any other digital content aggregator (to the extent I had to delete it from my phone).
They also do some things like sprinkle in random fresh videos, potentially unrelated to your interests, to your "For You" to get exposure to them and a base idea of how long people look at them, which is nice because it potentially boosts small creators to larger audiences.
Further, I'm pretty sure they see what creators are keeping the most people on their platform for the longest, and directly compensate them. This gives rise to a host of "lifestyle accounts" where folks can live doing the things that some chunk of the userbase wants to be doing, and they'll get paid directly from TikTok, rather than needing to source a third party company to sponsor them.
All in all, I think it's fantastically designed -- to the extent I'm not sure it should exist at all.
I've also found that TikTok does a good job of throwing in unexpected videos to test the waters of what the user is interested in. This diversity is one of my favorite things about it.
I wish youtube was better in this regard. The videos it shows me just keep getting more and more focused on what it knows I'm interested in, to the point that it's boring. I discover new things to watch through reddit instead.
Yeah, youtube is super boring. It just suggests my top N most watched channels, even years after I stop watching some of them, as I lost interest in the topic. For example I used to watch lots of vsauce, numberphille, veritasium etc but I gradually lost interest as they slowly morphed from genuine scientific shows into ads for their books or paid shows.
Guess what, youtube still regularly recommends them, even though I rarely click some of these suggestions.
It does for me? The ones which are history driven say "recommended for you" next to them, but most seem to be related to the video itself. Maybe this is due to almost exclusively using it for music.
Yeah, oddly enough all the recommendations are videos I have already watched, even liked. I remember when recommendations on the right side were "similar" music to the one I was listening to, this way I could find music. I cannot do this anymore. :/
I wish the recommendation algorithm had knobs users could manipulate. eg
- show me things that have < 20 views
- reduce the weight of popular channels
- don't send me into a political content feedback loop
- prefer to show me short videos
- show me something wildly outside my tastes every now and then
I just got the Youtube Music app and it had very crude music recommendation options. You have to pick from a pre-selected list of artists. If you click on an artist, it adds another row of tangentially-related artists, but only sometimes.
Suppose you like electronic music, but only niche electronic music. Basically you have to click on Daft Punk, and hope it gives you something closer to what you like, leap-frogging towards e.g. Actress.
It doesn't even let you just type in what you like.., music that you know is already available on Youtube. If I had to guess, that's probably because of stupid copyright shit, but if that's the case, then why can I listen to the probably-violating-the-TOS music in the app if I just search for it myself?? Makes no sense...
The biggest problem in user-facing software right now is that product managers assume they + an algorithm can figure out what you want based on behavior, and that you shouldn't have any direct input in that. The best you'll usually see is an "i don't like this" button.
Spotify sidesteps some of this by just giving you a bunch of different algorithms feeding different playlists, so at least there are options, but I miss having settings to tweak to gain more control, or to point it in the right direction better.
> The biggest problem in user-facing software right now is that product managers assume they + an algorithm can figure out what you want based on behavior
Given Tik Tok's success, this appears to be a correct assumption.
For every 1 engineer who wants ten different knobs to use the app like a power user, there are 100 or more regular users who don't want anywhere near that much complexity.
Catering to the power users only makes sense if they're your core market. For most public-facing apps, especially free apps, creating and maintaining power user level controls is far more trouble than it's worth.
TikTok appears to let their algorithm do a lot more random walking than other companies. That's in some ways different from power user knobs, but it serves a similar purpose in exploring a broader area instead of assuming they can intuit everything from previously-collected data.
That's an improved default that fits well for their app. But infinite scroll is not the only use case in the world.
This assumption that "what works for [one app] is the path I should follow because that app has been successful" is frustrating and shortsighted.
Catering to the power users makes sense if you care about your users. Power users aren't born, they're made - through repeated use, and space to grow. "Power user features" are that: space to grow better in using a tool.
This isn't happening on social media (except one notable place: interfaces for advertisers), because users there are to be turned into addicts and exploited. With such a design goal, it's reasonable you want to take away user's agency - you want them "engaged" in your "experience", and them having options detracts from that.
I'm serious, by the way, this is not just a "FB/IG/YouTube/TikTok are evil" rant. Their whole design is created around hooking users up. If you wanted to design an efficient, ergonomic app for sharing media content with friends or an audience, these social media platforms are almost the opposite of what you want.
It's part of the whole ~30yr trend of infantilizing adults and assuming you know better because you're the professional. Nobody believes people can make their own decisions about want they like and dislike anymore.
I hate the modern software trend of disallowing explicit filtering. On everything from Youtube to Netflix the user is limited to typing in what they want and hoping the algorithm correctly reads their mind. If it doesn't, you're left trying to trick it into searching on what you want or looking for an outside resource where another person has already done the grunt work.
A common comment I see on Tiktok videos is something like "I've finally made it to [niche interest] tiktok", it's something I've experienced myself when the algorithm started showing me videos from the crew on cargo ships as they talk about their favorite ports to dock at, or their experience going through E.G the Panama Canal. I don't know any other platform that emergently figures out that I have an interest in international logistics without me specifically searching for it, and it's one of the reasons TikTok is such a great platform for content.
The exact same thing happens for me on YouTube, although I guess YouTube doesn't start with an auto-playing random sequence of videos to quick-start the search for videos you find interesting. Still though, I end up in very odd niches that I never specifically searched for.
One thing I will say is that I feel YouTube doesn't do this to nearly as large of degree as TikTok. YouTube might suggest me videos that are outside my cluster of interests occasionally, but normally only those with a reasonably high view count. They will recommend low view count videos that are within my interests however.
TikTok on the other hand will sometimes show me a video with almost no views, that is completely random, and I actually do like it. I feel like YouTube has gotten slightly stale in that it tries so hard to show me either viral videos or videos it deems to be of my interests, that the content becomes repetitive. In recent months it's become better at it, but still plays it safe with recommendations.
Wikipedia has Random article for ages now. All TikTik did was put on a nicer UI. Of course UI are important most user are suckers for that stuff. However I found there really aren't any new ideas left just implementations.
I've been seeing a lot of low to mid-tier creators dropping out of the payment program lately after their view counts plummeted. It's speculated that TikTok only pays for video completion count while enrolled in the Creator Fund since view counts tend to go back up after leaving.
Having more leverage with third parties outweighs the tens of dollars they'd receive from a video, I guess.
I actually think it's got much more to do with the training data.
Given that you are watching a video, you need to either swipe away from it, or finish watching it. This provides either a 1 or 0 for the video classification model.
The important contrast here is with FB/IG feed where you can scroll aimlessly without engaging, leaving you with perhaps 1 engagement out of 10 (or whatever).
The attached doc suggests that they are only using unsupervised learning, which I find very hard to believe, to be honest.
I wonder if for random content discovery, unsupervised learning methods are less likely to be overconstrained by independent variables, and hence are freer to make better open-ended recommendations.
Supervised algorithms almost assume too much about a user -- they assume the correlation structures that are true in the past are also likely to hold in the future. This assumption holds in deterministic environments, but is false or unnecessarily constraining in stochastic domains, where it is widely known simple models and heuristics tend to perform much better.
I feel "new content discovery" is more of a stochastic problem rather than a deterministic one (which is the environment most conventional recommendation engines dwell in, hence most rec engines only make conservative recommendations).
For all we know TikTok's algorithm could simply be a combination of rules of thumb + simple clusters + randomization that happen to work well. I've seen so many instances (in real life) where simple models vastly outperform complex models in stochastic situations.
(p.s. in my opinion, YouTube's recommendations tend to be rather on the deterministic side. My recommendations seem to be mostly based on what I've watched and liked in the past and so the recs tend to be a bit boring. It clearly works great for YouTube from a monetization perspective, but it doesn't unearth a lot of interesting new content for me -- I have to search for those.)
Reducing engagement data to 0/1 would lose a lot of precious data, it treats getting 50s through a 1m video worse than watching the whole of a 5s clip, and watching a 2s clip the same as watching a 2s clip on loop 20 times because it's so incredible. Given the point of the app is to get people on it for a long time, it would make much more sense to track number of seconds watched and train to maximize that.
Dwell is a very powerful ranking signal in a binary classifier.
Pairwise association of videos watched by the same user consecutively or even just sampled pairs from their last N videos will get you a video embedding.
Pairwise sampling of users who watch the same video to the end will get you a user embedding.
Turking category tags will prime the pump for other types of embeddings.
These things can be ensembles, stacked, force learned jointly, etc.
All of this comes out of the box in Keras (though it’s up to you to feed the data in fast when you’ve got a lot office).
You can argue that getting latent representations/factorizations without explicit “user clicked show me more” is semi-supervised I guess, but if so all the recommender stuff since the Netflix Challenge meets that criteria to one degree or another.
You don't have that equivalent signal, that's what matters. i.e. Twitter doesn't know if you even finished reading the tweet which is what is providing a strong signal in the case of TikTok, skipped vs watched once vs watched multiple times.
Facebook and Twitter are social-graph subscription-based services. TikTok just gives recomendations without subscriptions while knowing nothing about my social graph.
But yeah, it appears (from TikTok's success) that the social graph may not be particularly useful for content-based recommendation (which really, we should probably have realised given the existence of Google).
> The attached doc suggests that they are only using unsupervised learning, which I find very hard to believe, to be honest.
Where are you seeing that?
I see:
> Recommendations are based on a number of factors, including things like:
> User interactions such as the videos you like or share, accounts you follow, comments you post, and content you create.
> ...
and
> A strong indicator of interest, such as whether a user finishes watching a longer video from beginning to end...
The idea to sprinkle in different videos is just for the algorithm that the bias is not more justified: You like X, i feed you only X, i learn that you absolutely love X.
That was the overall trend in YouTube since 2013 that makes sense for YouTube as a business but makes me a bit sad as a user. When they started to split up YouTube picked out the top, corporate friendly creators to let advertisers pick and choose how to target their (eventually content self-censoring) advertising money rather than offering YouTube as a whole bundle and letting the keyword auction do its work opaquely. This creates to inevitable incentive problem down the road we're all witnessing now such as asymmetrical content id powers, no monetization for grassroot creators, content censorship etc.
> It's pretty powerful too, I spend much more time looking at TikTok than any other digital content aggregator (to the extent I had to delete it from my phone).
Similar experience here. One night a few months ago my girlfriend and I decided to download it as a joke. We laughed at some videos for a while and went to sleep. A couple months later she'd spend hours upon hours every day looking at it, even foregoing some of her hobbies. Eventually she realized it was too much and also uninstalled it.
> It's a (seemingly) pretty simple matter of keeping track of how long someone looks at a video, and optimizing to show them videos they'll spend a long time looking at.
It just push the creators to try to keep the viewer the longer by pushing the actual content further and further into their 1 minute.
You can opt out of the algorithm by... not using the app, or just not using the 'for you' page and searching for every video you'd like. But even then, all of the platforms you list have billions or trillions of posts on their site, so there would be no way to opt out of algorithmic listing since something has to decide what posts show up at the top of the list. The only service I know of that actually allows you to do that is Twitter with tweetdeck, where searches can show a stream of new posts.
Fair and not an unreasonable comment. Regardless, I do think there should be a non-personalized option to at least fallback just as some sort of metric of a control group.
The ethics behind these black box bubbling users is the dirty dark secret of tech companies.
I don’t understand why more people are not concerned about it running wild (especially when kids sit on these apps all day non-stop)
I think your tone is a little abrupt, but I'm all for laws that mandate, at the very least, high level descriptions of data processing algorithms.
A bit of random adjunct, but if I buy a piece of KFC, how would I know if I'm allergic to one of the 11 herbs and spices, if the company isn't legally obliged to reveal it?
Likewise, how would you know if a company is processing data for a nefarious end, if you don't know how it's processed?
I know alruism seems unlikely, but how can you be so sure?
How TikTok's "algorithm" works is pretty obvious if you've spent any time on there.
It learns what you like by how much of a video you watch and how you interact with it, and it establishes some kind of weighted feature vector based on hashtags used in the description, words used in the comments, text drawn on the video, audio background, possibly some audio transcription of words said, if you commented on it, if you forwarded it to a friend, and so on. There may be some network based recommendations, based on who you follow but those seem to be weighted very weakly, and that makes sense if you're trying to keep the platform from getting botted.
It also seems to do some non-dominant sorting to keep from only showing you things from the same type of video.
Facebook's competing short video service is terrible in comparison. It only wants to show me Trevor Noah clips, and things tangentially related to things I may have "Liked" 12 years ago.
To be fair, the algorithm they told is mostly what we would have perceived, knowing if that is Machine Learning driven.
Although, I am quite intrigued what they would be showcasing in their "transparency centers". If they show what extent their data can be utilized. and not simply a simulation of the ML stuff.
I think it's interesting how they took the warnings on creating feedback loop "bubbles" and social media addiction causes and used that as their business model.
It's the logical result of where we are going as a society.
World of Warcraft may not have started out as a Skinner Box. But they realized what they had and used it to their advantage. The result is that all games today are casino-ified. Lootboxes, microtransactions, abstracted currency, etc. It all comes from Vegas.
TikTok and apps like Robinhood are the bleeding edge of this.
What I find strange about TikTok are the waves of themes I get to see. Being a 50-something I'm probably not the target audience but at first all I saw was shuffle dance (enjoyed that) followed by Indian and Chinese manufacturing videos (loved those) and now it's all cats (okay with that).
My daughters explained "Elite TikTok" and "Beans TikTok" so now I realize there's a whole world built on top of gaming the algorithm in order to bolster your insider status.
The recommendation algorithm generally tries to avoid showing directly similar content. It's surprising it is all cats especially if you ever swipe. Perhaps this is an A-B test and you are in a group with lower boredom avoidance settings :)
These articles explain it a bit better but what it boils down to is that a group of kids create strange content around a theme (beans, frogs, etc.) and then try to go viral within their peer group, e.g. other kids that are too cool to like what everyone else likes. Impersonating real brands is also a big part of it.
i had an account for a few weeks.. followed a few folks.
However my assessment is a video has much less revisit value than still images. The nature of a video requires much more time investment. Audience has to remain focused on a video for a period of time to achieve full reward. Unlike a still image, where a quick glance can gain satisfaction.
These platforms can not sustain on pure novel content. The model of user engagement must include revisit of content. You see a thumbnail of a video you've liked or have seen before. What is the threshold of intrigue, that will propel you to invest x amnt of time fully dedicated to that video content?
If my opinion above holds true, tiktok will not be a lasting phenomenon. The content will not keep up.
None of these problems your listing are relevant to TikTok at all. You don't seem to be familiar with how TikTok works or how immensely popular is world wide.
> However my assessment is a video has much less revisit value than still images.
I would say the same, only for text versus images. and yet for some reason I'll never understand besides being only in my early 30ies, people prefer images. and videos even more.
text and videos, I group into the same category due to the full consumption of the content is over a period of time that is much greater than an instant. Consuming text obviously is a much less passive pattern than video, so they diverge greatly in that respect.
Images, for the mass consumer audience, require orders of magnitude less time to fully consume.
The dynamic is very much different.
The Tiktok vs Vine comparison is very apt.. there is hardly any differences between them in terms of the process of content creation, and how compelling they are to a mass market audience
Actually, yes. This particular generation will eventually grow out of TikTok like they did with other social networks at their peak of the hype cycle: Snapchat (2014), YikYak (2014), Vine (2015), Shots (2012) and SNOW (2016) etc.
TikTok is no different and yet another hype masterpiece for the record books and the news media. The new kids will find another one to replace it and suddenly it will be crowned as 'cool'.
I didn't see anything in there about how their algorithm prevents people from seeing videos involving poor people[1] or Hong Kong protests[2]. Maybe that's part of the secret sauce.
TikTok is like a highly compressed version of YouTube.
YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out.
1 minute of content surrounded by 9 minutes of filler.
TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's no incentive to create filler - just the opposite.
Of course the downside is that there's only so much you can fit into a 60 second package, so you're not going to get a deep dive into anything. But for the kind of content that can be compressed like that, TikTok wins big time.
I watch a lot of YouTube and haven't really noticed any artificially long videos in my bubble of channels. I agree that intros are annoying (I have no idea why people do them), but other than that... it's just the usual video stuff. The nature of the format is that a one page blog post becomes a 10 minute video. People talk slower than they read, so it's somewhat unavoidable. There are also YouTubers that are really into the long form video essay format (hello, ContraPoints), but I don't think they're doing that to milk as much ad revenue as possible.
But again, that is just the random subset of YouTube that I watch. I doubt any of it is that popular, and I am always shocked when I get logged out and am on the default main page. So my experience could be totally off-base.
YT channels with frequent update formats are 10-20% filler with intro/endcard/credits/sponsors. Most talk slow enough that they can be comfortably watched at 2x speed. Channels like contrapoints - irregular long form content that's usually demonetized and funded via patreon doesn't waste your time as much. But they're still paced differently than TikToks which bias towards brevity. There's still some tedious videos... lots of old codger maker content that takes the full minute to show 10 second worth of content, but from my experience, that's still shorter than sitting through youtube filler.
Yeah, I'm making a generalization about the platform and what it incentivizes. It's definitely not to say that all channels create filler.
Being logged in and curating really makes all the difference. ContraPoints, CGP Grey, Joe Rogan, Kurzgesagt - I think that kind of content shines on YouTube, isn't possible on TikTok, and don't see them as artificially long.
I don't find much filler among the areas of interest that I specifically follow on YouTube (some of that of course being due to my own selection of who to subscribe to), but I do find that filler on YouTube when I'm searching for some other instructional information on Google.
Really? I feel like I could easily most of the videos I watch and still understand what the video is about.
The first few minutes are the intro, a message that the video is sponsored and we will find out about them later, last 3/4 minutes is always the "Like, Subscribe" and a sponsorship message usually. This means that both the front-and the back can be cut out easily without any affect on the actual content.
I agree. I’m slowly starting to realize that a lot of videos are filler myself. Because there are so many videos out there and I find they are often too long, I just skip skip skip ahead until they get to the point. That or just drop out of the video to find another one that does get to the point.
> TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's no incentive to create filler - just the opposite.
It's to a point that I skip any TikTok video that starts like “I was in the forest and found this mushroom. Did you know that this kind of mushroom can be used to... ”. I skip it as soon as I hear “I was in the“ (that and the usual tone that comes with it).
I don't care. Show me directly what you are doing with the mushroom.
Yeah I wish TikTok has variable speed / fast forward option. Another factor for ML to train and automatically speed up to the juicy bits. There's a lot of room to waste even in 1 minute formats. Though TikTok itself is mostly time wasting and this remark is more an indictment of my attention span.
In flash fiction (i.e. very short fiction), the same elements exist, but due to brevity constraints, it starts the story in the middle, i.e. the point of conflict or climax, and then resolves the other parts along the way.
I think creators who understand how to exploit this structure could potentially make some really compelling short videos.
I don't really think so (at least regarding my tiktok usage)
Being thrown right into the mushroom thing, into the action, forces me to keep focus and attention. The pseudo-introduction is just dulling my mind while not bringing anything of consistence.
I had some tiktoks that started like "So I read something incredible the other day and I just had to try it. I am gonna show you what happened when I decided to eat an orange under the shower". Don't care. Starts with you eating the orange.
As another sibling commentator wrote it's a different narrative structure.
> Being able to get consistent, instant stimulation from doing effectively nothing seems dangerous to me.
The same could be said from books but I get what you mean. I still read books, so I guess I am okay (it could be worse !).
edit: to be fair, I have very low tolerance and patience for people who beats around the bush :/. On the other hand, when walking outside I can stop and look around me for 2 or 3 minutes every 100 meters because there's always some piece of architecture or natural landscape to look at.
I wish I had data on the average length of youtube videos over time. I swear 5 years ago an explainer video or a game review would be less than 5 minutes consistently, which is what I personally want from a platform like youtube, a digestible snack of information. Nowadays career youtubers are often pushing well over 10 minutes which just occupies too much of my time to be worth the distraction.
Can someone explain to me why this comment was flagged dead? It's also my understanding that the inflation of video times to certain intervals was driven by how many ads were allowed at a given video length.
I used to watch YouTube now it is the place of last resort for any content or explanation because its too slow. I once had to spend 9 minutes watching a video about how to get a tricky gasket out of a spot on my truck. The short answer is that you take a 3/4 inch threaded pipe twist it into the gasket and pull down. The long answer was a 9 minute video. Almost always, when a link takes me you YouTube, I just assume I don't need to know that bad. Its like a recipe site that can't be scrolled or searched.
I really moss when people wrote what they did in little snpits on blogs.
> YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler.
Given how popular obnoxious jump cuts are on YouTube, I would assume this wasn't the case. What part of YouTube incentivizes this?
I always attributed the increased intros/outros/ad spot/filler lengths to the general trend of YouTube getting more "professional" than it did 10 years ago.
Advertising placements. If a video is longer than eight minutes (formerly ten minutes), it's eligible for "midroll" ads and will earn the creator more money.
Historically YouTube would downregulate short videos in their algorithms. The cutoff was 10 minutes, if I remember correctly. So the biggest content creators got used to stretching out any video to at least 10 minutes. I think YouTube has relaxed that a bit recently.
Which is funny in the greater historical context, because in the early days of YouTube you couldn't upload videos longer than 10 minutes.
EDIT: As an aside, YouTube's algorithms also punish you if you aren't regularly uploading content. It was really a perfect storm of shitiness where content creators needed to pump out content constantly, while also ensuring those videos were all 10 minutes or longer. It drastically affected the YT landscape. Content creators like Noah Caldwell Gervais who only sporadically release content because they spend HUGE amounts of effort on what they do release were getting slammed by the algorithm. Again, the algorithm has gotten a little better in that regard over the years. Though it's gotten worse in other ways.
> incentivizes creators to make artificially long videos
One of the reasons I use a FF addon to make the default speed of all youtube videos 1.5X. So few v-bloggers are succinct and fluid enough to have to listen at 1X.
Generally when I stumble upon a minute long video, it's either really, really good or clickbait that's completely uninteresting. It tends to be the latter. The sweet spot for videos in my opinion is 15 seconds or so. Short enough that I can't possibly get bored but long enough to have a meaningful tidbit of content.
Finding subsequent parts of a multi-part series generally isn't worth the effort.
The real threat is that it allows China to get a foothold into the international tech game, and thus allows them to become more powerful in comparison with their adversaries.
Which is a legitimate threat to some people, but it's not that much of a threat to the common man.
It s a company the US can easily block without impact in other business. Instead shutting down huawei or other big chinese companies causes major reshuffling
Its the first tech social app i.e. Facebook, MySpace and the like from China, a country that is not America. That is all there is to it. America realizes that China is catching up, and considering there are a billion Chinese, it is inevitable that China will be a foremost power that either competes with or simply overtakes America.
The current foremost power, America, doesn't like that.
At the very least, this whole thing has made clear that principles that America repeatedly espouses are forgotten quite quickly when it wants to retain power.
Why should principles be given to countries that do not reciprocate? All major American tech companies are banished from China, and the only ones left are ones who agreed to transfer IP to be stolen by the CCP.
The US never had any such principles. Whenever any country competes with American cash-cows, the US "discovers" a reason to ban them. TikTok is not the first one, see what happened with Bombardier for an example closer to home.
Or, when Canada was threatening the US superiority in engine tech, after which the US government threatened to embargo Canada from avionics if they proceeded with the Arrow program. They do the same with the Swedish Gripen, they put an embargo on their avionics if Sweden sells them to a country which is also considering the F-35.
American tech compamies are banned from China unless they store data on Chinese servers and comply with Chinese law; there is no need to transfer IP. Do you think Google would have transfered Search IP under Project Dragonfly? Do you think Facebook has given their Ad IP?. Why wouldn't the US use the same standard of requiring full compliance with US law as enforced by a liaison and data being on US soil?
Because the question isn't of reciprocity, it is of dominance. That's why TikTok was banned under the Joker of "National Security", not under any other standard. The US is simply scared of losing it's advantage in tech to China. Which of course will likely happen anyways.
In any case, the average person isn't harmed by TikTok anymore and indeed less so than Facebook.
>Chinese law requires companies to hand over root access and data, and also effectively forbids E2E encryption. This is IP transfer. [2]
By that standard the US also requires IP transfer. Have you already forgotten about PRISM?
>The US has several tech companies with market caps more than double Tik Tok.
Sure, but if China continues in this trajectory then it won't be for long
>Depends if you trust your data with the same government who regularly disappears journalists, I guess.
Depends if you trust your data with the same government who regularly fabricates evidence as an excuse to go and kill a million or so people every once in a while. Or the one that regularly assassinates or threatens to assassinate activists, I guess. The chance of your own government killing you is at least four or five orders of magnitude higher than that of China killing you, same for most other metrics of harm. Can't say the same if you're a foreigner with the US government, though! That being said, US control of the press is much more sophisticated, so there is really no need to kill any American journalists except in exceptional cases. But don't delude yourself, if it ever came to that there would be no hesitation - look at what the USG did to Fred Hampton of Gary Webb, or what they tried to do with Martin Luther King.
>There's evidence Tik Tok has been promoting CCP propaganda. [3]
There's a difference between people on TikTok defending China and TikTok conspiring to push Chinese propaganda. There is evidence for the first one, and absolutely no evidence of the latter. There's people who defend China and push propaganda on Facebook and Instagram too, if you go looking for them. And there's also American propaganda on TikTok.
> Why should principles be given to countries that do not reciprocate?
I don't think Principles are really given. You have principles and you can abide by them.
> All major American tech companies are banished from China
Nope. They aren't banishing anyone. They simply have laws that make it difficult for them to work in China. What law was TikTok not following that they are being forced to sell to a US company? Its essentially extortion.
If the US makes a law, i.e. all foreign-owned companies must store data on US soil or whatever else the issue with TikTok is, then that would actually fit. Rather the administration is forcing Bytedance to sell to a US company i.e. give it to us or we will kill it.
Does anyone know of an open source recommendation engine?
I would love to just explore how one works bet it seems like this solely in the realm of proprietary software at the moment.
I have a dream of naively making a open source recommendation engine using data from good reads and imdb.
There is a real utility here that I worry will be captured entirely by companies without an open source alternative.
TLDR: We do some math
https://xkcd.com/1838/
On a more serious note I think there is a lot of sophistication and that is being left out from this very simplified explanation. To say something like 'we show things from other clusters every so often' over looks so many questions about how far apart the clusters are, and how often these are shown. These values are foundational to the UX and understanding how to tune them should get a lot more focus than it does. In the future it would be good for the AI community and the associated HCI researchers in AIUX to focus on how these settings change the experience of a ML pipeline.
This is not an algorithm; this is a heuristic. An algorithm is (loosely) a method or procedure for achieving some specified end.
"... avoid redundancies that could bore the user, like seeing multiple videos with the same music or from the same creator" is the goal, and they have heuristics to try to work towards that, and algorithms and software that implement those heuristics.
I think the ship has sailed on this, but when you are in circles where both heuristics and algorithms are in play, this blurring of lines makes for very confusing conversations.
I find that it takes device into account very interesting. What does me using an iPhone XS vs a Samsung Note vs a generic cheap Android phone say about me? I wonder if ends up being a proxy for income, age, or gender...
The main reason for me why YouTube is out of choice is too many Ads interruptions and absence of Portrait mode video support. Even Telegram supports Vertical videos - you can zoom into any horizontal video, and I can move that window so that I can watch full screen on my phone.
What's been funny is how there's been so many trends that relate to exploiting the engagement stats. For a long time, there were a lot of videos begging for likes or claiming that the heart would be purple on the particular video; this seemed to lead to a de-emphasis of likes. There's a lot of videos/sounds that involve a long build up leading to a short reveal, making sure the viewer finishes the video. One-frame image reveals encourage downloads and replays, and content hidden behind the interface can often lead to many downloads. Videos explaining how to repeatedly hit the share link button will sometimes have more shares than views. There's endless numbers of alternative spellings of words like sex and porn to avoid the edgy content filters, but I suppose that's a given. The hashtags are weird to me since #xyzcba seemed to actually have an effect for a while.
The things I didn't know in this article were the stuff about device type (what is that used for?) and the initial 8 videos. Perhaps the next trend I'll see is flexing on having an unlocked, AT&T-branded Samsung G892A.
Devil is in the details, a lot of which seem to have been lost in this explanation. This seems to be mostly a view into their value model (ie prioritize P(Click) * value + P(Heart) * value etc), but the hidden issue arise from the type of content that is likely generate clicks / hearts / long watch times.
I think a huge reason, I didn’t see in the article, that TikTok is able to train their ML to be far more effective than any other video platform out there has to do with the size of those clips and how easy it is to flip through them.
If we are watching 30 second clips, flipping through them quickly, that’s 2 signals a minute and 20 signals every 10 minutes (the average size of a YouTube video) and that is assuming we watched 20 videos, we might have flipped through 60 in 10 minutes.
That’s 60 signals where YouTube would’ve gotten 1 signal. Whose ML will be better trained with your preferences? Obviously TikTok, I don’t think there is anything special about their algorithms it’s just a function of their product and the type of content.
189 comments
[ 2.7 ms ] story [ 217 ms ] threadRumor is that the actual secret sauce here is human curation; people hand-select videos with high appeal and label them "viral", "popular", etc. in order to astroturf eyeballs and clicks. I suppose that admitting this directly would contrast sharply with the Chinese-harmonious-technocracy veneer that they work to project.
* although with popularity of TikTok, one needs to wonder if that team have become THE trendsetters themselves...
They had teams of editors for Trending. Then, in 2016, conservatives complained that there was a liberal bias (which I don't really think there was, except obvious lies were filtered out). The editors were then removed, with the result that Pope Francis endorsed Trump a few days before the 2016 election.
Funny how things work out, I guess.
but I wonder why I end up in YouTube holes with bottomless pits though until I hit the reset button.
Tictok and Toutiao in China often push irrelevant content or new content by new creator to users to give them potentially interesting things outside their usual sweet spot and also to give new creators chances to win followers. They are exceptionally good at this.
"We're a 2-year-old company operating with the expectations of a 10-year-old company," said Michael Beckerman, TikTok's vice president in charge of U.S. public policy. "We didn't have the opportunity to grow up in the golden years of the internet, when tech companies could do no wrong. We grew up in the techlash age ,where there's a lot of skepticism of platforms, how they moderate content and how their algorithms work.""
Well, yes, Compete or die. It's the same as a tobacco company start up.
Oh, wait, that's Juul.
What happened to them?
Though, you can very much argue that Juul is both Tech & Tobacco. But, imagine the same statement from Juul.
"Once TikTok collects enough data about the user, the app is able to map a user's preferences in relation to similar users and group them into "clusters." Simultaneously, it also groups videos into "clusters" based on similar themes, like "basketball" or "bunnies.""
Although I do wonder, and maybe someone else with more experience could shed some light here, whether or not it is likely that TikTok has some fundamentally super advanced algorithm, or if they just do a better job of collecting data/training & evaluating their models?
Probably why multi-part videos are so popular (you have to click their profile then find the part 2 video to finish the story they were telling).
This is on top of the documented effort by YouTube to perfect their recommendation algorithm using the best ML minds they got [1] only to polarizing response from its users.
Clearly tiktok has other advantages (homogeneity in some content characteristics, viz. Extremely short videos which probably correlates to their simplicity) and has clearly tuned a fundamentally better recommendation algorithm that even the minds at Google brain couldn't figure out.
[1] https://www.theverge.com/2017/8/30/16222850/youtube-google-b...
If you don't quickly find something you'd like to watch on YouTube you're very likely to leave and find something else to entertain you. Meanwhile TikTok is automatically feeding you an infinite source of quick and easy to digest content, all of which you'll probably like to some degree.
YouTube could do something similar and give users an automatic continuous feed (ala a TV channel), but I think YouTube's content is much too longform to work well in this format. This burden of choice problem affects Netflix in the same way which also has superb recommendations.
This is exactly its main weakness. Once it finds one or two things you like, you're hardly ever going to see anything diversity, and if you ever do, it won't be long before that thing completely fills your feed.
I have two Google accounts, and I switch between them for non-YouTube-related reasons. The one is currently all Periodic Videos all the time, the other is all History Matters all the time. It's not as if I lose all interest in chemistry if I switch accounts. If I have a few minutes to kill, I'll end up thinking "History Matters isn't exactly what I want to watch, but I don't want to spend ten minutes trying to find something else", so I watch it, and the algorithm pats itself on the back and cues up another one of the same.
The first video was titled in big letters "Was Gandhi a racist who spanked women?".
Make no evil. Duh!
Pulling useful features out of videos is no small task. The fact that everyone raves about how good the recommendations are indicates to me that this is where their innovation lies.
Using ML it is very easy to tag videos.
Why would you be surprised to learn TikTok is doing video content analysis?
Video understanding is an active field of research and I'm not sure state of the art is there yet for capturing nuance like engagement potential, categories etc.
Me neither. Good thing TikTok provides variety and helps users discover new things.
> But I can see how it might appear to people who just want to be stimulated with stuff.
Don't we all? Is that not why we're on Hacker News, to find stimulation targeted more specifically to us than other websites?
I thought it was just preteens dancing until I realized my girlfriend uses it for some sort of daily exercise routine, food prep / cooking, and even some sort of podcast-like thing where a conversation is broken up into a bunch of autoplaying short videos.
I still couldn't be bothered to figure it out and use it (though I feel like I should as a self-respecting technologist and app developer who should know what the good people like), but there seems to be more there than the first impression reveals.
* Cockpit walkthroughs by professional airline pilots
* A first-person plane crash and water evacuation that I had somehow missed from 2014 (and was very interested to see)
* Music theory videos explaining how current pop songs work
* Baking videos explaining how to make things I'd never thought of before
* Dermatology video showing laser skin retexturing in action (its amazing to see, not what I expected); helped me discover a new podcast that I now really enjoy
* Linguistic analysis of English and how certain grammar rules came to be
* A PSA about melatonin dosage (I already knew but discovered recently that most supplements are dosed too high)
* HIIT cardio routines you can do at home
* Planet Money TikTok (actually very solid and educational - e.g. economics of horror movie creation)
I think the site you're talking about is YouTube, which does in fact repeatedly show nonsense and never figures out how wrong it is.
They also do some things like sprinkle in random fresh videos, potentially unrelated to your interests, to your "For You" to get exposure to them and a base idea of how long people look at them, which is nice because it potentially boosts small creators to larger audiences.
Further, I'm pretty sure they see what creators are keeping the most people on their platform for the longest, and directly compensate them. This gives rise to a host of "lifestyle accounts" where folks can live doing the things that some chunk of the userbase wants to be doing, and they'll get paid directly from TikTok, rather than needing to source a third party company to sponsor them.
All in all, I think it's fantastically designed -- to the extent I'm not sure it should exist at all.
- show me things that have < 20 views
- reduce the weight of popular channels
- don't send me into a political content feedback loop
- prefer to show me short videos
- show me something wildly outside my tastes every now and then
I just got the Youtube Music app and it had very crude music recommendation options. You have to pick from a pre-selected list of artists. If you click on an artist, it adds another row of tangentially-related artists, but only sometimes.
Suppose you like electronic music, but only niche electronic music. Basically you have to click on Daft Punk, and hope it gives you something closer to what you like, leap-frogging towards e.g. Actress.
It doesn't even let you just type in what you like.., music that you know is already available on Youtube. If I had to guess, that's probably because of stupid copyright shit, but if that's the case, then why can I listen to the probably-violating-the-TOS music in the app if I just search for it myself?? Makes no sense...
Spotify sidesteps some of this by just giving you a bunch of different algorithms feeding different playlists, so at least there are options, but I miss having settings to tweak to gain more control, or to point it in the right direction better.
Given Tik Tok's success, this appears to be a correct assumption.
For every 1 engineer who wants ten different knobs to use the app like a power user, there are 100 or more regular users who don't want anywhere near that much complexity.
Catering to the power users only makes sense if they're your core market. For most public-facing apps, especially free apps, creating and maintaining power user level controls is far more trouble than it's worth.
That's an improved default that fits well for their app. But infinite scroll is not the only use case in the world.
This assumption that "what works for [one app] is the path I should follow because that app has been successful" is frustrating and shortsighted.
This isn't happening on social media (except one notable place: interfaces for advertisers), because users there are to be turned into addicts and exploited. With such a design goal, it's reasonable you want to take away user's agency - you want them "engaged" in your "experience", and them having options detracts from that.
I'm serious, by the way, this is not just a "FB/IG/YouTube/TikTok are evil" rant. Their whole design is created around hooking users up. If you wanted to design an efficient, ergonomic app for sharing media content with friends or an audience, these social media platforms are almost the opposite of what you want.
TikTok on the other hand will sometimes show me a video with almost no views, that is completely random, and I actually do like it. I feel like YouTube has gotten slightly stale in that it tries so hard to show me either viral videos or videos it deems to be of my interests, that the content becomes repetitive. In recent months it's become better at it, but still plays it safe with recommendations.
the answer is always whatever port your at on crew change day.
Having more leverage with third parties outweighs the tens of dollars they'd receive from a video, I guess.
Given that you are watching a video, you need to either swipe away from it, or finish watching it. This provides either a 1 or 0 for the video classification model.
The important contrast here is with FB/IG feed where you can scroll aimlessly without engaging, leaving you with perhaps 1 engagement out of 10 (or whatever).
The attached doc suggests that they are only using unsupervised learning, which I find very hard to believe, to be honest.
Supervised algorithms almost assume too much about a user -- they assume the correlation structures that are true in the past are also likely to hold in the future. This assumption holds in deterministic environments, but is false or unnecessarily constraining in stochastic domains, where it is widely known simple models and heuristics tend to perform much better.
I feel "new content discovery" is more of a stochastic problem rather than a deterministic one (which is the environment most conventional recommendation engines dwell in, hence most rec engines only make conservative recommendations).
For all we know TikTok's algorithm could simply be a combination of rules of thumb + simple clusters + randomization that happen to work well. I've seen so many instances (in real life) where simple models vastly outperform complex models in stochastic situations.
(p.s. in my opinion, YouTube's recommendations tend to be rather on the deterministic side. My recommendations seem to be mostly based on what I've watched and liked in the past and so the recs tend to be a bit boring. It clearly works great for YouTube from a monetization perspective, but it doesn't unearth a lot of interesting new content for me -- I have to search for those.)
It's a HN comment, not an in-depth blog post :)
Pairwise association of videos watched by the same user consecutively or even just sampled pairs from their last N videos will get you a video embedding.
Pairwise sampling of users who watch the same video to the end will get you a user embedding.
Turking category tags will prime the pump for other types of embeddings.
These things can be ensembles, stacked, force learned jointly, etc.
All of this comes out of the box in Keras (though it’s up to you to feed the data in fast when you’ve got a lot office).
You can argue that getting latent representations/factorizations without explicit “user clicked show me more” is semi-supervised I guess, but if so all the recommender stuff since the Netflix Challenge meets that criteria to one degree or another.
Yet people don't spend hours per day browsing tweets.
What is it about tiktok which is so much more effective?
But yeah, it appears (from TikTok's success) that the social graph may not be particularly useful for content-based recommendation (which really, we should probably have realised given the existence of Google).
Where are you seeing that?
I see:
> Recommendations are based on a number of factors, including things like: > User interactions such as the videos you like or share, accounts you follow, comments you post, and content you create. > ...
and
> A strong indicator of interest, such as whether a user finishes watching a longer video from beginning to end...
which indicates the opposite
https://newsroom.tiktok.com/en-us/how-tiktok-recommends-vide...
So like, what Adsense did to the internet and exactly what Youtube is trying to undo.
Similar experience here. One night a few months ago my girlfriend and I decided to download it as a joke. We laughed at some videos for a while and went to sleep. A couple months later she'd spend hours upon hours every day looking at it, even foregoing some of her hobbies. Eventually she realized it was too much and also uninstalled it.
It just push the creators to try to keep the viewer the longer by pushing the actual content further and further into their 1 minute.
At this scale, TikTok, Google, YouTube, Twitter, Facebook.. it should legally be required to either:
- Open Source It
- Provide clear option to opt-out
- Provide the params to be 100% configurable
Wake up people!
The ethics behind these black box bubbling users is the dirty dark secret of tech companies.
I don’t understand why more people are not concerned about it running wild (especially when kids sit on these apps all day non-stop)
I’m just imagining that scenario and companies choosing to “opensource” their obfuscated/assembly code.
I would say there's a very good option to opt-out of TikTok - don't use it.
Facebook is much harder to opt-out, as they can track you even if you don't have an account.
A bit of random adjunct, but if I buy a piece of KFC, how would I know if I'm allergic to one of the 11 herbs and spices, if the company isn't legally obliged to reveal it?
Likewise, how would you know if a company is processing data for a nefarious end, if you don't know how it's processed?
I know alruism seems unlikely, but how can you be so sure?
It learns what you like by how much of a video you watch and how you interact with it, and it establishes some kind of weighted feature vector based on hashtags used in the description, words used in the comments, text drawn on the video, audio background, possibly some audio transcription of words said, if you commented on it, if you forwarded it to a friend, and so on. There may be some network based recommendations, based on who you follow but those seem to be weighted very weakly, and that makes sense if you're trying to keep the platform from getting botted.
It also seems to do some non-dominant sorting to keep from only showing you things from the same type of video.
Facebook's competing short video service is terrible in comparison. It only wants to show me Trevor Noah clips, and things tangentially related to things I may have "Liked" 12 years ago.
Although, I am quite intrigued what they would be showcasing in their "transparency centers". If they show what extent their data can be utilized. and not simply a simulation of the ML stuff.
Indeed.
World of Warcraft may not have started out as a Skinner Box. But they realized what they had and used it to their advantage. The result is that all games today are casino-ified. Lootboxes, microtransactions, abstracted currency, etc. It all comes from Vegas.
TikTok and apps like Robinhood are the bleeding edge of this.
https://www.forbes.com/sites/sergeiklebnikov/2020/06/17/20-y...
Turning the stock market into a slick slot machine with all the same feedback and nudges. It's sickening, but not surprising.
What I find strange about TikTok are the waves of themes I get to see. Being a 50-something I'm probably not the target audience but at first all I saw was shuffle dance (enjoyed that) followed by Indian and Chinese manufacturing videos (loved those) and now it's all cats (okay with that).
My daughters explained "Elite TikTok" and "Beans TikTok" so now I realize there's a whole world built on top of gaming the algorithm in order to bolster your insider status.
Can you explain a bit more about Elite TikTok? I found this article but it contains quite a jumble of ideas https://www.theguardian.com/culture/2020/jun/22/what-is-elit...
https://www.nytimes.com/2020/06/10/style/elite-tiktok.html
https://www.distractify.com/p/elite-tiktok-meaning
flavor of the month/year?
There's so much more to Tiktok than random dance videos.
However my assessment is a video has much less revisit value than still images. The nature of a video requires much more time investment. Audience has to remain focused on a video for a period of time to achieve full reward. Unlike a still image, where a quick glance can gain satisfaction.
These platforms can not sustain on pure novel content. The model of user engagement must include revisit of content. You see a thumbnail of a video you've liked or have seen before. What is the threshold of intrigue, that will propel you to invest x amnt of time fully dedicated to that video content?
If my opinion above holds true, tiktok will not be a lasting phenomenon. The content will not keep up.
I would say the same, only for text versus images. and yet for some reason I'll never understand besides being only in my early 30ies, people prefer images. and videos even more.
Images, for the mass consumer audience, require orders of magnitude less time to fully consume.
The dynamic is very much different.
The Tiktok vs Vine comparison is very apt.. there is hardly any differences between them in terms of the process of content creation, and how compelling they are to a mass market audience
Actually, yes. This particular generation will eventually grow out of TikTok like they did with other social networks at their peak of the hype cycle: Snapchat (2014), YikYak (2014), Vine (2015), Shots (2012) and SNOW (2016) etc.
TikTok is no different and yet another hype masterpiece for the record books and the news media. The new kids will find another one to replace it and suddenly it will be crowned as 'cool'.
Rinse and Repeat for another generation.
[1] https://theintercept.com/2020/03/16/tiktok-app-moderators-us...
[2] https://www.washingtonpost.com/technology/2019/09/15/tiktoks...
YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out.
1 minute of content surrounded by 9 minutes of filler.
TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's no incentive to create filler - just the opposite.
Of course the downside is that there's only so much you can fit into a 60 second package, so you're not going to get a deep dive into anything. But for the kind of content that can be compressed like that, TikTok wins big time.
But again, that is just the random subset of YouTube that I watch. I doubt any of it is that popular, and I am always shocked when I get logged out and am on the default main page. So my experience could be totally off-base.
Being logged in and curating really makes all the difference. ContraPoints, CGP Grey, Joe Rogan, Kurzgesagt - I think that kind of content shines on YouTube, isn't possible on TikTok, and don't see them as artificially long.
They're really long and typically use text to speech instead of a human.
The first few minutes are the intro, a message that the video is sponsored and we will find out about them later, last 3/4 minutes is always the "Like, Subscribe" and a sponsorship message usually. This means that both the front-and the back can be cut out easily without any affect on the actual content.
Yeah no kidding. The stuff that's popular on Youtube is like it's from another planet to me. I stick to my subscription page.
It's to a point that I skip any TikTok video that starts like “I was in the forest and found this mushroom. Did you know that this kind of mushroom can be used to... ”. I skip it as soon as I hear “I was in the“ (that and the usual tone that comes with it).
I don't care. Show me directly what you are doing with the mushroom.
normalcy -> conflict -> (rising action to) climax -> resolution.
In flash fiction (i.e. very short fiction), the same elements exist, but due to brevity constraints, it starts the story in the middle, i.e. the point of conflict or climax, and then resolves the other parts along the way.
I think creators who understand how to exploit this structure could potentially make some really compelling short videos.
Example:
"For sale: baby shoes, never worn."
Being able to get consistent, instant stimulation from doing effectively nothing seems dangerous to me.
We're training ourselves to not be able to focus on anything.
Being thrown right into the mushroom thing, into the action, forces me to keep focus and attention. The pseudo-introduction is just dulling my mind while not bringing anything of consistence.
I had some tiktoks that started like "So I read something incredible the other day and I just had to try it. I am gonna show you what happened when I decided to eat an orange under the shower". Don't care. Starts with you eating the orange.
As another sibling commentator wrote it's a different narrative structure.
> Being able to get consistent, instant stimulation from doing effectively nothing seems dangerous to me.
The same could be said from books but I get what you mean. I still read books, so I guess I am okay (it could be worse !).
edit: to be fair, I have very low tolerance and patience for people who beats around the bush :/. On the other hand, when walking outside I can stop and look around me for 2 or 3 minutes every 100 meters because there's always some piece of architecture or natural landscape to look at.
I'm not sure if this has anything to do with my general preference for sub 5-minute videos that just get to the point.
I really moss when people wrote what they did in little snpits on blogs.
Given how popular obnoxious jump cuts are on YouTube, I would assume this wasn't the case. What part of YouTube incentivizes this?
I always attributed the increased intros/outros/ad spot/filler lengths to the general trend of YouTube getting more "professional" than it did 10 years ago.
Advertising placements. If a video is longer than eight minutes (formerly ten minutes), it's eligible for "midroll" ads and will earn the creator more money.
Which is funny in the greater historical context, because in the early days of YouTube you couldn't upload videos longer than 10 minutes.
EDIT: As an aside, YouTube's algorithms also punish you if you aren't regularly uploading content. It was really a perfect storm of shitiness where content creators needed to pump out content constantly, while also ensuring those videos were all 10 minutes or longer. It drastically affected the YT landscape. Content creators like Noah Caldwell Gervais who only sporadically release content because they spend HUGE amounts of effort on what they do release were getting slammed by the algorithm. Again, the algorithm has gotten a little better in that regard over the years. Though it's gotten worse in other ways.
One of the reasons I use a FF addon to make the default speed of all youtube videos 1.5X. So few v-bloggers are succinct and fluid enough to have to listen at 1X.
Finding subsequent parts of a multi-part series generally isn't worth the effort.
Except for maybe collecting too much data, but that’s any app, and luckily Apple is taking their measurements for certain things
Which is a legitimate threat to some people, but it's not that much of a threat to the common man.
The current foremost power, America, doesn't like that.
At the very least, this whole thing has made clear that principles that America repeatedly espouses are forgotten quite quickly when it wants to retain power.
Or, when Canada was threatening the US superiority in engine tech, after which the US government threatened to embargo Canada from avionics if they proceeded with the Arrow program. They do the same with the Swedish Gripen, they put an embargo on their avionics if Sweden sells them to a country which is also considering the F-35.
American tech compamies are banned from China unless they store data on Chinese servers and comply with Chinese law; there is no need to transfer IP. Do you think Google would have transfered Search IP under Project Dragonfly? Do you think Facebook has given their Ad IP?. Why wouldn't the US use the same standard of requiring full compliance with US law as enforced by a liaison and data being on US soil?
Because the question isn't of reciprocity, it is of dominance. That's why TikTok was banned under the Joker of "National Security", not under any other standard. The US is simply scared of losing it's advantage in tech to China. Which of course will likely happen anyways.
In any case, the average person isn't harmed by TikTok anymore and indeed less so than Facebook.
Provably false. [1]
>comply with Chinese law
Chinese law requires companies to hand over root access and data, and also effectively forbids E2E encryption. This is IP transfer. [2]
>The US is simply scared of losing it's advantage in tech to China
The US has several tech companies with market caps more than double Tik Tok.
>National Security
There's evidence Tik Tok has been promoting CCP propaganda. [3]
>the average person isn't harmed by TikTok anymore and indeed less so than Facebook
Depends if you trust your data with the same government who regularly disappears journalists, I guess.
[1] https://www.wsj.com/articles/forced-tech-transfers-are-on-th...
[2] https://en.wikipedia.org/wiki/China_Internet_Security_Law
[3] https://www.youtube.com/watch?v=pOlu624glKw
By that standard the US also requires IP transfer. Have you already forgotten about PRISM?
>The US has several tech companies with market caps more than double Tik Tok.
Sure, but if China continues in this trajectory then it won't be for long
>Depends if you trust your data with the same government who regularly disappears journalists, I guess.
Depends if you trust your data with the same government who regularly fabricates evidence as an excuse to go and kill a million or so people every once in a while. Or the one that regularly assassinates or threatens to assassinate activists, I guess. The chance of your own government killing you is at least four or five orders of magnitude higher than that of China killing you, same for most other metrics of harm. Can't say the same if you're a foreigner with the US government, though! That being said, US control of the press is much more sophisticated, so there is really no need to kill any American journalists except in exceptional cases. But don't delude yourself, if it ever came to that there would be no hesitation - look at what the USG did to Fred Hampton of Gary Webb, or what they tried to do with Martin Luther King.
>There's evidence Tik Tok has been promoting CCP propaganda. [3]
There's a difference between people on TikTok defending China and TikTok conspiring to push Chinese propaganda. There is evidence for the first one, and absolutely no evidence of the latter. There's people who defend China and push propaganda on Facebook and Instagram too, if you go looking for them. And there's also American propaganda on TikTok.
I don't think Principles are really given. You have principles and you can abide by them.
> All major American tech companies are banished from China
Nope. They aren't banishing anyone. They simply have laws that make it difficult for them to work in China. What law was TikTok not following that they are being forced to sell to a US company? Its essentially extortion.
If the US makes a law, i.e. all foreign-owned companies must store data on US soil or whatever else the issue with TikTok is, then that would actually fit. Rather the administration is forcing Bytedance to sell to a US company i.e. give it to us or we will kill it.
This could be very bad for so many reasons.
We need new regs for the new world order.
I have a dream of naively making a open source recommendation engine using data from good reads and imdb.
There is a real utility here that I worry will be captured entirely by companies without an open source alternative.
This is not an algorithm; this is a heuristic. An algorithm is (loosely) a method or procedure for achieving some specified end.
"... avoid redundancies that could bore the user, like seeing multiple videos with the same music or from the same creator" is the goal, and they have heuristics to try to work towards that, and algorithms and software that implement those heuristics.
I think the ship has sailed on this, but when you are in circles where both heuristics and algorithms are in play, this blurring of lines makes for very confusing conversations.
The things I didn't know in this article were the stuff about device type (what is that used for?) and the initial 8 videos. Perhaps the next trend I'll see is flexing on having an unlocked, AT&T-branded Samsung G892A.
this has to be a paid PR piece for tiktok. not surprising given the reality of current "journalism", but tedious none-the-less....
If we are watching 30 second clips, flipping through them quickly, that’s 2 signals a minute and 20 signals every 10 minutes (the average size of a YouTube video) and that is assuming we watched 20 videos, we might have flipped through 60 in 10 minutes.
That’s 60 signals where YouTube would’ve gotten 1 signal. Whose ML will be better trained with your preferences? Obviously TikTok, I don’t think there is anything special about their algorithms it’s just a function of their product and the type of content.