I don't get what the ask is. Steam should be even more heavy handed? They should hire people to playtest other people's games? I don't see any alternative reality where they have already taken these actions and are less hated for them.
I think the author wants to suggest a solution where anyone can submit a game, but there is still some sort of quality check before it goes on sale.
Currently, we have floods of garbage hitting Steam, with no curation at all. Pre-Greenlight, Steam was a closed shop that was hard to get into.
What is needed is a healthy balance, that doesn't have zero chances for an indie game to be seen because it's buried under all the AAA titles, yet doesn't allow trash like Slaughtering Grounds to be sold.
The article suggests getting Valve employees to playtest any submission - however, I feel this still wouldn't work due to Valve's 2 main issues. Their flat structure, where people work on what they want - who wants to look after content curation? Secondly, they take a cut of every game sold - regardless of how awful it is. They are financially incentivised to keep subpar (or even scammy) games on the store.
I'm not going to pretend I have a solution, but I feel all the issues with the Steam store currently are issues of Valve's own making - the problem is, they don't (seem to, at the very least) care.
Well, I was attempting to frame it in the context of the current, free-for-all situation.
So, more "who wants to wade through potentially piles of garbage unity-based asset flips made by one guy in his garage, while potentially stumbling across a couple of commercially viable games"
Personal Anecdote from a former videogame QA tester and min-spec discoverer: Playing video games as a job is not playing video games. It's software testing a GUI.
Basic "does it run on X platform" is even more boring (but more automatable).
But back in the day people used to hate on Steam because they were a closed shop that was hard to get into. So from their perspective they have to go from being unpopular one way to being unpopular another?
The ask was pretty clear to me: "Game is submitted to the store. Human plays game, finds it doesn’t work. Human emails developer and says, “This game doesn’t work, fix and resubmit.” Done."
Doesn't even have to be a human for some of the "does it launch on a sample of devices" tests.
You can't get your game onto XBox or PS without passing these kinds of basic checks, and this has an impact on the Valve brand. I'm just not sure Valve has someone in charge of ensuring that their brand is maintained.
If Apple can manually review and curate the thousands upon thousands of apps a day being submitted to the app store, I think it's reasonable for Steam to manually check that 30 new games a day load and actually work.
I think you could use Steam Achivements and playtime to recommend games to people.
If I've played a game to completion on the highest difficulty presumably I like it. Find me other games that people with the achievement also play and have the equivalent achievement on and you've got something basic.
Max or close-to-max achievements for a game also is an indicator of skill, could be used to weight a games scores more towards "recommended by skilled players" vs. "recommended by casual players" ?
Yep, you could use achievement rarity too.
If 100% of all players get an achievement don't use it for recommending or 1% have it increase the weighting?
I would have a hard time equating achievement coimpletionism as skill, to me they are more of a measure of time spent in a game. Yes, some achievements in some games do require a mediocrum of skill, but most of them do not; they're used primarily as a form of feedback for the developer on how people play their game, and secondarily as a lure for completionists who want the points on their profile.
It only dawned on me today that achievements aren't really for the players, they're the gaming equivalent of Mixpanel, allowing developers to see which parts of the game players are actually experiencing.
Yeah, that's the real weirdness here -- that Steam/Valve can't come up with some way to leverage the absolutely staggering amount of data they must have on how people play the games in their Steam library to filter the wheat from the chaff.
Well, they could. They could have the slickest UI and the best search experience. But Steam brings in staggering profits as-is. Valve doesn't need to do much of anything, unless a serious competitor arises. This is the problem with "benevolent monopolies" like Valve and Google. All is well when they're making users happy. But when they aren't, there's no accountability, incentive, or alternative.
Wouldn't that just motivate developers to hand out lots of achievements early and often (lets say with 4 levels, 75% in the first level, 15% in the second, 5% in the last two each), so that the people who play the game even just a few hours "recommend" the game by the high number of achievements?
Uhhhh... Steam needs a decent startup time. I only play computer games about once a month. When I click on the link on my desktop, Steam always takes 5-10 minutes to update itself.
This is ridiculous. I never have this problem with games on my Nvidea Shield.
I would prefer it if the update is unimportant if it downloaded it while steam was running and installed it when it starts up next time. But at least for me I seem to always have to wait for a download, which is especially annoying when trying to play with friends
How many updates does the Shield itself and the games on there get per month? I know Android automatically updates apps in the background, just like iOS; if you have Steam autostart in the background (which I do),the startup time becomes negligible and the updates unnoticeable.
But that's the benefits of PC gaming, you get the choice on whether or not to have things start up and keep updated in the background - and if you choose not to, you end up with a situation like that.
I think it's pretty cool that they have a development pipeline that allows for frequent releases.
I just wish to understand why they need an application update so often to what should be a simple distribution product. Do they hard code item availability or such?
A fair number of those updates go out 2 or 3 times with little fixes but the same "release notes," so that's not a particularly accurate count of the number of times Steam will want to update.
Valve is famously organized as a place where everybody gets to pick the problems they want to work on (https://www.bloomberg.com/news/articles/2012-04-27/why-there...). Manually reviewing dozens of (mostly crappy) games every day in the hope of finding a few that are worth recommending is the kind of dull, grinding work that nobody would ever pick for themselves to work on.
Core complaints of article (please add any that I missed):
1. Steam receives dozens of releases a day and it's difficult for developers/publishers to make their title stand-out without performing marketing.
2. Author perceives that Steam's display algorithm results in display of outliers (very good/bad).
3. Very low quality titles are released/being sold by Steam.
4. Reviews are a very poor metric as reviewers can be shills or rate content poorly out of pettiness or malice.
5. Author thinks that staff-curated titles are the solution.
My responses:
RockPaperShotgun is a game review site. Reviewing games is how they manage to make ad revenue (well, besides click-bait articles). If the defense is that more content comes out than RockPaperShotgun can review, then RockPaperShotgun should hire more people. If that's cost prohibitive, and there's market demand for it, then why are they asking Steam to do it?
1. Welcome to selling anything.
2. I don't know if this is true; I'm not rejecting this claim, but I'd love to see the data.
3. Quantifying "good enough to sell" is a risky game. I'm okay with spending $5 (most of a beer at a bar) on something kind of buggy if it's amusing for 30 minutes. For someone else, that amount may be double or $0 on principle.
4. If this subproblem gets solved without background checks on reviewers, you have a serious business that Amazon/Yelp/etc would love to buy from you.
5. Beyond catching content that simply _does not do what it says on the box_, this is all personal taste.
And as an aside, there's a "curious" market that exists in low-quality games that have DLC/content that ends up on the Steam Community Market (Steam gets a % of each transaction), so the "how did a broken game end up on Steam?" question has more depth than a casual observer would guess.
Asking review sites to sift through the dredges in the spirit of "more business is better!" is naive. Jimquisition comments regularly on this and it has turned into a longstanding and ever constant critique of Valve and its "Whatever goes" attitude to getting on their Steam platform. At 20 or 30 new games _a day_ consumers don't have time to go on review websites and see what's on the "do not touch list". What a monumental task for consumers! If steam would block games that, out of the gate, did not work, that would be great. As the author stated:
This morning in half an hour I got through three interesting-looking games that didn’t have functioning controls.
You can't even play them! How are unplayable games on Steam!? Additionally they should make sure that the min spec IS an actually min spec that run the game in a playable state. No studio should be immune to such a level of scrutiny (see Rocksteady's Batman Arkham Knight).
Paying someone 10$ an hour to do a quick look at a game for 2 hours * 30 new games a day ~= 220k/year, double that to pay for games and your looking at ~450k/year which is not free, but that expensive either.
Really this could stand on it's own as a product either as a review site or an industry group that supports several review sites.
PS: An yes that seems like a high estimate the average is less than 20$/game and most of this junk does not need 2 hours.
Well, the article says the guy found three broken games in 30 minutes. Assuming Valve's reviewers took three times that, 30 minutes of testing per game, it'd still only cost them $55k - less than one developer's salary. Hell, they could charge $10 to list a game, and make a healthy profit out of it!
Valve wouldn't have to pay for the games - vendors let retailers inspect their product for free /all the time/.
You argument is that its cheap to review. Sure, but why should all review sites duplicate the work? Why can't valve do the review, stop the crap at the source.
Stop apologizing for shitty content making it to the store shelves.
> or an industry group that supports several review sites.
So, no I don't think any one site needs to take on this cost rather anyone can get it off the ground for a few months and then sell a feed to other people thus defraying the costs.
Sure, steam could have higher standards, but IMO there is value in letting poor games into their ecosystem that may be improved with time. UnReal World for example been in development by one guy for 26 years and right now it's a solid game though dated game. But, 20 years ago it was a buggy mess that would only really interest a tiny group of people even though he had been working on it for 6 years.
I'd rather Steam get back to the core business of selling games, rather than trying to be some sort of oddball social network.
Drop the achievements (do a large number of players actually pay attention to that?), drop the trading cards (I still don't know WTF those are for). Drop everything that isn't about selling a game to a player. Then hire some competent UI people to simplify and clean up that abhorrent interface that tries to do too much.
In fact, separate Steam from Valve entirely. Let Valve focus on producing games and let Steam focus on selling them, but put a wall between them.
The trading cards are part of the Steam economy. It's a money-making ploy. I don't know if it works or not, but it probably does.
If they dropped everything that isn't about selling games to players (Steam Matchmaking, social features, the Steam economy especially wrt CS:GO and Dota) I don't think Valve would be able to sell and host games as cheaply as it does, and I think more people would pirate games rather than getting them on Steam.
Ding ding ding! Each one of those trading cards facilitates trade. When there is more trade, there are more transactions for valve to skim. It's simple broker economics, and it works great for everyone involved.
Valve doesn't seem to really care about making games for the sake of making games, they make games if they can do something new/cool which hasn't really been done before.
In other words, it's not that Valve can't produce games right now it's that they don't want too. They're too busy with Steam/VR cause those things are new/cutting edge.
I'm an applied researcher in search science. This is a very typical domain specialized search problem. These days it uses Big Data, Machine Learning, constant experimenting / prototyping, A/B testing. I get to geek out each and every day. It is my dream job :)
I consider it one of the main reasons Apple lost so much content market share to Amazon despite Apples first mover advantage - Amazon has better search and you can actually find the stuff you want. AFAIK Apple tried to start up an applied research group to fix the problem but they refused to pay enough and ended up with the dregs. They tried to hire me 3 years ago for $130K. It highlighted that they didn't understand the importance of the problem and we're likely to fail. I typically make ~$240 as FTE and more as a contractor. Apple also has a culture of secrecy where as search science is pretty open.
Anyway; I checked out Steam. The search there is clearly very poor and can easily be improved. The overreliance on tags is usually a good indication that they are not digging into their usage data for their recommendations.
It's basically the union of Information Retreival, Big Data, and Machine Learning. There is a lot of good and bad info out there.
It's best to tailor your learning around a problem so you can get feedback on what works. Building bespoke search engines is expensive and often not worth it until the problem is big enough. Machine Learned optimization is even more expensive.
So in the likely instance that your problem is small; I'd stick to an off the shelf Lucene. If you need more specialization write plugins for it. If you need more speed then DIY OkapiBM25 in native (maybe Rust these days). If you need Big Data I'd use Spark. If you need ML then GDBT in R. If you need advanced NLP then Deep Learning. At each stage it's usually diminishing returns. So quit once you start losing money on the additional effort.
Edit: As a fun aside. Page Rank doesn't work very well and AFAIK Googles major advance was from creating 'meta documents' using anchor text and search queries. Google has a habit of sending out red herrings to guard their important ideas. So if some blogger is waxing lyrical about Page Rank you know they're full of it.
Page rank was very useful when they started, 19 years ago. It was killed by their own success as normal people stopped posting link sites, and bad actors added their own.
The people that I know at MS and Yahoo told me that Page Rank was never as good as meta-documents. And meta-documents was more than sufficient to explain the improvement in relevance.
Of course this is anecdotal and I would be happy to see evidence to the contrary.
People love to demand that Valve or Google hire lots of people to do manual curation or phone support. Yet that costs money, and people would buy their games elsewhere it it's two cents cheaper.
I don't know. There might well be websites or apps where $game_that_i_buy is 2 cents cheaper, but I don't even bother with anything else than Steam. I've just decided to buy into the ecosystem. The question then is: How common is my behavior among Steam users (or gamers in general)?
Honestly I have been thinking more and more about the problem with steam, and I realized that even as a foss, anti-drm proponent, steam is one of the few things I have bended for on those positions because of the quality of the marketplace. If they continue to fail, I truly hope something like Lutris or Desura or GoG steps up to the plate and open sources a game store (bonus if DRM free like GoG is). I mean, does anyone remember the days of hating games that required you to have a CD in the tray so you would get a noCD warez just to play? Steam solved those issues, which is what propelled it to it's current state. I don't think that's a problem with modern games though, so perhaps steam is weaker than Valve realizes if people ever actually thought about it. Now it's main function is as a game library stored in a cloud.
That makes me really sad to realize too, because I feel like Valve is one of the only companies that has the resources to really push linux gaming into a new zone, but this hasn't been delivered on yet. I highly suspect they are waiting for Vulkan, but the time can't come soon enough to offer many people the last reason they need to switch to linux from windows: gaming.
I have also noticed this sort of let the customers do the work for us attitude far too often lately. For example, I am loving working in Unreal Engine 4, but they promised a native linux editor and then ingored it for a long time, only once linux native users submitted enough patches to their own fork(s) before Epic finally spent some time to push those changes to the official repo. I think sometimes open source users who want things do work to get them but some companies abuse this process. In Epic's case, we still don't have a marketplace launcher on linux, and there are tons of editor bugs, such as I can't even edit skeletal animations/rigging without crashing the client.
There is a balance to be had between user help and the companies that need it, but that pendulum seems to have swung far too far over to the user lately, and it needs to readjust.
I've been using Steam for many years and I play 5-10 hours a week. I simply find it great, to purchase games and play them (unlike EA origin and all it's clones out there, last time I checked)
The way I decide to purchase a game is simple: 1) I see/read a review about a game 2) If it piques my interest I go check out some videos and read a few more reviews 3) Add it to my wishlist 4) Purchase it when it's on 'special'
Finally, I almost never buy games unless they are at least 1 year old (save a few exceptions). This article seems mostly to have a contention about new releases and the 'suggestions' made by Steam (which I almost never look at anyways).
Hope this can help others with their Steam purchases.
I understand that the reviews and discoverability are legit problems for smaller developers trying to sell their games, but I have to wonder what percentage of people using Steam are simply "in and out" customers who are looking for the known titles anyways. Those people (like me) don't care so much about user reviews or discoverability.
I'm an "in and out" customer - I love games but I basically only buy big name titles (old behaviour from my console days). I know I'm missing out on some other great titles, but I only have a limited number of hours to put into games.
The only time I might get a lesser known game is when it got a lot of buzz outside of Steam, but the most likely scenario is via a heavily discounted bundle that contains a big name title that interested me.
Of the ~100 games in my Steam library, only about 20% are games I specifically wanted the buy. The rest were basically freebies that came along for the ride.
60 comments
[ 3.0 ms ] story [ 163 ms ] threadCurrently, we have floods of garbage hitting Steam, with no curation at all. Pre-Greenlight, Steam was a closed shop that was hard to get into.
What is needed is a healthy balance, that doesn't have zero chances for an indie game to be seen because it's buried under all the AAA titles, yet doesn't allow trash like Slaughtering Grounds to be sold.
The article suggests getting Valve employees to playtest any submission - however, I feel this still wouldn't work due to Valve's 2 main issues. Their flat structure, where people work on what they want - who wants to look after content curation? Secondly, they take a cut of every game sold - regardless of how awful it is. They are financially incentivised to keep subpar (or even scammy) games on the store.
I'm not going to pretend I have a solution, but I feel all the issues with the Steam store currently are issues of Valve's own making - the problem is, they don't (seem to, at the very least) care.
Are you asking, who wants to play video games for pay?
One expects that someone could be found...
So, more "who wants to wade through potentially piles of garbage unity-based asset flips made by one guy in his garage, while potentially stumbling across a couple of commercially viable games"
Basic "does it run on X platform" is even more boring (but more automatable).
They are destroying their own brand by not doing it.
Doesn't even have to be a human for some of the "does it launch on a sample of devices" tests.
You can't get your game onto XBox or PS without passing these kinds of basic checks, and this has an impact on the Valve brand. I'm just not sure Valve has someone in charge of ensuring that their brand is maintained.
If I've played a game to completion on the highest difficulty presumably I like it. Find me other games that people with the achievement also play and have the equivalent achievement on and you've got something basic.
"Fail to evade 5 shots in a row with a full powered and upgraded engine."
It's almost pure luck. Wiki mentions ways that you can increase odds for you but still it's mainly luck.
Whatever system you use developers will try and game it but I think there are ways of cutting out obviously bad achievements
This is ridiculous. I never have this problem with games on my Nvidea Shield.
What would a reasonable time for startup be in your eyes beyond 5-10 minutes before you play your games once a month? 1 minute? Instant?
Even stand-alone installs (which are stupidly rare these days) auto-update themselves on first load.
But that's the benefits of PC gaming, you get the choice on whether or not to have things start up and keep updated in the background - and if you choose not to, you end up with a situation like that.
I think it's pretty cool that they have a development pipeline that allows for frequent releases.
Anyway, there's a detailed changelog[1] for all the updates, so you can have a look at the kinds of things they change.
[1] http://www.steampowered.com/platform/update_history/index.ph...
Steam has a:
- bundled web browser
- a messaging client with a decent set of features
- the whole "simple distribution product"
- probably tons of other things I'm forgetting right now
- and everything included is obviously network facing, ergo vulnerabilities, ergo security patches
Valve is famously organized as a place where everybody gets to pick the problems they want to work on (https://www.bloomberg.com/news/articles/2012-04-27/why-there...). Manually reviewing dozens of (mostly crappy) games every day in the hope of finding a few that are worth recommending is the kind of dull, grinding work that nobody would ever pick for themselves to work on.
1. Steam receives dozens of releases a day and it's difficult for developers/publishers to make their title stand-out without performing marketing.
2. Author perceives that Steam's display algorithm results in display of outliers (very good/bad).
3. Very low quality titles are released/being sold by Steam.
4. Reviews are a very poor metric as reviewers can be shills or rate content poorly out of pettiness or malice.
5. Author thinks that staff-curated titles are the solution.
My responses:
RockPaperShotgun is a game review site. Reviewing games is how they manage to make ad revenue (well, besides click-bait articles). If the defense is that more content comes out than RockPaperShotgun can review, then RockPaperShotgun should hire more people. If that's cost prohibitive, and there's market demand for it, then why are they asking Steam to do it?
1. Welcome to selling anything.
2. I don't know if this is true; I'm not rejecting this claim, but I'd love to see the data.
3. Quantifying "good enough to sell" is a risky game. I'm okay with spending $5 (most of a beer at a bar) on something kind of buggy if it's amusing for 30 minutes. For someone else, that amount may be double or $0 on principle.
4. If this subproblem gets solved without background checks on reviewers, you have a serious business that Amazon/Yelp/etc would love to buy from you.
5. Beyond catching content that simply _does not do what it says on the box_, this is all personal taste.
And as an aside, there's a "curious" market that exists in low-quality games that have DLC/content that ends up on the Steam Community Market (Steam gets a % of each transaction), so the "how did a broken game end up on Steam?" question has more depth than a casual observer would guess.
6. Games that received poor reviews are relegated to the trash bin forever.
Response:
Untrue.[1] The converse also applies, games don't keep their good reviews forever - this helps identify abandonware or post-hype truth.
[1]:http://store.steampowered.com/news/21695/
Really this could stand on it's own as a product either as a review site or an industry group that supports several review sites.
PS: An yes that seems like a high estimate the average is less than 20$/game and most of this junk does not need 2 hours.
Valve wouldn't have to pay for the games - vendors let retailers inspect their product for free /all the time/.
Stop apologizing for shitty content making it to the store shelves.
So, no I don't think any one site needs to take on this cost rather anyone can get it off the ground for a few months and then sell a feed to other people thus defraying the costs.
Sure, steam could have higher standards, but IMO there is value in letting poor games into their ecosystem that may be improved with time. UnReal World for example been in development by one guy for 26 years and right now it's a solid game though dated game. But, 20 years ago it was a buggy mess that would only really interest a tiny group of people even though he had been working on it for 6 years.
Drop the achievements (do a large number of players actually pay attention to that?), drop the trading cards (I still don't know WTF those are for). Drop everything that isn't about selling a game to a player. Then hire some competent UI people to simplify and clean up that abhorrent interface that tries to do too much.
In fact, separate Steam from Valve entirely. Let Valve focus on producing games and let Steam focus on selling them, but put a wall between them.
If they dropped everything that isn't about selling games to players (Steam Matchmaking, social features, the Steam economy especially wrt CS:GO and Dota) I don't think Valve would be able to sell and host games as cheaply as it does, and I think more people would pirate games rather than getting them on Steam.
In other words, it's not that Valve can't produce games right now it's that they don't want too. They're too busy with Steam/VR cause those things are new/cutting edge.
I anecdotally do.
I consider it one of the main reasons Apple lost so much content market share to Amazon despite Apples first mover advantage - Amazon has better search and you can actually find the stuff you want. AFAIK Apple tried to start up an applied research group to fix the problem but they refused to pay enough and ended up with the dregs. They tried to hire me 3 years ago for $130K. It highlighted that they didn't understand the importance of the problem and we're likely to fail. I typically make ~$240 as FTE and more as a contractor. Apple also has a culture of secrecy where as search science is pretty open.
Anyway; I checked out Steam. The search there is clearly very poor and can easily be improved. The overreliance on tags is usually a good indication that they are not digging into their usage data for their recommendations.
It's best to tailor your learning around a problem so you can get feedback on what works. Building bespoke search engines is expensive and often not worth it until the problem is big enough. Machine Learned optimization is even more expensive.
So in the likely instance that your problem is small; I'd stick to an off the shelf Lucene. If you need more specialization write plugins for it. If you need more speed then DIY OkapiBM25 in native (maybe Rust these days). If you need Big Data I'd use Spark. If you need ML then GDBT in R. If you need advanced NLP then Deep Learning. At each stage it's usually diminishing returns. So quit once you start losing money on the additional effort.
Edit: As a fun aside. Page Rank doesn't work very well and AFAIK Googles major advance was from creating 'meta documents' using anchor text and search queries. Google has a habit of sending out red herrings to guard their important ideas. So if some blogger is waxing lyrical about Page Rank you know they're full of it.
Of course this is anecdotal and I would be happy to see evidence to the contrary.
That makes me really sad to realize too, because I feel like Valve is one of the only companies that has the resources to really push linux gaming into a new zone, but this hasn't been delivered on yet. I highly suspect they are waiting for Vulkan, but the time can't come soon enough to offer many people the last reason they need to switch to linux from windows: gaming.
I have also noticed this sort of let the customers do the work for us attitude far too often lately. For example, I am loving working in Unreal Engine 4, but they promised a native linux editor and then ingored it for a long time, only once linux native users submitted enough patches to their own fork(s) before Epic finally spent some time to push those changes to the official repo. I think sometimes open source users who want things do work to get them but some companies abuse this process. In Epic's case, we still don't have a marketplace launcher on linux, and there are tons of editor bugs, such as I can't even edit skeletal animations/rigging without crashing the client.
There is a balance to be had between user help and the companies that need it, but that pendulum seems to have swung far too far over to the user lately, and it needs to readjust.
The way I decide to purchase a game is simple: 1) I see/read a review about a game 2) If it piques my interest I go check out some videos and read a few more reviews 3) Add it to my wishlist 4) Purchase it when it's on 'special'
Finally, I almost never buy games unless they are at least 1 year old (save a few exceptions). This article seems mostly to have a contention about new releases and the 'suggestions' made by Steam (which I almost never look at anyways).
Hope this can help others with their Steam purchases.
All games DRM free
Games are tested before being listed
I'm an "in and out" customer - I love games but I basically only buy big name titles (old behaviour from my console days). I know I'm missing out on some other great titles, but I only have a limited number of hours to put into games.
The only time I might get a lesser known game is when it got a lot of buzz outside of Steam, but the most likely scenario is via a heavily discounted bundle that contains a big name title that interested me.
Of the ~100 games in my Steam library, only about 20% are games I specifically wanted the buy. The rest were basically freebies that came along for the ride.