Very nice idea and it's something I'd definitely want to use.
Just one issue: it would be great if the service allowed me to accumulate credit by reviewing other sites, and have my site reviewed by many people when I launch it (using my credit).
Sure. Better to assume it's ok and have a "flag" option, though. On the contrary, it'll put too much of a burden on the submitter and if they fail to approve your comment it won't count.
It'd be more interesting to allow some sort of optional rating system which feeds into something like a gaming matchmaking system. People who consistently send out good reviews will get a rank of "A", for example, and be matched to others with similar rankings.
That way, even low-quality posts ("i liek that u have a logo") can be accounted for, without treating them like spam. As the reviewer becomes better at reviewing, they will see an increase in their ranking and better reviews.
Sounds like a lot of the same problems the review forum on ozzu.com had/has. Look how that turned out. Usually when there is some motivation for reviewing other sites the quality of the reviews goes down.
Loved the concept. And kudos for a frictionless onboarding process. Just submitted 3 reviews. Some suggestions:
- I forgot to submit the "star" ratings. Just entered the comment. May be you can have 1. give rating, 2. provide specific feedback (I would not use the word "comment")
- It would be nice to see the reviews I submitted and edit if needed
- Also, I'm not clear if the reviews will be made public. make it clear upfront.
- Not sure if you already do, I would like to see "Rating" of the reviewer. That would help to judge the feedback.
You might want to make sure that you're first on queue if you sign up your site and do a bunch of reviews.
I just did three reviews and it should've been possible to make sure I got one in return by now, but that didn't seem to happen yet (or is it because my site is HTTPS only?)
I can second the comment about star ratings as well; the default of 1 star especially was not in line with my intent for at least one of my reviews.
Great idea, but the UI is a little confusing. After writing a comment do I need to click 'Add comment' AND 'Submit review'? Or does either do the job? Also I forgot to set a star rating, perhaps default to zero and prompt user if they don't set it? Why can't I see the reviews/comments I've submitted on my account page?
Cool idea. Great job on launching - I'm sure there were many things you'd like to clear up and fix for a "proper" launch, but you finished something and you got people using it.
As many people said already, it would be great to know the difference between "Add comment" and "Submit review". Seems like they could do the same thing.
Keep polishing it, and then post a changelog somewhere - it's always nice to see the list of bugs people have worked through at the early stages of a site. It's also really motivating for the developer as well.
Thank you for the positive feedback. I really appreciate it.
"Add comment" is on its way out. There will be no way to post more than 1 comment, as ppl. post 1.1 comment on the average so it's just unnecessarily complicating things.
I'm keeping a backlog; good idea, posting it, thanks!
Unfortunately, it looks like you're already getting spammed. I just reviewed Google.com and what I think would have been nsfw site had I let it load.
Also, it takes too long to generate the screenshot. I waited a few minutes before giving up on my third review. Why not just open the site in an iframe?
Getting to try it out without having to log into anything is fantastic.
Generating the screenshot was slow but didn't take anywhere near a few minutes for me. Maybe ~5 seconds. Site probably getting a lot of traffic, I got some errors at first but it works fine now.
Ah, I checked when it was first posted but it seems to be better now. It was probably the site itself and not the screenshot functionality that was the problem.
As suggested in another reply, linking via iframe would still be a nice option to have when submitting.
Thanks for the follow-up. IFrame had actually been there in version 1 but people use really non-standard stuff to break out of frames and some are not even aware of it because they use third-party frameworks.
Screenshot + link to open in a new window actually started as a sort of a workaround but I think this allows people to focus more on first impressions instead of going into usability testing.
There were some temporary problems. Google.com sites were also eliminated, some 'spam' will leak in but anyway provided the people contribute, they submit google.com or microsoft.com because they're not ready to use their own site yet. An option to just post reviews without submitting site will take care of it. Thanks for the heads up!
A Hot or Not clone for screenshots that does nothing to give the designer/developer actual feedback and unfairly pits e-com sites against content-heavy sites and brochure sites?
Great idea, getting close on execution. I've reviewed 5 sites so far though, and all of them are awful. Would be great to put together a list of 1000 great websites and let users review those, because it makes being a reviewer more fun.
Thank you for pointing that out! I may want to try it in the next release. It would require a clear point system though. I want to keep things as simple as possible for the time being.
Oki-doki. It's a good start, but you need to take it a notch up to make it actually useful.
Specifically, let say I am on a receiving end of reviews made by some random people. I don't really care what they have to say unless it's credible or representative.
Let's look at latter, as it's an easier one. The way to ensure a review is representative, is to ask other people to look at the review and say if they agree with it or not. Once it gets 10-20-30 meta up/down votes, it'd be more or less possible to say if the review is meaningful or if it was a brain fart.
Make the meta reviewing process mandatory. You want to get your site reviewed, pull a "meta" duty on a dozen of past reviews and (perhaps) write a review. When selecting a review for meta, do it randomly, but favor those with low vote count and controversial ones (over those voted way up or way down).
This would be an MVP of your idea. Without this, it's nice, but it's generally useless.
(edit) Oh, and perhaps simplify reviewer's life a bit and have 3 sections - "what's good", "what's bad", "free-form comments".
"This would be an MVP of your idea. Without this, it's nice, but it's generally useless."
But you've pointed out at least 5 new features. Let's say they add all of them and it fails.. Could they have learned that with only 4? 3? 1, new features?
To me, the MVP of this idea is that someone can review a site and have their site reviewed. Everything else is just a feature. Not to say they wouldn't greatly enhance the product, but MVP literally means minimum viable product; not minimum awesome product.
Thanks, that was exactly the idea -- solicit feedback on something that works but hasn't too much 'chrome'. I received more than 200 really useful suggestions (and bug reports) and the tool will definitely evolve. It already is.
The star rating seems almost unnecessary, compared to the rich feedback you get from the comments. I notice my time incrementing, but what does that mean to me?
Overall, nicely done. LOVE the on-boarding process.
I just wrote a lengthy review about a site that I feel could be improved but after clicking Submit the button went into a loading state and nothing has happened since. The comment field was blanked out after I pressed "Add comment". Any way to get my review back in the JS console?
Re the site itself, I think it's a good idea but I agree with the meta-scoring. Add a point system that levels you up both by writing a review and later by reviewing reviews of the same site/commenting on them. The higher your level or the more points you have, the more your sites will be reviewed.
And please make https sites fail more elegantly. It's okay that you can't load them in an iframe, but make it look less broken. I submitted an https site and I doubt many people will review it when they are met with a huge screen saying "http only" through no fault of their own. Instead say "This site is using https (learn more), please click here to open the site in a new tab"
Similar experience here. What's the difference between comments and reviews for that matter? There's an add comment button and an add review button, apparently tied to the same field.
Also got some after-the-fact request to sign up before getting the review of my site I was promised... not too interested in that.
Yeah, the 'Add comment' link is pretty confusing. I'd consider moving the '1 comment' text below the comment box, and display the truncated comment text instead. It felt very much like the comment I'd entered had just disappeared with the current implementation.
OP: Hi guys, I'll do my best to reply to all comments posted here a bit later.
As it is now we're barely handling the flood of reviews. :) It's a good thing and very exciting but we had to scale up the number of heroku instances the second time in a row. So please bear with all the errors for a while, we're working on some issues esp. screenshot generation as it is not coping with the load.
Wow! We didn't expect an instant success like this. Thank you all for your contributions.
I love the idea behind this site and if I get some useful feedback on my website, I'll be ecstatic.
Anyway, your site not really a success yet. You've just been Hacker Newsed. Traffic will probably go back down after today. It's happened to me before.
Anyway, it looks like a major bottleneck you are experiencing is that you have to moderate every review that's written. I don't think that's going to be sustainable. I'd try having using rate reviews "helpful" or "not helpful" like they do for reviews on most online stores. And then maybe have an automatic filter for easily detectable things like foul language or gibberish.
I didn't get an email confirming my account, on the sign up area after doing my first review, I tried both of my gmail addresses and have yet to see anything in either.
Cool concept, add a meta aspect as others have said.
a. I can't find a way to sign up as a pure critic or a viewer - a user without a website.
There will be plenty of people (the category also known as end-users) who don't have a website yet who would still like to look at and review designs by other folks.
I wasn't able to sign up because my website is down for now.
b. It'd be nice to have a way to 'save' credit and expend it later on reviews for sites that I may build later.
What about reversing the order? Make it so you can't submit your site for review until you've reviewed another. Obviously quality may suffer, as people will just type garbage for the sake of getting theirs submitted, but with volume that can be fixed via other means down the road.
Like the concept. Just submitted 4 reviews and haven't got one back for my site after 15 mins. Any way of shortening the queue to provide a quicker reward for my critiques would be awesome. It'd also be nice to see your site's screenshot to know what people are looking at when they review my site.
Moderating is a bottleneck but it'll improve. There's currently a queue of sites but the idea is to make it as short as possible. Yeah, instant feedback would be really cool! :)
I need a terms and conditions link. I want to submit my company's website but I want an assurance that the feedback won't be visible by the general public; i.e. through a google search for "example.com review" or something. and Ts and Cs page or data policy would clear that up. As it stands, I'm reluctant to use this service.
What prevents anyone from submitting your website on your behalf? Do they use some kind of "put this HTML file in your root folder" or something similar?
Oh, nothing at all. I'm just saying that the reason I haven't submitted it is because there's no indication of what will happen when I do. I just don't want "example.com uses criticue" plastered on their twitter/facebook/etc, and there's nothing reassuring me that it's semi-private.
I also wasn't specific enough; I'm talking about the website of the company that I work for, not one I own. So the reason I'm reticent about using the service is that I'm merely casually interested in what people have to say, rather than submitting the site in an official capacity.
If you work for a company that requires secrecy, you should probably go with a professional evaluation anyway (via consultants and user testing).
I think this site was meant for quick 1 or 2 sentence feedback from the general public for starter/personal sites and the feedback examples on the front page of their site clearly shows this.
I don't agree with your reason but am totally behind you when you say there should be a t&c page. I need to know how my user will be treated and if the founders care enough about my placing their trust in me etc.
157 comments
[ 2.3 ms ] story [ 246 ms ] threadJust one issue: it would be great if the service allowed me to accumulate credit by reviewing other sites, and have my site reviewed by many people when I launch it (using my credit).
Perhaps submitter should be able to score feedback as helpful or not, so people can't spam asdf on every site to gain "credits"?
That way, even low-quality posts ("i liek that u have a logo") can be accounted for, without treating them like spam. As the reviewer becomes better at reviewing, they will see an increase in their ranking and better reviews.
- I forgot to submit the "star" ratings. Just entered the comment. May be you can have 1. give rating, 2. provide specific feedback (I would not use the word "comment")
- It would be nice to see the reviews I submitted and edit if needed
- Also, I'm not clear if the reviews will be made public. make it clear upfront.
- Not sure if you already do, I would like to see "Rating" of the reviewer. That would help to judge the feedback.
Keep it up!
Failing that, how about defaulting the rating to 3 star so at least if you do forget you don't end up giving the lowest rating possible?
Enjoying the whole process so far though, nice job.
I just did three reviews and it should've been possible to make sure I got one in return by now, but that didn't seem to happen yet (or is it because my site is HTTPS only?)
I can second the comment about star ratings as well; the default of 1 star especially was not in line with my intent for at least one of my reviews.
As many people said already, it would be great to know the difference between "Add comment" and "Submit review". Seems like they could do the same thing.
Keep polishing it, and then post a changelog somewhere - it's always nice to see the list of bugs people have worked through at the early stages of a site. It's also really motivating for the developer as well.
"Add comment" is on its way out. There will be no way to post more than 1 comment, as ppl. post 1.1 comment on the average so it's just unnecessarily complicating things.
I'm keeping a backlog; good idea, posting it, thanks!
Unfortunately, it looks like you're already getting spammed. I just reviewed Google.com and what I think would have been nsfw site had I let it load.
Also, it takes too long to generate the screenshot. I waited a few minutes before giving up on my third review. Why not just open the site in an iframe?
Getting to try it out without having to log into anything is fantastic.
Cool site.
As suggested in another reply, linking via iframe would still be a nice option to have when submitting.
Screenshot + link to open in a new window actually started as a sort of a workaround but I think this allows people to focus more on first impressions instead of going into usability testing.
So don't feel too bad OP.
Application Error
An error occurred in the application and your page could not be served. Please try again in a few moments.
If you are the application owner, check your logs for details.
Look at how other people do it for other sectors (for example Critters for fiction writing and okcupid for photos)
Specifically, let say I am on a receiving end of reviews made by some random people. I don't really care what they have to say unless it's credible or representative.
Let's look at latter, as it's an easier one. The way to ensure a review is representative, is to ask other people to look at the review and say if they agree with it or not. Once it gets 10-20-30 meta up/down votes, it'd be more or less possible to say if the review is meaningful or if it was a brain fart.
Make the meta reviewing process mandatory. You want to get your site reviewed, pull a "meta" duty on a dozen of past reviews and (perhaps) write a review. When selecting a review for meta, do it randomly, but favor those with low vote count and controversial ones (over those voted way up or way down).
This would be an MVP of your idea. Without this, it's nice, but it's generally useless.
(edit) Oh, and perhaps simplify reviewer's life a bit and have 3 sections - "what's good", "what's bad", "free-form comments".
But you've pointed out at least 5 new features. Let's say they add all of them and it fails.. Could they have learned that with only 4? 3? 1, new features?
To me, the MVP of this idea is that someone can review a site and have their site reviewed. Everything else is just a feature. Not to say they wouldn't greatly enhance the product, but MVP literally means minimum viable product; not minimum awesome product.
Re the site itself, I think it's a good idea but I agree with the meta-scoring. Add a point system that levels you up both by writing a review and later by reviewing reviews of the same site/commenting on them. The higher your level or the more points you have, the more your sites will be reviewed.
And please make https sites fail more elegantly. It's okay that you can't load them in an iframe, but make it look less broken. I submitted an https site and I doubt many people will review it when they are met with a huge screen saying "http only" through no fault of their own. Instead say "This site is using https (learn more), please click here to open the site in a new tab"
Also got some after-the-fact request to sign up before getting the review of my site I was promised... not too interested in that.
1. Enter comment. Submit it. 2. Repeat step 1. 3. Submit review.
Took me a few seconds as well to figure that out.
I do hope that issue gets sorted asap.
After posting on Y there was an unexpected surge in traffic. Everything works fine after increasing the number of Heroku instances.
As it is now we're barely handling the flood of reviews. :) It's a good thing and very exciting but we had to scale up the number of heroku instances the second time in a row. So please bear with all the errors for a while, we're working on some issues esp. screenshot generation as it is not coping with the load.
Wow! We didn't expect an instant success like this. Thank you all for your contributions.
Anyway, your site not really a success yet. You've just been Hacker Newsed. Traffic will probably go back down after today. It's happened to me before.
Anyway, it looks like a major bottleneck you are experiencing is that you have to moderate every review that's written. I don't think that's going to be sustainable. I'd try having using rate reviews "helpful" or "not helpful" like they do for reviews on most online stores. And then maybe have an automatic filter for easily detectable things like foul language or gibberish.
Good luck, I like you site.
Yes, moderating is a bottleneck. :)
Thank you for the feedback! Greatly appreciated.
Cool concept, add a meta aspect as others have said.
a. I can't find a way to sign up as a pure critic or a viewer - a user without a website.
There will be plenty of people (the category also known as end-users) who don't have a website yet who would still like to look at and review designs by other folks.
I wasn't able to sign up because my website is down for now.
b. It'd be nice to have a way to 'save' credit and expend it later on reviews for sites that I may build later.
(a) TBD soon. (b) This will follow.
I'll also second the n-for-1 suggestion.
Scrolling down, I did also forget to leave a star rating. Maybe that can be clearer?
Nothing.
I also wasn't specific enough; I'm talking about the website of the company that I work for, not one I own. So the reason I'm reticent about using the service is that I'm merely casually interested in what people have to say, rather than submitting the site in an official capacity.
I think this site was meant for quick 1 or 2 sentence feedback from the general public for starter/personal sites and the feedback examples on the front page of their site clearly shows this.