Show HN: Noctie – A chess AI that predicts your rating (noctie.ai)
I built over the last two years a human-like neural network chess engine that tries to predict your rating from a single game. It automatically adapts to your play and tries to play like a human at your level would play, giving you a balanced game.
At the core I’m using an AlphaZero / Leela Chess Zero style neural network that I have trained on 1 billion human games from the lichess.org open database. Around this network I have built a chess engine in Rust with algorithms that use the outputs from the NN to produce human-like moves at a given rating from beginner to world champion, as well as predicting the level of the opponents play.
I want to develop this into kind of an AI coach that you can spar certain positions against and get feedback suited for your level. Happy for any suggestions!
105 comments
[ 3.9 ms ] story [ 166 ms ] threadOn the user side - I think the review area would benefit from principle-based analysis similar to what is seen in chess.com. I am guessing that for it to happen, there needs to be a separate analysis engine that can detect positional motifs. I think chess.com did a decent job with that (better and more actionable than decodechess in my opinion).
On the tech side - I’m curious how this project differs from existing neural nets trained for lc0 such as bad gyal and maia?
On the tech side, what Maia is doing I think is similar to our NN component. It turns out to get realistic play (include randomness, avoid too many one-move blunders, same level of tactical play as positional play, realistic endgame play, etc.) you have to do quite a lot on top of that – much more than I initially thought. I have tested the AI over the last year with friends of different strengths to try to make the gameplay more realistic and engaging. I also think the automatic adaptation to your level of play, and predicting the rating, is unique. Bad gyal AFAIK uses a different training method where Stockfish evaluations is involved, but I haven't really tested it enough to know how it stands vs. Maia.
This happened twice now (at move 3 the second time).
I'm not sure it should take that long especially in opening moves.
Is it doing very deep analysis? Am I up against depth 40 stockfish calculations here?
Was this in the rate-me page or as a logged in user? You can mail me with the e-mail in my profile if you'd like to help debug!
Yea it definitely took way more than 10 seconds.
Nice peoject tho! I'm interested in how it'd rate me.
I'm using Chrome on Android and was only using the rate me page.
The promotion menu disappears so I couldn't promote.
It also draws wrong if I preemove a recapture and then change to a new move. More generally, it allows infinite premoves and crashes.
Also, the clock is distracting, especially because the proportional font keeps moving the digits. Besides that, pretty good. And the colors are mushy and low contrast.
1176
Ouch, I did beat the AI.
I've played probably 10,000 anonymous games. My chess puzzle rating is currently around 2100, peak of 2400. Playing against stockfish 4 I can generally win every game. Stockfish 5 is hard.
When I do rated games, I will crush each person in placements and usually get to 2200 or so before I lose the ?
The last time I did a rating run. I won each game and placed at 1977. Then I went on a losing streak of 20 games down to 1300. I am playing 1300s which play like stockfish 5 or harder. These 1300s are playing ridiculously strong compared to stockfish 4.
I wonder what I'm doing wrong.
Yes.
Stockfish 2 is pretty easy, play solid and it eventually does something ridiculous to earn its lower rating. The trick, you have to see the mistake. Basically punches itself in the face, you have to survive until it does this. Make sure everything is defended.
Stockfish 3 is when it starts playing reasonably and only makes a mistake when you've put enough pressure on it.
Stockfish 4 you have to play against it. You need to see depth of about 3-4 moves for both of you and you should beat it every time.
Stockfish 5 is hard, it's seemingly giving you tricks, but if you look at the trick its a crazy computer line where they manage to barely hold on or in most cases you shouldn't have taken what it was giving.
>I've found stockfish at these lower levels doesn't play like humans at all. It will play a near perfect game for 25 moves and then all of a sudden make a horrible and senseless move. If your style is slow and defensive I think you probably fair better. My style is hyper aggressive and stockfish crushes me :)
From my experience stockfish never plays like a human. I do recall there are some bots on lichess which were machine learning to play like a human.
Alas I'm 1100 rated according to op.
Opening theory and tactics are a good place to start.
An easy step is make sure you know a few common openings and have at least one response prepared to each. But also a good understanding of principles. Opening principles are more important, you can wing it if you don't know the theory. There are thousands of videos on YouTube about these topics.
And obviously just don't hang pieces and take your opponent's hung pieces. At 1300 on lichess, your opponent will make a bunch of tactical mistakes which you can just take if you see them, after you can just win if you play solidly for the rest of the game. The way to study here is with puzzles.
Sorry to patronize. I have played thousands of 1300's on lichess and improved after I sat down and studied. But I can tell you, if you're actually 1300 you are not strong enough in some of these areas.
P.S. I don't believe the comment above yours. The dude went from 2200 to 1300??
Like I could provide links to my games. I'm not a liar. I estimate I am a 1800-2000 player.
The thing I don't seem to know, it would seem to me when I play against a 1300 who never makes a blunder or mistake. It then happens 20 games straight. They certainly play various inaccuracies, and I hold myself as well as I do compared to stockfish 5.
In 1 game I was up 2 pieces and the computer didnt think they made a mistake nor blunder. >90% accuracy as a 1300...
I guess, I'm missing some fundamental thing.
So you're saying that you could crush me but are getting beat by 1300s who anyone at 1800 could crush easily? F to doubt.
Edit: for example https://www.318chess.com/elo.html. there is a statistical analysis of lichess data as well but it went down a few years ago.
I play chess because I find it fun, relaxing and a great way to blow off stress. Studying opening and caring too much about rating ruins that. Rated play is awesome because I always get paired with someone of similar skill. I do feel I'm improving slowly, even if my rating does change much. I like to mix things up and often play strategies I know are bad because it leads to fun and novel positions.
And yes, I understand that I'm not a "strong" player. It's pretty obvious when lichess tells me directly I'm in the bottom ~30% lol.
The game does become more interesting as you get better. At least up to 1850 in my personal experience. At <=1400 level the game is more like gambling imo, hoping opponent will fall into trap or miss a move.
It seems like as you increase in rating, chess would become more about who can memorize the most lines or how well you have studied the opening theory. That personally sounds miserable to me.
I actually enjoy the fact that lower rated matches allow many fun lines that are "objectively bad" to be completely viable. Of course, I hope my game will continue to improve, but I'm careful not to take it too seriously.
There's a saying in skiing that I like to apply to all my hobbies:
The best skier on the mountain is the one who's having the most fun :)
But tactics are very fun. They're not about memorization, but about calculation and pattern recognition. If you get good enough, you get to sacrifice all kinds of material if you calculate. Bxg6. Raaaar!
Nitpick: "20% higher" doesn't work when talking about Elo ratings, since it's a system based on rating differences between players (i.e. subtraction).
In particular, if two players are ranked x and y respectively, that's equivalent to them being ranked x+k and y+k for any value of k. On the other hand, multiplying ratings by a constant has no meaning.
> Equivalent lichess.org rating: 965
I like chess, I know i'am not good but it was nice to beat a AI once ;)
As I was playing I felt like I had a definite advantage, which I did as it turns out. But because I was playing a computer I kept second-guessing myself as I didn't yet know it was programmed to blunder. I then managed to resign in a drawn pawn endgame - I wonder if this was used as signal to estimate my rating.
It got my lichess rating right within 300 points after just one game.
But what's even more impressive is that the game felt great to play, I didn't get this feeling that the AI would pick on every of my smallest mistakes like you have with Stockfish. I know there has been several research work [0,1] recently to improve the human-ness of chess AIs, but it's even more impressive that you are able to do it "online", in the course of a single game, bravo. I'd be interested in a blog post with details on the underlying algorithm
[0] https://arxiv.org/abs/2006.01855
[1] https://openreview.net/forum?id=fJY2iCssvIs
Most importantly, what I actually wanted to do was get the game out and analyse it. But now the text was no longer selectable so I had to go grubbing around in devtools.
Turns out the "forget me and play again" button just dismissed the popup and didn't actually start a new game but I had no way of knowing this short of playing another test game in another tab.
Suggestions: (a) don't use modal dialog boxes! Let me create an account in the middle of a game if I want. Let me copy and paste the moves at any time, including after the game is over. Don't have a button to "play again" if it's not actually going to play again.
(b) asking for email addresses annoys the hell out of people. Let me play again without forgetting who I am. If I want my identity to persist across browser sessions then sure, let me create a username and password. If I want to be sure I don't forget the password, sure, let me tie it to an email address. But it should be step three, not step one. Otherwise people will just never come back.
(c) "excited", "Thanks for the game", the eyebrows? It's a bot, it's not capable of excitement or gratitude or facial expressions. Pretending it is seems, well, patronizing. It's reminiscent of chess.com with its cutesy bots (and persistent nagging).
Another option is to consider 3rd party OAuth like Google.
edit: the computer now flagged on my screen, but I did not get any message of a result
Unfortunately, wildly off, I was ~1800 FIDE even when I used to play. Will try a few more games to see whether that was a fluke or whether I should consider getting into chess again.
All in all, great submission, thank you
https://i.imgur.com/g6R1FM5.png
I didn't even know the rules of chess allowed for that, so I've actually learned quite a bit!
This feels like a great thing for beginners.