T O P
  • By - ev-

ev-

Full game on lichess : [https://lichess.org/0KqgCkzl](https://lichess.org/0KqgCkzl)Guess what match? :)Thinking of building this as a mobile app - would anyone be interested in using something like this? Edit 2021-01-11: The beta is almost ready! If you want to use the app, signup at https://knightvision.io/


buttons_the_horse

I'm officially a chess nerd. I recognized it without having to look it up. >!The Evergreen Game: Adolf Anderssen vs Jean Dufresne!< https://www.youtube.com/watch?v=uqzxnz6d7JM


ev-

Well done!


BlueVentureatWork

I'm curious, at what point did you realize what game it was?


Low-Establishment-94

Not him, but I realised it when white played nf6+, because I only remember what the final position looked like


buttons_the_horse

Actually about the same for me. I remember first watching the game thinking white was dead when the black Q captured the knight on f3 with the threat if mate in 1. Then that insane forced checkmate line just stuck with me.


Mateo_O

Man I'm MORE than interested ! This could be so cool to easily record the games we play with friends ! Very cool, thanks for your work !


ev-

Thank you for the kind words! I’ll let you know when there is an app to download! :)


MrOtto47

please ping me when its ready also. will this be android or ios?


ev-

Will do! Hopefully both!


nakovalny

Hey man, if r/chess mods delete your post with the app announcement (they can say it's self-promotion), can you post it in your profile so that we could find it? I'm gonna monitor your profile from time to time, because I really want to try this app!


ev-

I definitely will do that. Thank you so much!


Euprathes

Ping me please!


Mast3rpw

Please ping me too


MooseMusical

Ping me to this is amazing!


ev-

Will do, thank you!


KlingonPacifist

I’d like to get pinged for this too please!


durama00

Awesome. Ping me please (Y)


RajjSinghh

I can see this being useful for IRL streamers and that kind of thing but I feel like it would be way more interesting to have at chess tournaments to broadcast them. Like if you're playing at a lower board on a chess mat, just set up a webcam of phone camera and you have a full record of the game. Awesome project!


MLNotW

I'd love a ping as well


Whisdeth

Amazing! Any rough eta so that I can have the remind me bot ping me at ~the right time. In the meantime RemindMe! 1 month "chess vision app"


Shmockyy

You should ask [chessvision.ai](https://chessvision.ai) to partner with you! You might be able to do something with them or sell your design to them.


Aalynia

YES! I’m a shitty, shitty chess player. But my five year old is amazing (well, as far as five year olds go—his OTB rating is 571. He’s been playing 5 months). Anyway, ever see a little kid try to notate? Handwriting is terrible. Writing letters in reverse (b,d). Slow. This would be a great way for younger players to keep track of their OTB games. I could see coaches recommending such an app to their students (and their students’ parents). Even older players who may struggle with notation—dysgraphia, tremors—could use this to quickly keep track of their OTB games. I can see a lot of usefulness for this.


ev-

Great feedback and that's amazing to hear about your son. I have a 4.5 year old that refuses to learn the rules(His own rules include the knight being thrown into the ground from time to time). All the best wishes for your son! I started playing when I was 15 / can't imagine how much better I would have been if I started that young!


Aalynia

To call a spade a spade, the kid is profoundly gifted. It’s been hard dealing with him because he’s so bored all the time, but chess has been his siren’s song. I hope he sticks with it. He even loses graciously (95% of the time lol) because he’s intrigued by how he lost. It’s delightful. Just don’t be that parent that pushes their kid to hard. I see so many parents at the chess center yelling at their 5-6 year olds for blunders and such. I’ve been asked what I do to incentivize my son and make him a good player. People are crazy. I’m sure you’re not doing that, but yeah. And sometimes kids fall in love with things you’re interested in when they’re a touch older! My 8 year old just started coding video games. That’s also awesome. 4.5 is still so young. Have you seen that story time chess set? Or No stress chess? No stress chess is fun because even once you know how to play, having to move based on what a card tells you to do can make for some interesting games! Good luck with the app development! I look forward to seeing it come out!


jonhuang

I could have written this post. This would be amazing for my kid too!


xAptive

I'm very interested in this. Would be cool to play faster time controls without having to log. But yeah, would have to be a mobile app. What language did you use for your current program?


jaugjaug

Hi, I'm one of the developers behind this. It's all written in a relatively new language called Julia which I'm a big fan of.


senya77

Will you make it open source?


xAptive

Wow, never heard of that, but looks really cool. At first glance, it reminds me a lot of Go (not the syntax, so much), but maybe better for data applications.


_3NiGMa_

I hate that Julia seems to be python++++++, has all the nice features without any of the drawbacks *cough*speed*cough*


killnars

Nice Julia is really taking off! We are using it at work (engineering related)


PenPaperTiger

Yes, my chess club would love this


HuckleberryStock3091

Very impressive. if you used TF it shouldn't be too complicated to transfer the model to a mobile app. keep us updated!


DenseLocation

I would pay ($3-5) for a tool like this.


porn_on_cfb__4

so did you make the program, or are you planning to make it in the future?


ev-

The video is made from the program. It works great, but it is not a mobile phone app yet.


porn_on_cfb__4

Very cool. Do you have a desktop Version, or a github where we could track the progress?


raymond459020

how accurate is it? does it make mistakes occasionally?


Dangerous-Bid-6791

I feel like it must: surely it at least gets confused if it sees something that is not a chess piece and possibly may be incompatible with certain chess piece designs?


baconmosh

Depends on how many images it was trained on, and how varied they were.


3_horned_Bull

Very interested!


sticky_water

Would absolutely love something like that!


theymayneverknow

subscribe! feels like the best games my friends and I play are OTB, having a tool like this would be amazing! using pure memory to remember interesting endgame positions to analyse is tough (for a bunch of 1300-1600s at lesst)


FowlKreacher

I have been looking for exactly this for months! Most of the OTB games I play nobody else feels like documenting, but I feel like I would learn so much from them!


ev-

I can totally relate! I'll let you know when there is an app available!


Vicubix

This would make a really useful mobile app! Let us know if you release the source code, I'd be curious to try to make it work with React Native or something.


sotolibre

This would be awesome. As others mentioned, it would be great with friends. When I play OTB with a friend I like to record the moves on the Lichess mobile app or something so I can study it later, so this would be a life saver


Clever-username-7234

I would love it!


dmvaz

> Thinking of building this as a mobile app - would anyone be interested in using something like this? Absolutely yes. I think you'd have a lot of people interested in this.


p4nz3r_95

Man ping me please when the app is testable! I dont know match about AI but Im a Game Developer so if you need something just ask


ev-

Will do, thank you!


BraadPrit

Can it record the time used per move too?


ev-

Yes in theory. It recognizes when a move is made, so could record the time as well.


draktitor

Super interested


DavyKer

Definitely. A friend and I came up with exactly the same idea but never implemented it.


DavyKer

You are using a very nice chess set in the video, with quite standard piece shapes. I suppose you have considered the fact that app users will have a wide range of piece types.


ev-

Yes of course. The one in the video is a classic board with Staunton pieces. We have a set of other boards that are quite different and not as “nice”, and it works well for those as well. I believe we can get the program very stable for most boards, but of course this is a early release!


Zebrinny

Dude this is incredible. Could you technically stream from a phone to a computer to let it do the processing?


ev-

Thank you!:) Yes the video streams from the phone to a desktop-app right now, but it could also be made directly on the phone.


Zebrinny

Super impressive stuff, huge practical potential for larger scale adoption. Awesome work! Imagine everything from better tournament broadcasting, to people going to their first otb tournament to record their own game and be able to run analysis right after without inputting it themselves. Streamers with a better board view during irl actions… seriously cool project. I guarantee chesscom would love to have their hands on this software.


Sam5253

Something tells me that having a phone recording a tournament game would somehow be against the rules. But outside of tournaments, I agree that it has great real-world potential!


Sufficient-Piece-335

The arbiter can authorize it, but agree that it's better to use a webcam though.


HeroDGamez

This has a lot of potential for organizers that cannot afford magnetic boards to stream the moves. Would be great to make chess games more accessible in general.


h00pz

DGT gonna show up to your house


causa-sui

Please post git link


Enfoil

Well, if this is real and actually works, you just made DGT boards obsolete.


Vistaf

The problem is reliability, computer vision is always great until it isnt, DGT boards work, and work (semi) reliably. This software is great, but in order to replace DGT boards it would require a massive amount of testing.


notcaffeinefree

Cost plays a factor too though. DGT boards are freakishly expensive. Sure for top tournaments it's not necessarily unreasonable, but for others were cost might be more of a factor a webcam and free software is way more appealing.


thelambdamale

How much do those boards they use in the top tournaments cost?


PolarTimeSD

> computer vision is always great until it isnt Though, as an AI scientist who always rags on ML, this is actually a pretty great usage because the problem space is pretty small, so if there are bugs it wouldn't be hard to fix.


Kennzahl

Computer vision has come a loooong way in the last 5 years. We have cars driving themselves with just the input of cameras. There is no reason to believe this wouldn't work consistently if the model is trained right.


rs10rs10

This is not going to be an issue for this application. It is definitely simple that this will work (with less error than a human writing down the moves).


gst_diandre

It can be nearly perfect if made to start from the starting position and assume that only one piece is being moved at a time. Unless you're playing hyperbullet or something. Recognizing any position from the get-go is more error prone.


ev-

It works! Need to make it into something more user friendly / accessible but working on it! :)


GoatBased

Can you share the rough version as-is for people who want to explore?


[deleted]

I'm a computer engineering student and I certainly would like to know what references you used for learning to do this.


ev-

Hi! For a good theoretical introduction to machine learning I really like "the elements of statistical learning". There is a lot of online resources for learning about computer vision, machine learning and neural networks. We do a combination of a couple of different approaches. Neural networks for piece recognition, some more traditional computer vision for finding the board and tracking things in video. Best way to learn is to start coding on something you are interested in!


PJBthefirst

How well does the network perform with chess sets of different color schemes or sets with unique piece designs?


ev-

Very unique piece designs is definitely a problem. We're using a couple of different methods all at once to avoid scenarios where the network would have a hard time.


bautim

could you add a caliber option (for an app) where you can analize your own pieces and relate them to standar chess . Is just an idea, i dont know how could It work. (sorry for grammar, english is my second language) Excelent work, love it.


ev-

Yes I think that calibrating should be doable down the line. Nice idea!


realeaty

could have "tagged" tournament pieces (officially approved for the app) that offer guaranteed detection


rosinsvinet_

If players put the piece down real hard, you can add a "!" As an annotation


ghillerd

Brb building dgt board with force sensors


hunter_starr

That's great, good job! I would definitely download it if it was an Android app. Can it process a recorded video from a game as well? I think that would be a very useful feature. Is it a private project or you're planning on releasing the code as open source?


ev-

Thank you! Processing an already recorded video is a cool feature, will definitely think about that. The plan is to build an android app as the next step!


ghillerd

Just hold the video up to your phone camera. Processing time is a little high though ;)


CounterfeitFake

Yeah, my main thought was that I might want to record the game on a gopro on a tripod and then process the games later at home on my PC. So processing a recording would be nice, even if the input formats are limited.


ennuinerdog

How good is the ai at recognizing promotions?


Mutte23

That's awesome, are you planning on releasing it on android?


ev-

Yes that is what I'm thinking if there are others who would want to use it!:) It is a desktop app/julia-program right now.


Mutte23

Of course people want to use it, I remember playing some games at my university and couldn't study it after the game because my notation was garbage, couldn't figure out what did I play, with an app like that I could just play freely while my phone is recording and check the png after the game.


ev-

That’s exactly the same for me. So many 5 0 games played that I never can replay :)


ravonaf

Use it? With the right marketing, features, and a clean interface, you could throw it up on an app store for a couple of bucks and make a LOT of money.


GreatSymphonia

I would even pay for it! If you put it on GitHub, I would be very interested to participate :)


xYsfOW

please make we know when you release the program. This is great.


ev-

Thank you! I definitely will!


SaneSiamese

Random question... does your reddit name stand for Expected Value Minus?


ev-

Haha yes negative expected value. If you’d look at my chess games you’d realize that I win 99% of my games using luck


Alternative_Big3215

So glad that you choose the Evan's gambit


buddaaaa

This is so cool and so genius and has a *ton* of practical value. Would take any live, irl chess content (educational, entertainment, or otherwise) to the next level. Amazing work, OP!!!!!!


ev-

Thank you for the kind words!


bobsstinkybutthole

That's dooope. What libraries did you use?


jaugjaug

Mainly the Julia packages Flux.jl, Chess.jl, ImageTransformations.jl and Mux.jl


Scalarfieldtheory

Is there an app which reads in chess notation and creates for example a file which you can read into lichess?


Wavy-Curve

Damnit. I literally had an idea to make this yesterday.


PauloSantoro

Very good! How can I subscribe to know when app is available, please?


ev-

Thanks! I’ll let you know when it’s ready!


chessicaParker

This guy created this years ago : https://www.youtube.com/user/spiderchess32 http://webcamchess.fr/


de_animo

This is really impressive! Does it work with various chess sets (different boards and pieces)?


ev-

Thank you! Yes it does. However really odd boards/pieces (like a starwars chess set) will be problematic.


de_animo

That's fantastic, can't wait for the app :)


1_Chaos

When I start playing chess, I remember that I searched for a program like this, and I didn't believe it didn't exist. But I left it aside because I thought I should have a reason to. looks like I was wrong. To give you an idea on how this is helpful: Here in Brazil, 2021 Brazilian championship was played, and unbelievably there were no dgts boards. In other words, the biggest championship in the country, it didn't have the most important thing for a tournament of this size. The streamers however wanted to broadcast the tournament and managed to get 2 cell phone cameras placed and streamed online, where the streamers had to pass what they were seeing onto a lichess board. Such a program would be very useful. I hope you stay on it. Congrats for the work.


ev-

That is a great story, thank you. Will definitely stay on it, I will let you know as soon as an app is available!


TwitchyMcJoe

As long as the program could take a recording as an input, this would be great. I think that some clubs/organizations might not like having a phone recording the game, but they would be okay with any other camera on a tripod. Could you make it take an mp4, avi, etc. as an input?


ev-

Thank you for the input! That would also work. Started out using a web camera before switching to a phone.


TwitchyMcJoe

One other cool idea: this could eventually morph into an alternative to investing in expensive DGT boards.


ev-

Very cool! That was the idea in the beginning. Played a lot of tournaments where only the top 3 boards were streamed (because of expensive DGT boards). Would be cool to play at a tournament where every game was live recorded :)


recursiveorange

Dude, don't open source it or give it away for free. This interesting app could grant you a wealthy life.


ikefalcon

This is amazing! Would be super useful for blitz/rapid OTB tournaments where players don’t take notation. Or for short time situations in classical


RattleyCooper

Sweet :D


thermomax

I run my school's chess club and it would be quality to show chess tournament finals on the projector screen in real time


qlunch

Very interested in this! Desktop or android app would be fantastic.


TheHigherSpace

Oh nice .. So no more commentators complaining that they didn't get the PGN from a competitor website ..


CubesAndPi

Awesome work! Any details on the implementation? Just curious :)


jaugjaug

Hi, I'm one of the developers behind this. It's all written in a relatively new language called Julia. We use a mix of traditional machine learning and neutral networks, mostly with the Flux.jl library.


MySpaceLegend

Very interesting! Would you mind explaining how this works? How does the software recognize the pieces?


jaugjaug

Hi, I'm one of the developers behind this. We use a mix of neural networks and other learning methods to piece together a prediction of the game


powlette

This is a fancy way of saying, we start with the starting position and then compare frames to see which square no longer has a piece on it. Then we can infer what move just happened. We have no idea what piece is where unless it got to where it by a series of moves from the starting position. If you knock over a couple pieces and put them back in slightly different locations, all bets are off. Source: am a developer.


shamaalama

So where does the ML take place? If it’s as simple as seeing that the pawn was on e2 a couple of frames ago and now its on e4 what is the need for ML? My guess is the ML is necessary to filter out and not process frames where the player is moving a piece?


powlette

Here's two stills laid on top of each other from the video. It doesn't take HAL to see what just happened and it can be done by simply checking for how much of the board is visible at each coordinate. Some people would call this "machine learning" but I'd call it a bunch of if statements and twiddling with how "filled" a square has to be to determine a piece is on it. I'm guessing nudging the camera slightly also causes the detection to break. [https://i.imgur.com/Wr95JTY.png](https://i.imgur.com/Wr95JTY.png)


MySpaceLegend

That's cool! Have you like taken a million different pictures of chess pieces and trained the AI to recognize them or something?


onowahoo

You didn't really answer the question


anichari

Superb! Cant wait for the app


ev-

Thank you!


snkscore

This is awesome. Does it only work for specific boards and camera angles?


jaugjaug

We have trained on a few different boards, and the goal is that it should work on most traditional boards, but I wouldn't hold my breath for more obscure pieces like "Lord Of The Rings" sets. It seems to be quite robust to the viewing angle.


ev-

Thanks! No it works as long as the camera can see the board. Weird piece layouts can be a problem, want to make it more resilient later on.


Schaakmate

Amazing! Everybody will want this!


ev-

Thank you!


charliealphabravo

this is incredible!


Crooklar

Something to sell to streamers? And Ytubers


AuthoritativeComet

How does it respond to nonsensical moves, like when [that street hustler tried to take 2 pieces in a single move](https://youtu.be/U5vnpOp0U_g), or [players knocking pieces all over the place in a time scramble](https://youtu.be/vNIpKPmJzYY)


ev-

It tries to recover from positions that doesn’t make sense. I’m really clumsy so we got a lot of training data with positions where I accidentally knock pieces over :)


CounterfeitFake

Could be neat if you could have it report on positions it thought were confusing and maybe even preserve video of the previous X seconds, etc. so you could correct it manually if needed.


ev-

Makes sense. Will look into that!


saidthewhale

This is a amazing! I hope to see it on android :) Remindme! 1 year


RemindMeBot

I will be messaging you in 1 year on [**2022-11-05 19:02:01 UTC**](http://www.wolframalpha.com/input/?i=2022-11-05%2019:02:01%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/chess/comments/qneak9/made_a_program_that_in_real_time_creates_a_pgn/hjg9hdn/?context=3) [**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fchess%2Fcomments%2Fqneak9%2Fmade_a_program_that_in_real_time_creates_a_pgn%2Fhjg9hdn%2F%5D%0A%0ARemindMe%21%202022-11-05%2019%3A02%3A01%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%20qneak9) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|


ev-

Thank you:) hopefully sooner than that!


saidthewhale

Can't wait!


5pitt4

Is it Open Source?


Ganda1fderBlaue

Is it actually able to distinguish the pieces, or does it get the information needed just from the board itself?


ev-

It can distinguish pieces from each other. Then it tracks states-board positions as well. We play a lot of 960 which works very well with that approach :)


Ganda1fderBlaue

Cool


kwright9509

That is awesome! Please share your progress and updates.


DeltaT37

Remarkable stuff man I'm surprised this doesn't exist already!! If you can get it to mobile, that's huge! Changes live chess forever


thiscarhasfourtires

RemindMe! 2 months


sequentialusername

This is huge


nailenTV

Isn't it a computer graphic implementation, what kind of AI are you referring to? Since my studies I hate computer graphics, your implementation is awesome!


fireheart2008

wow


ravonaf

You might want to go into hiding. Some big companies are looking to hire hitmen about right now.


coldfact

Add clock recognition (which side’s clock is ticking and times for each) and you have a full DGT replacement!


mraoos

When would it make sense to check up on you? End of January?


ev-

Hopefully! I’ll post an update as soon as there is a beta available!


LegacyLemon

This is incredible. I'm going to keep an eye on your profile for updates


withaining

bro this is fucking amazing.


douglad17

Chess24 could do with this to save the poor presenters from having to keep up with blitz games!


sidarok

Man, this is genius. I imagine analysis, book reading etc would be amazing with such a tool.


nonbog

What did you write this in? Is the source code available to look at? It sounds great!


Historical-Truth

Really cool project! Do you plan to release it under a free software license?


davgmo

Dude this is fantastic! Please ping us when ready to try it out! Happy to help you beta test it if it helps!


ev-

Thank you! I definitely will let you know when it’s ready for testing!


Willing-Elevator-695

Would love to have this app


quixoticM3

Easy to do…


Aromatic-Teach-4122

This is really great job. Waiting for the app. Also, just a thought: this can probably be modified to be used as a tool where people can play in realtime with physical chess sets remotely with their friends. That would really be awesome as it will give people a chance to get the real feel while enjoying all of the digital benefits.


ev-

Cool idea! We talked about building that, might be possible(without building a board), but not 100% sure.


thecubersahil

Love this!!


NuBluDru

Impressive


SpiritualCat553

Wow! This is a game changer. Really looking forward to this project.


queerstomper

Absolutely incredible work


relevant_post_bot

This post has been parodied on r/AnarchyChess. Relevant r/AnarchyChess posts: [Made a device that in real time tracks your game from a physical board using your eyes and hands. Requires nothing more than a piece of pampers and a pencil.](https://www.reddit.com/r/AnarchyChess/comments/qnsiqo/made_a_device_that_in_real_time_tracks_your_game/) by abcdefgother [^(fmhall)](https://www.reddit.com/user/fmhall) ^| [^(github)](https://github.com/fmhall/relevant-post-bot)


_klx

DGT in shambles


The_bored_jedi

This is awesome!! I would love to see where this goes. Remindme! 6 months


shimsimma

This is awesome dude. Good work.


shimsimma

If you need a backend developer at any point let me know. Id be happy to help.


_limitless_

and, i'm going to guess, your exact board and piece set.


Cornel-Westside

Reset the counter


r_chess_bot

Counter is reset! There's been no reset for: 22 hours, 25 minutes and 39 seconds Congratulations on your first reset /u/Cornel-Westside! [Last reset](https://www.reddit.com/r/chess/comments/qmw1gs/still_cant_believe_i_found_this_idea_in_a_blitz/hje0wlw/) was on 2021-11-05 07:34:02 by /u/CautiousRice


Replicadoe

oh wow thats very impressive good job man


peterpotamux

I'm interested in your program. Anything you can share or at least document? Thanks


louter_genieten

Is it possible to recognize multiple boards from a eide stance? For a or my chessclub that would be awesome and make the expensive DGT and horrible working DGT obsolete.


AlMansur16

Man, now I have no reason to buy a DGT board.


cantab314

Can it work off stills? There's a good game on the other sub to test it out with :-D


kaperisk

Very cool. Would be great to market to chess clubs