On just a quick glance, the breadth of the topics covered here is stunning.
I also liked this quote from Lecture 23.
> A lot of times we ... confuse value with complexity.
> And many of the things that were the simplest in this subject are actually the most powerful.
> So be careful about confusing simplicity with triviality and thinking that something can't be important unless it's complicated and deeply mathematical.
Nice observations. The trade off is he doesn't go too deeply into anything, but it's a great survey. Especially for people not acquainted with AI outside of ML.
>So be careful about confusing simplicity with triviality and thinking that something can't be important unless it's complicated and deeply mathematical.
That's true of most things in CS, I think. Maybe life in general. I'm educated in cybernetics, and our most advanced stuff is hardly ever used. Too costly to implement, tune and test, very few areas where it's truly necessary. PID will work well enough for your application 99 times out of 100. Model Predictive Control is very cool, but it's going to cost you a lot. In places that need advanced regulation but where strict guarantees are not needed a properly trained ANN will beat a more analytical approach and require way less education to pull off.
I had the same question, so I was sure to be there to audit this class. It was great. He goes into Geoff Moore's product distribution model... underscoring the importance of finding early adopters rather than already established huge guys who think a new technology is too risky. He acknowledges that if you build it they're not guaranteed to come. He shares several anecdotes from his time building Ascent Technologies, which I believe is airport-plane-traffic-management software (with AI) used by 80% or so of all airports in the U.S.
I took this class given by Prof. Patrick Winston a while back, 43 or 44 years ago. I liked it so much back then, it served me well in grad school and for many years after. Next week I'm visiting with an AI company on behalf of potential investors.
I took his AI class, 6.258, 45 years ago, in the Spring of 1971. It was probably the first or second time he taught it. He's a great teacher and inspired me to focus on AI. One aspect I remember was that our exams were all take home exams that we had several days to work on. They were great learning experiences.
Yes, I was a math major at MIT in 1969. I started taking EE classes and eventually ended up with a second degree in EE/CS. It was during this time that I took Winston's AI class. The lectures took place in the Institute's largest lecture hall; it was a popular class for good reason, Prof. Winston's class was great.
I'd had take-home tests before but Prof. Winston's were the first and only 24 hour tests that I've ever had. I remembered that he said that they had tried out the problems on some of the TA's for the class and that it would only take us a few hours to finish. For me at least, it took 24 hours and I still wasn't done.
Prof. Winston's class material was mostly in Lisp, but we also looked at Planner, kind of a DSL for AI. I was very impressed by a program presented by Winston that was capable of performing symbolic integration. Prof. Winston remarked, after we had discussed the program, that really, it wasn't a complicated system, just a simple algorithm with a data base of facts about integrals.
He also had a very funny, perhaps apocryphal, story about Joseph Weizenbaum's program ELIZA, a program that carries on a conversation with its user in the manner of a psychotherapist (like the doctor command of Emacs). Again, it turns out to be a simple program in Lisp with a small database of keywords and responses. Apparently, Weizenbaum had been working on the program on an MIT timesharing system and another professor had seen he was working late so he used the system's chat program to attempt to communicate with Weizenbaum (somewhat like the Unix talk command that lets one user contact another currently logged in user). However, Weizenbaum wasn't actually there he had gone to sleep and had just left his terminal with ELIZA still running and connected to the I/O of the terminal. The professor asked a question like "What are you working on so late?" and ELIZA responded in it's typical fashion: "Is there a reason that it's important to know why I am working so late?". The professor, a bit put off said something like "You were on the computer late and I was just curious." and ELIZA said back "Why do you feel that you are curious?" the conversations continues: Prof: "Why are you acting so strangely?", ELIZA: "Tell me more about your feelings that I am acting so strangely."
Finally, the professor is fed up with the crazy indirect answers and just calls Weizenbaum on the phone directly. At the late hour he is answers sleepily: "Hello" the professor says "Why are you acting so strange tonight?"; Weizenbaum replys "Why is that you are asking me why I am acting so strange?"
I started programming while in high school. At the time 1967, there was no way to do programming at home, there was no internet. I taught myself by reading a 1965 edition of McCracken's A guide to FORTRAN IV programming. The first program I wrote (on paper) was a program to solve linear programming problems using the Simplex algorithm which I had seen a high level description of. I punched up a program to do it on cards using the high school's data entry IBM 026 keypunch machine. I gave the cards to a friend that gave my program to someone over in the school district's administrative building to run on the school districts only computer (I think it was an IBM 1130).
Naturally, my first program didn't work. So it was back to rereading the book on programming and starting out with simpler examples. Turn around time continued to be about 3 days but I viewed programming as a hobby, a bit like being a Ham radio enthusiast. At MIT I continued to view programming as a hobby, but after a while I realized it was worth taking it seriously.
I didn't always do well in my undergraduate classes. I had gone to very bad schools growing up and everything was too easy for me until I got to MIT. I finally learned how to study by the time I go...
Does anyone think these lectures would be a good place to start for someone with web development experience (LAMP + JS) and zero AI & ML knowledge ?
If yes, what is missing from these lectures ( related to AI or ML or Deep Learning ) which has been discovered or developed recently and should be learned during the start ?
I have a side project to monitor the homepage of every domain and I want to detect the type of site - ecommerce, blog, forum etc. I've just started on it and ultimately I want to be able to automatically extract data, such as product information from ecommerce sites. Would these videos help here or is there anywhere else that someone would recommend. I know I sounds very naive here... and I am, but there might be someone here who can give me a steer. I've started looking at AI/ML... or whatever its now called and getting a bit confused!
It is possible for subject recognition, but the data extraction would be much more difficult. For the simplest subject recognition, you would need a corpus with all the kinds of subjects that you want to recognize, and many examples of each. It would be very tough as a first project in machine learning.
Edit Note: I'm very new to machine learning as well, but this is what I've gathered so far.
> ultimately I want to be able to automatically extract data, such as product information from ecommerce sites
A little OT, but although you could probably train models to figure that out, many (if not most) eCommerce sites should already be tagging their sites with semantic tags for product information, since it assists with indexing for search engines.
You could absolutely do it. Here's a few things you could do to get started (feel free to email me if you need more specific help, or have any additional questions- my email's in my profile):
1. Get as much data about each site as possible. Try to find data that you think will separate the non-ecommerce sites from the ecommerce sites. This data should be in a table form- think of an Excel spreadsheet, where each row represents a different site.
2. Label the data as belonging to each type of site. This is going to be tedious and take some time. You can also try hiring people via Mechanical Turk to do this.
Check your results and see how happy you are with them. Weka has a lot of visualization capacities, so you can see how the data that you've collected aligns with the different types of sites.
Now, you can start iterating, which is the key part of any ML project. Consider which aspects of the model you think you can improve on- adding more data, adding more kinds of sites, using a different machine learner.
> 1. Get as much data about each site as possible. Try to find data that you think will separate the non-ecommerce sites from the ecommerce sites. This data should be in a table form- think of an Excel spreadsheet, where each row represents a different site.
This will be the hardest step. Feature extraction from html for identifying specific things like products, for any given site, is very hard (in my experience, which I will admit is pretty limited). Would love to be proven wrong though.
I was thinking you could approach it as a document classification problem, where you extract the text from the HTML and work with that.
I had a lot of success finding duplicate bug reports by comparing the text from the bug reports with reference documents in a variety of topics (e.g. security, networking, C++), and getting a sense of how similar the text is to the reference text. That gives you a score of how relevant each subject is to the document.
You could do something similar here- download the text from 10 ecommerce sites, run some sort of topic extraction algorithm (like LDA) on it and then compare the text from the sites you're trying to classify with the text from the reference sites.
Professor Winston (the lecture in these videos), also teaches a higher-level, seminar based AI reading class called "The Human Intelligence Enterprise".
Could you recommend any good books that would supplement this course? Also, why does this course seem so weird. What's with the "communication" assignments. Also, what's the deal with his project "genesis." Really interesting stuff. I hope I get a chance to pick your brain.
Minsky's Society of Mind would be a good launching off point. Other than that particularly famous example, I would just check out any of the authors of the papers in that link.
This class fulfills a communication requirement at MIT, so there are a one-page response/reflections due at each class. These reflections basically just ensure that you have a decent grasp at the main points of the paper, so that you would be ready for a seminar-style discussion in class.
Genesis is pretty interesting. Here's a link to the details: http://groups.csail.mit.edu/genesis/ But in short, it's an attempt to get computers to understand stories — and to discover what is the fundamental difference between the human mind and all other animals (spoiler alert: Winston's group believes it's the ability for humans to take two concepts and merge them into a new concept, indefinitely. AKA: creating "stories"). So the goal of Genesis is story-understanding.
Thanks for the link. I read through the three main papers. Very interesting. The project seems to be dead though. Why hasn't there been any development in the past few years? Did you guys reach a hurdle you couldn't overcome? Or have people's interests shifted? Thanks for taking the time to answer Kenneth.
Amazing! I hope you don't mind but I cross-posted to /r/artifical. Thanks for the videos! This will make sure I get nothing done tonight but I should be smarter in the morning!
I took this class a couple of years ago. It's outdated and overrated imho, with nowhere enough mathematical rigor to be useful. For example, the discussion of support vector machines (and classification in general if I recall correctly) was limited to two dimensions so that you didn't need linear algebra. The class also spends a lot of time on problems like path finding that you should be able to solve with your standard CS algorithms toolkit or just "logic" rather than needing to reach for anything that deserves the name "artificial intelligence" (at least today). Prof. Winston furthermore spends way too much time on vague truisms that may sum up or organize what's in his brain but aren't helpful to students. ("What if the answer doesn't depend on the data at all? Then you've got the trying to build a cake without flour.")
I hate to dismiss something as ambitious as this course and just tell people to blindly follow trends, but my honest advice would be to just skim these notes if you're interested and go take a normal machine learning course instead.
It depends on what you want out of it! If you're looking for immediately practical skills, of course you're better off taking 6.036 (machine learning). That's not the point of 6.034, which is something more vague that emphasizes intuition over rigor. Personally I think that offering both - a skills-oriented class and an idea-oriented class - is super cool and unique and I wish that pattern was more common.
54 comments
[ 4.5 ms ] story [ 124 ms ] threadI also liked this quote from Lecture 23.
> A lot of times we ... confuse value with complexity.
> And many of the things that were the simplest in this subject are actually the most powerful.
> So be careful about confusing simplicity with triviality and thinking that something can't be important unless it's complicated and deeply mathematical.
That's true of most things in CS, I think. Maybe life in general. I'm educated in cybernetics, and our most advanced stuff is hardly ever used. Too costly to implement, tune and test, very few areas where it's truly necessary. PID will work well enough for your application 99 times out of 100. Model Predictive Control is very cool, but it's going to cost you a lot. In places that need advanced regulation but where strict guarantees are not needed a properly trained ANN will beat a more analytical approach and require way less education to pull off.
Wish you listed a method of contact. I really enjoy having conversations with more experienced hackers.
I'd had take-home tests before but Prof. Winston's were the first and only 24 hour tests that I've ever had. I remembered that he said that they had tried out the problems on some of the TA's for the class and that it would only take us a few hours to finish. For me at least, it took 24 hours and I still wasn't done.
Prof. Winston's class material was mostly in Lisp, but we also looked at Planner, kind of a DSL for AI. I was very impressed by a program presented by Winston that was capable of performing symbolic integration. Prof. Winston remarked, after we had discussed the program, that really, it wasn't a complicated system, just a simple algorithm with a data base of facts about integrals.
He also had a very funny, perhaps apocryphal, story about Joseph Weizenbaum's program ELIZA, a program that carries on a conversation with its user in the manner of a psychotherapist (like the doctor command of Emacs). Again, it turns out to be a simple program in Lisp with a small database of keywords and responses. Apparently, Weizenbaum had been working on the program on an MIT timesharing system and another professor had seen he was working late so he used the system's chat program to attempt to communicate with Weizenbaum (somewhat like the Unix talk command that lets one user contact another currently logged in user). However, Weizenbaum wasn't actually there he had gone to sleep and had just left his terminal with ELIZA still running and connected to the I/O of the terminal. The professor asked a question like "What are you working on so late?" and ELIZA responded in it's typical fashion: "Is there a reason that it's important to know why I am working so late?". The professor, a bit put off said something like "You were on the computer late and I was just curious." and ELIZA said back "Why do you feel that you are curious?" the conversations continues: Prof: "Why are you acting so strangely?", ELIZA: "Tell me more about your feelings that I am acting so strangely."
Finally, the professor is fed up with the crazy indirect answers and just calls Weizenbaum on the phone directly. At the late hour he is answers sleepily: "Hello" the professor says "Why are you acting so strange tonight?"; Weizenbaum replys "Why is that you are asking me why I am acting so strange?"
I started programming while in high school. At the time 1967, there was no way to do programming at home, there was no internet. I taught myself by reading a 1965 edition of McCracken's A guide to FORTRAN IV programming. The first program I wrote (on paper) was a program to solve linear programming problems using the Simplex algorithm which I had seen a high level description of. I punched up a program to do it on cards using the high school's data entry IBM 026 keypunch machine. I gave the cards to a friend that gave my program to someone over in the school district's administrative building to run on the school districts only computer (I think it was an IBM 1130).
Naturally, my first program didn't work. So it was back to rereading the book on programming and starting out with simpler examples. Turn around time continued to be about 3 days but I viewed programming as a hobby, a bit like being a Ham radio enthusiast. At MIT I continued to view programming as a hobby, but after a while I realized it was worth taking it seriously.
I didn't always do well in my undergraduate classes. I had gone to very bad schools growing up and everything was too easy for me until I got to MIT. I finally learned how to study by the time I go...
If yes, what is missing from these lectures ( related to AI or ML or Deep Learning ) which has been discovered or developed recently and should be learned during the start ?
https://ocw.mit.edu/courses/electrical-engineering-and-compu...
https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Really stands out as one of the best lectures I have had the pleasure of watching.
Edit Note: I'm very new to machine learning as well, but this is what I've gathered so far.
A little OT, but although you could probably train models to figure that out, many (if not most) eCommerce sites should already be tagging their sites with semantic tags for product information, since it assists with indexing for search engines.
Here, for example, is the Schema.org for a product (https://schema.org/Product).
1. Get as much data about each site as possible. Try to find data that you think will separate the non-ecommerce sites from the ecommerce sites. This data should be in a table form- think of an Excel spreadsheet, where each row represents a different site.
2. Label the data as belonging to each type of site. This is going to be tedious and take some time. You can also try hiring people via Mechanical Turk to do this.
3. Use [Weka](http://www.cs.waikato.ac.nz/ml/weka/) or [Vowpal Wabbit](https://github.com/JohnLangford/vowpal_wabbit/wiki) to run some preliminary estimations on the model. Weka and VW are great tools as they come with a lot of the configuration done out of the box, so you won't have to write any code to get started.
Check your results and see how happy you are with them. Weka has a lot of visualization capacities, so you can see how the data that you've collected aligns with the different types of sites.
Now, you can start iterating, which is the key part of any ML project. Consider which aspects of the model you think you can improve on- adding more data, adding more kinds of sites, using a different machine learner.
This will be the hardest step. Feature extraction from html for identifying specific things like products, for any given site, is very hard (in my experience, which I will admit is pretty limited). Would love to be proven wrong though.
I had a lot of success finding duplicate bug reports by comparing the text from the bug reports with reference documents in a variety of topics (e.g. security, networking, C++), and getting a sense of how similar the text is to the reference text. That gives you a score of how relevant each subject is to the document.
You could do something similar here- download the text from 10 ecommerce sites, run some sort of topic extraction algorithm (like LDA) on it and then compare the text from the sites you're trying to classify with the text from the reference sites.
It too, is an incredible class. Here's the schedule & linked papers from last semester: https://courses.csail.mit.edu/6.803/schedule.html
(Disclaimer: Professor Winston is my current advisor)
This class fulfills a communication requirement at MIT, so there are a one-page response/reflections due at each class. These reflections basically just ensure that you have a decent grasp at the main points of the paper, so that you would be ready for a seminar-style discussion in class.
Genesis is pretty interesting. Here's a link to the details: http://groups.csail.mit.edu/genesis/ But in short, it's an attempt to get computers to understand stories — and to discover what is the fundamental difference between the human mind and all other animals (spoiler alert: Winston's group believes it's the ability for humans to take two concepts and merge them into a new concept, indefinitely. AKA: creating "stories"). So the goal of Genesis is story-understanding.
I hate to dismiss something as ambitious as this course and just tell people to blindly follow trends, but my honest advice would be to just skim these notes if you're interested and go take a normal machine learning course instead.