REVIEW GUIDELINES Each review should start with the following information 1. Project number and Title Project: 12 Title: Autonomous Learning in a Simulated Environment 2. Should this project be considered for the Best Project award? (yes/no) no 3. Should this project be considered for the top 3 project awards? (yes/no) no 4. On a scale of 1 to 10, how would you rate the overall organization/clarity of the project report? (1-10) 10. 5. On a scale of 1 to 10, how would you rate the overall project idea? (1-10) 8. 6. On a scale of 1 to 10, how would you rate the overall research contribution of the project idea, methodology and/or results?  (1-10) 8. Then write approximately 2 pages of helpful feedback to the project's author(s). The following questions should help you organize your feedback: Abstract ---------- Good. Introduction --------------- This introduction is more focused than the introduction in your proposal. It is clearer and easier to read. You are on the right track with the structure. I also like the 'Changes from the Proposal' section. You could improve the motivation. It should be more precise. Make it more about game playing and less about solving general artificial intelligence. With these changes, the introduction would become much stronger. The last two paragraphs of the introduction are good, but they were written in a completely different style than the first five paragraphs. You changed from a passive voice to an active voice. Also, words like 'thrilled' would not be acceptable in a publication. Prior Work -------------- The prior work is also too broad in scope. Try to frame it for the specific problem you've addressed in the paper, which is showing how the snake can learn. One long term goal for the field of AI is general game learning. Machine learning solutions are created for robots to not only win at specific games, but to learn how to play them too. Because the algorithms learned how to play the game, they can be applied to different games, and one day maybe even real-world problems. Thus, they are sort of related to developmental solutions to game learning, which is what you have proposed. Perhaps some analogy from developmental robotics could be used to improve the current state-of-the-art in game play (or at least hit the problem from a different angle, which is also beneficial). If you could find something like that, it would be an even better way to motivate your intro and to frame your prior work. Looking at the next section, the twist of your approach could be its first-person style of learning. Obviously, AI has been developed for the computer players in first-person shooter games. But maybe nothing like it has been done yet in the arena of learning how to play a game. So, if little related work has addressed this type of learning, make this theme the central component of your work. Frame everything around it. Algorithms ------------- I liked the description of the SVM vs. Neural Network. Results --------- Can you say anything about the snake's motivation to become bigger? For example, over time in continuous learning, does the snake fail less and get bigger faster? Some of the result graphs are unclear. Does the presence of several light squares in the game board mean that the snake is longer than 1 unit? You could make it more clear by showing a sequence of game board states using smaller pictures. Conclusion -------------- The conclusion is accurate. Future Work --------------- You placed a lot of focus on robots, but the research seems to be pretty far off from where you're at now. Do you have other plans for the clean user interface you made besides what you used it for in this class? It seems like you spent a lot of time on it. * Overall, is the project report clear, concise, and well-organized? Yes. * How does the project idea and methodology fit within the framework of Developmental Robotics? The snake learns as its body changes size. This mirrors what happens in development. * Describe what you like BEST about the project? I liked that the snake exhibited different behaviors based on its motivation. * Describe what you like LEAST about the project? The ideas may not be publishable. The work was framed too broadly to tell. * Do the methods, results and contributions of the final project correspond to what was presented in the initial project proposal? Yes. * Are there any major details left out with regards to the methods, algorithms, or experimental design described in the report? No. * Do the experimental results reported in the paper demonstrate success? Somewhat. The snake program demonstrated intrinsic motivation in the behaviors that it performed. The snake program learned using SVM. * Do you have any suggestions for improvement and future work? The scope of the paper was too broad. Try to find a better way to motivate it. * How close is the final project report to being publishable as a conference or journal paper (consider the research papers that were part of the course reading)? What would it take to get there? You would have to find a good way to motivate this paper before the ideas could get published. That would require a new set of experiments as well. You already have the simulator, though, which sounds like it took a lot of time to make.