1. Project #12: Autonomous Learning in a Simulate Environment

2. Should this project be considered for the Best Project award? (yes/no)

  Yes

3. Should this project be considered for the top 3 project awards? (yes/no)

  Yes

4. On a scale of 1 to 10, how would you rate the overall organization/clarity
of the project report? (1-10)

  9

5. On a scale of 1 to 10, how would you rate the overall project idea? (1-10)

  9

6. On a scale of 1 to 10, how would you rate the overall research
contribution of the project idea, methodology and/or results?  (1-10)

  9

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

****** Overall, is the project report clear, concise, and well-organized? ******

  Yes. The report is organized into clearly marked sections, and the structure of the sentences
and paragraphs makes it easy to read and understand.

****** How does the project idea and methodology fit within the framework of Developmental Robotics? ******

  The overarching theme in the project is intrinsic motivation. Robots need to explore their 
environment in order to develop properly, but there needs to be some sort of force driving the 
robot from within, because it's not practical to have humans telling the robot what to explore 
during the entire developmental period. The idea of using intrinsic motivation to have a robot 
learn while playing a video game is just one application of this approach, and it can be expanded 
for other purposes in which a robot must learn about its environment on its own.
  The project used Support Vector Machines and Online Support Vector Regression machines to allow
the robot to learn. The fact that learning could be accomplished while the program was running
allows for much more robust systems, because robots can learn on the fly.


****** Describe what you like BEST about the project? ******

    I like the idea of using intrinsic motivation to teach a robot to learn. It's certainly true
that robots won't be able to be truly autonomous unless they can develop and live on their own, and
it's not possible for a robot to develop properly on its own without intrinsic motivation. If human
infants didn't have this internal driving force to explore the world, then surely we wouldn't be 
the intelligent beings that we are today.


****** Describe what you like LEAST about the project? ******

    I didn't find anything that I hated about the project, but I wasn't sure what I thought about
removing the walls. I know that it made it a safer environment for the snake, but I think it would
have been more interesting with the walls there. A bigger board would have reduced the chance of
a wall collision, so that could be something to think about.


****** Do the methods, results and contributions of the final project
correspond to what was presented in the initial project proposal? ******

   For the most part, the final project matches the intial proposal very well. SVMs were used
instead of Artificial Neural Networks, and the user interface was redone sightly from the proposal,
but everything else was virtually the same.


****** Are there any major details left out with regards to the methods,
algorithms, or experimental design described in the report? ******

  No. The project is explained in sufficient detail. 


****** Do the experimental results reported in the paper demonstrate success? ******

  Yes. The prediction accuracy of the SVM was very high, mostly in the upper 90 percent range. 
Compared to blindly guessing, the error rate with this approach is less than half. Because of the
significant difference in error rates, it's clear that the SVM made a difference in learning accuracy.


****** Do you have any suggestions for improvement and future work? ******

  I think that it would be very interesting to see a robot play a game with a joystick, while
at the same time learning how to play it. Humans are unique in that they can sit down at something,
play around with it, and learn how to master it, all with no prior knowledge of it. If a robot could
do the same thing with an actual body, as opposed to a software program running in its "mind", it 
would definitely be an accomplishment.
  Something I was wondering about is how long the snake got. It was stated that the snake spent less
than 2% of the time at a body size of 5 or higher, but did the snake ever get to larger sizes? I imagine
that a 10-size snake would be pretty difficult to maneuver effectively.


****** How close is the final project report to being publishable as a
conference or journal paper (consider the research papers that were
part of the course reading)? What would it take to get there? ******

  I think that the report was quite professional in appearance and tone. There were no glaring errors
in the report, and it was very clear and well-written. I think that, given a few revisions and some 
careful proofreading, the report could be publishable as a conference or journal paper.