1. Project number and Title: project 2 (behavior grounded object identification, grouping, and ordering, by a humanoid robot)

2. Should this project be considered for the Best Project award? yes

3. Should this project be considered for the top 3 project awards? yes

4. On a scale of 1 to 10, how would you rate the overall organization/clarity of the project report? 10

5. On a scale of 1 to 10, how would you rate the overall project idea? 9

6. On a scale of 1 to 10, how would you rate the overall research contribution of the project idea, methodology and/or results?  9

* Overall, is the project report clear, concise, and well-organized?
As a whole, this paper is very well written.  It supplies a huge amount of background information, which establishes the need for this experiment.  The paper is written in a well-organized manner that leads the reader through a gradual understanding of the project as it increases in complexity towards the end.

* How does the project idea and methodology fit within the framework of Developmental Robotics?
The place in robotics for this project is really quite clear.  The idea is to explore objects based on interactive behaviors.  This is very important because the underlying key of developmental robotics is that intelligence must develop as a robot moves through a stage of development.  The developmental stage is where the ‘being’ must interact with the environment in order to understand it.  If we can find a good general way to replicate these behaviors, and respond to interactions, we will be a step forward in developmental robotics.

* Describe what you like BEST about the project?
I am very impressed with the content of this paper.  Your team definitely dug much deeper into the project than I had imagined when reading your proposal.  I like that you explore so many viewpoints of how the robot could interact and generalize the objects.  Any one of the three tests (object grouping, recognition, and ordering) would make a good project, so I am impressed that you were able to explore each as extensively as is indicated in your paper.  
I would have to say my favorite part of this project is the size of your experiment combined with some of the results that you obtained.  Object recognition, ordering, and grouping are all fundamental skills that an intelligent robot should be expected to know upon performing tasks.  Even many of the simplest everyday tasks would employ such skills.  The big difference between where we are and the goal is that many current projects and papers use maybe a couple of different feedback modalities to use in data extraction, whereas the future could a robot that has 100+ sensory channels and many thousands of objects to differentiate.  Therefore, I really like your project in that it is an effort to expand the limit of objects that a robot can know.  Your conclusions also support the idea that as you add more sensory data, recognition rates increase.  This supports the generalization of recognition based on robot-object interaction.
*Describe what you like LEAST about the project?
Though your project supports the generalization of object recognition, there is still a long way to go.  If and when someone tries to attempt the experiment with say 20 sensory systems, will they still be able to use the same methods as you used?  I imagine with more inputs your methods of object grouping would be especially hard.  Then, what happens when you increase the number of sensory systems even more, say to 100.  Then just finding a way to implement this technique would be a painstaking and redundant task.  Then, how could these methods be used for real world implementation of an actual robotic system?  There would be so much data, and uncertainty between which result is the most correct that it would be hard to effectively implement these methods in a completely general sense.

* Do the methods, results and contributions of the final project correspond to what was presented in the initial project proposal?
The proposal was well written as well.  It was clear that when the proposal written, the team had a good idea of what they were going to explore.  Now that the final paper is written, it is clear that they knew what their experiment explored, knew what they wanted to find out, and for the most part, have continued in the same direction as given in the project proposal.

* Are there any major details left out with regards to the methods, algorithms, or experimental design described in the report?

Actually, I would say quite the opposite.  This paper has an excellent explanation of the techniques and methods used to evaluate data.  Not only are the methods explained, but they are shown mathematically and in variable space where needed.  This paper has a sufficient amount of explanation about the overall experimental setup.

* Do the experimental results reported in the paper demonstrate success?
I would definitely say that the results demonstrate success.  There are mixed levels of success in different aspects of the experiment, but even taking the lowest success rates are far better than if taken completely random.  The concluding idea that accuracy increases with the number of sensorimotor contexts is a rather elegant result.  I was glad to see this result, and it ties many aspects of the paper together.

* Do you have any suggestions for improvement and future work?
Now that you have a good solid format to which you can perform your experiments, a good direction of improvement would be to make the object groups vary more in their characteristics.  I would presume that it is pretty unlikely for a robot to interact with a whole bunch of green cones varying in size, for instance, or many stuffed animals of precisely the same size.  It would be nice to see how the results of this experiment would differ if the intended groups of objects had larger variation.  Then of course, you would end up with “exceptions” where special case items may group themselves wrong, but maybe with all of your feedback, this problem could be eliminated.

* How close is the final project report to being publishable as a conference or journal paper (consider the research papers that were part of the course reading)? What would it take to get there?

This paper is very close to being publication quality.  My only suggestion would be to proofread a couple times. – I did notice a couple spelling/grammatical things that could be improved on.  Within the scope of this course, I don’t see it as being important, but a publication is for the whole world to see.  You want it to look good.  I hope you guys publish this, they are very good results, and it’s clear that you have put the time into this project.