Project review: 1. Project number: 13 “Dropping Disks on Pegs” by Adam Campbel 2. Should this project be considered for the Best Project Award? No. While the project is well motivated, the methodology and results fall short. 3. Should this project be considered for the top 3 project awards? No. 4. On a scale of 1 to 10, how would you rate the overall organization/clarity of the project? 5.0. It is somewhat well organized, but the experimental procedure and methodology is not all that clear. 5. On a scale of 1 to 10, how would you rate the overall project idea? 7.0. The idea is very much developmentally sound. 6. On a scale of 1 to 10, how would you rate the overall research contribution of the project idea, methodology and/or results? 6.0. The biggest contribution here, in my opinion, is the experimental methodology of how the robot interacts with the pegs. However, the contributions in terms of the results are weak. * How does the project idea and methodology fit within the framework of Developmental Robotics? It fits within the framework very well because the robot in this case learns about the objects in a way much like a child does. It is not clear how the results fit within that framework, however. * Do the methods, results and contributions of the final project correspond to what was presented in the initial project proposal? Almost, although the proposal was much more ambitious. * Are there any major details left out with regards to the methods, algorithms, or experimental design described in the report? There are many details left out. Specifically, the experimental procedure done by the robot needs to be described in much more detail. * Do you have any suggestions for improvement and future work? Several: 1. The fact that the robot was successful in so few trials needs to be addressed. The best way to do that would for the robot to learn a knn predictive model in real time so that it does’t try a drop position for which it already knows that it won’t work. 2. The robot needs to interact with more pegs and disks. This requires #1, but if that is achieved, it can start learning which pairs of objects can fit together and what characteristics of them make that fitting possible. Proposal Review: 1. Proposal number: 13 “Dropping Disks on Pegs” by Adam Campbel 2. Should this proposal be considered for the Best Proposal prize? no 3. On a scale of 1 to 10, how would you rate the overall organization/clarity of the proposal? 4 4. On a scale of 1 to 10, how would you rate the overall project idea? 6 Then write 2-3 pages of helpful feedback to the proposal's author(s). The following questions should help you organize your feedback: * Overall, is the proposal clear, concise, and well-organized? The proposal is relatively clear with regards to motivation and method. The writing style, however, seems a bit too personal. Also, some of the behavior exploration routines proposed for the robot need to be explained more precisely and in much more detail. * Does the proposal meet the posted proposal guidelines? The related work section doesn’t seem to be all that related. The robotics research cited in this proposal deals with the tower of hanoi game, but not with other developmental approaches in which robots learn to manipulate objects. While there may not be other papers specifically addressing the task in this proposal, there are still many related ones. For example, Shane Griffith’s publication on categorizing containers comes to mind. Details regarding the algorithms and code libraries to be used are also missing. The proposal should provide some background on how the sensory feedback data recorded from the exploration trials will be analyzed. * How does the project idea fit within the framework of Developmental Robotics? This project addresses a manipulation problem using several principles of developmental robotics. First, the robot’s representation will be grounded in its own experience with the world. Second, the project proposal also indicates that the robot will perform behaviors that verify the outcome of its previous action. The task is itself is a manipulation task often performed by infants. * Describe what you like BEST and LEAST about the project idea. My favorite part of this proposal is the overall approach to solving the task - rather then pre-programming the robot to put the disk at a specific location, the robot will explore the objects and learn for itself what works and what doesn’t. I also like the aspect of detecting success and failure by performing subsequent behaviors on the peg. The least that I like about this proposal is that it could have been written much better. * Do you have any concerns about the project? The only concern I have is whether one person would be able to write all the code to process the robot’s sensory data. * Does it seem doable in the remaining time? Yes. * Does it seem too difficult? It doesn’t seem easy. * Are there any major details left out? Yes, many. How are the individual sensory motor data from each trial going to be represented? For example, many of the papers presented in class had a specific notation - such details are missing from this proposal. * Does the idea rely upon technologies that are not currently available? No. * Do you have any suggestions for improvement? Do you have any suggestions for related work that should be cited? One suggestion would be to expand the experimental setup to include several different pegs and/or disks, such that not all possible pairs can fit together (e.g., some pegs might be too thick for some disks, etc.). If the robot explores several combination, it would be able to detect which ones work and which ones don’t, based on the different types of outcomes with each pair. Furthermore, if the robot can perceive the thickness of the peg roughly from its visual sense, than it could perhaps learn a model that can estimate whether a given peg and disk could work together. My suggestion for the project proposal author is to read the papers by Griffith et al dealing with containers - in those paper, the robot explored the objects in a similar manner, by dropping a puck randomly over containers and non-containers. Other work from Stoytchev lab that is also relevant includes the study by Ritika Sahai on writing, since in that experiment, the robot was also studying how two different objects relate to each other. For example, if the peg is too thick, the robot should be able to find out that the disk never goes on it and perhaps predict whether a new peg will be too thick based on observation in the visual field. A final suggestion for the author is to spend more time on the writing. The proposal could have been much better if it had been proof read and edited a few more times. The layout and the positions of the figures is also pretty bad and can hopefully be fixed before the final write up is submitted.