Project Review: 1. Project number: 17 “Learning Musical Instruments Through Spectral and Temporal Analysis” 2. Should this project be considered for the Best Project Award? No. The project presents an exercise in sound classification, but little else. It is not entirely clear how it is developmental in nature. 3. Should this project be considered for the top 3 project awards? No. 4. On a scale of 1 to 10, how would you rate the overall organization/clarity of the project? Relatively clear and organized: 8.0 5. On a scale of 1 to 10, how would you rate the overall project idea? The project idea gets a 6.0. The problem of learning different sounds is a relevant problem to developmental robotics. 6. On a scale of 1 to 10, how would you rate the overall research contribution of the project idea, methodology and/or results? In terms of research contribution, this project gets a 4.0. The project amounts only to an exercise in sound classification using engineered features. Developmental aspects of the tasks (e.g., how does the agent’s performance vary as more sound sources are added, or more experience is added for training) are not covered. * Overall, is the project report clear, concise, and well-organized? It is relatively clear and organized. The writing style, however, needs to be improved and be more formal. * How does the project idea and methodology fit within the framework of Developmental Robotics? While the problem is relevant to developmental robotics, the methodology chosen by the author does not seem all that developmentally inspired. For example, the initial set of features is a heavily engineered - it would be more appropriate to use wider variety of features and let the agent discover which work and which don’t. In addition, developmental aspects of sound learning are not discovered - for example, how can a new sound source be incorporated incrementally into the framework? * Do the methods, results and contributions of the final project correspond to what was presented in the initial project proposal? Yes, the final project report was about what I expected. * Are there any major details left out with regards to the methods, algorithms, or experimental design described in the report? There are many details missing with regards to the classification experiment. It is not clear what the performance of the two algorithms is, and also there needs to be better notation. * Do you have any suggestions for improvement and future work? Several: 1. Implement a procedure that would allow the agent to quickly add new types of sounds into its knowledge base by relating their features to existing ones. 2. Allow the agent to learn about multiple sounds played at the same time - the ICA algorithm might help in this task. 3. Study the problem with a developmental approach - can the agent map out the space of sounds, learn how similar different sound sources are, etc. Proposal Review: 1. Proposal number: 17 “Learning Musical Instruments Through Spectral and Temporal Analysis” 2. Should this proposal be considered for the Best Proposal prize? no 3. On a scale of 1 to 10, how would you rate the overall organization/clarity of the proposal? 5 4. On a scale of 1 to 10, how would you rate the overall project idea? 7 Then write 2-3 pages of helpful feedback to the proposal's author(s). The following questions should help you organize your feedback: * Overall, is the proposal clear, concise, and well-organized? The organization is ok, but overall, the writing in the proposal would have been much better if it had been proofed and edited a bit more. * Does the proposal meet the posted proposal guidelines? Most of them are met, but some are missing. For exampple, there is no related work cited. The proposed method could have been explained in more detail as well. * How does the project idea fit within the framework of Developmental Robotics? It is not entirely clear from the proposal. While there are many approaches to sound classification, not all of them are developmental. In addition to simply classifying the sounds by instrument, a developmental approach would also need to be able to autonomously categorize the sounds and/or categorize the instruments. Also, it is not clear whether the sounds will be segmented manually before hand, or if the module will automatically segment the incoming audio stream and classify each sound segment. * Describe what you like BEST and LEAST about the project idea. I like the idea of a system which can learn about sounds through experience, especially the idea of learning sound categories automatically, since such a method would have applications in a wide variety of settings. What I don’t like about this proposal is that there is little detail regarding the developmental aspects of it. * Do you have any concerns about the project? Does it seem doable in the remaining time? It seems doable. * Does it seem too difficult? It really depends on how far the author is willing to go. Simply extracting sound features and using a MATLAB classifier is easy. * Are there any major details left out? Quite a few. There is little distinction in the proposal between classification and categorization, for example. Methods for feature extraction are also not very well defined. * Does the idea rely upon technologies that are not currently available? No. * Do you have any suggestions for improvement? Do you have any suggestions for related work that should be cited? I have several suggestions since I think the project needs to go somewhat beyond simply classifying the sounds according to instruments using a trained classifier. First, the author could also include the task of unsupervised sound categorization, sort of like Shane’s work on containers in which the robot learned a hierarchical taxonomy of sounds. Such a learned hierarchy can be evaluated on whether it matches some known information about the sounds, such as their instruments. Second, it would also be nice to explore intelligent methods of feature extraction - there are many types of features in a sound, but if the relevant features are identified by the programmer rather than the learning module, then the project wouldn’t be in the spirit of developmental robotics. Another possible route could involve the use of active learning algorithms which ask for labelled examples in an intelligent manner, to maximize learning progress.