1. Project number and Title

17 - Learning Musical Instruments through Spectral and Temporal Analysis

2. Should this project be considered for the Best Project award? (yes/no)

No

3. Should this project be considered for the top 3 project awards? (yes/no)

No

4. On a scale of 1 to 10, how would you rate the overall organization/clarity of the project report? (1-10)

6

5. On a scale of 1 to 10, how would you rate the overall project idea? (1-10)

8

6. On a scale of 1 to 10, how would you rate the overall research contribution of the project idea, methodology and/or results?  (1-10)

6


* Overall, is the project report clear, concise, and well-organized?

No, this proposal is lacking in all of these areas.  First, it is not clear how the results of this report were determined.  Instead this project makes broad assertions about the usefulness of the timbre properties without validating them.  Such a shortcoming will be detailed later in this analysis.  Second, the report is not concise, there are approximately ten pages of figures in a section where a three or four figures would suffice.  Third, the paper lacks organization.  There are approximately thirteen references cited but only two are actually found in the paper.  It is not clear why the remaining eleven reference were of value for this project.

* How does the project idea and methodology fit within the framework of Developmental Robotics?

The original project idea does fit within the framework of Developmental Robotics because it seeks to recognize properties that are inherently found in the sound of a musical instrument.  However, the methodology performed does not reflect this original project idea.  Instead, the results section is drawn from the observations of the paper's author and not learned or developed by a machine.  Such a deficiency in the work greatly detracts form the original idea.

* Describe what you like BEST about the project?

The original idea and motivation behind this project are sound in principle.  I believe there is merit in using components of the timbre for classification of musical instruments.  

* Describe what you like LEAST about the project?

The lack of results and minimal effort put forth in the subsequent report struck me as the worst parts of this project.

* Do the methods, results and contributions of the final project correspond to what was presented in the initial project proposal?

While the original intent was not lost between the proposal and the project implementation, there were several major components that were not completed.  In the proposal, it was stated that this project would attempt to use a collected dataset for several musical instruments.  This component of the proposal was not fulfilled but was excusable because of the explanation of the author.  Still, the lack of adequate results are the project's greatest shortcoming.  

Computing the spectral features from a digital library of instruments appeared quite trivial.  It should have played a minor part in this research with a subsequent focus on classification.  Instead, computing the spectral features was the only portion of this project that was actually completed.  Furthermore, it was not described how these features could be translated into appropriate features that could then be classified using machine learning algorithms.

* Are there any major details left out with regards to the methods, algorithms, or experimental design described in the report?

Related work for this project is all but completely absent in this report.  With the exception of a minor citation, the author fails to describe this project's context in the field of developmental robotics or music.  Other missing sections are due to the lack of tangible results and methodology completed for this project.

* Do the experimental results reported in the paper demonstrate success?

No, the success criterion as outlined in the proposal has not been met.  Several steps will need to be taken before the success criterion can be evaluated.  The author makes the claim that loudness and pitch did not unique identify an instrument but that attack time did.  Still, it was unclear how the author arrived at these conclusions as they were not outlined in the final report.

* Do you have any suggestions for improvement and future work?

The first suggestion is to actually complete this project as it was described in the proposal.  Only after this step is completed can future work be evaluated.  As for the report itself, the author needs to re-evaluate his presentation style.  The lessons learned section is completely unnecessary and appeared to serve as a section for making personal excuses.  Additionally, there were a few instances of grammatical errors, such as using the wrong "to" or using "setup" instead of "set up".  In the future, the author needs to proof his paper with a more discriminating eye.

* How close is the final project report to being publishable as a conference or journal paper (consider the research papers that were part of the course reading)? What would it take to get there?

This final report is not close to publication.  It has severe deficiencies in content and results.  In order to make it ready for publication, this project would need to be completed and the report would need a complete revision.  Furthermore, it would need to outline the missing sections described above and provide a better description of existing sections.  Additionally, the lessons learned section would need to be cut.