ReviewerReactions

Plain Text icon reviews_ci_2015_00516w.txt — Plain Text, 8 kB (8442 bytes)

File contents

1.1: I found this paper extremely difficult to read and review.  

1.2: hat I think is described is the following:
The authors suggest the following protocol:  each compound in a library is split into fragments using a RECAP procedure.  The docking of the compounds using AutoDock to the target protein provides a set of poses and corresponding energy of interaction for each compound.  The pattern of interactions with the target made by all the fragments from all the poses of all the compounds is analyzed to identify particular receptor-ligand interaction features.  This is then used (via an array I don’t understand) to characterize the binding site, identifying key interaction features that would be favorable for including in a compound. 
At least I think this is what is proposed.  This is not described clearly in the manuscript, not in the abstract, nor in the methods or the results.  

1.3: This much I think I understand.  The example presented in figure 3.7 was helpful in shedding some light on the ideas.  What I found difficult to penetrate was the detail of how the RLIFs were calculated and a clear description of the “demonstration of performance superior to all known results” promised in the current abstract.  This may well be my failing but I found it difficult to follow the language used and the often rather abstract description of the methods and results.  

1.4: It maybe that other reviewers find this paper easier to understand.  The central idea is I think a good one – using large scale docking and analysis to identify what is often observed experimentally, that similar chemical features can be found from different ligands making similar interactions with a feature of the binding site.   However, I feel this paper needs a major rewrite to present the ideas in a format that is accessible to others.

2.1 I would like to commend the authors on the description and presentation of what could be an interesting new scoring method.  However, I am recommending that the manuscript titled "Fragment-based analysis of ligand dockings improves classification" only be accepted for publication after major revisions.  There are three reasons for my recommendation:

2.2: 1) The authors did not provide enough detail in the methods description such that the method could be independently reproduced.  Had code been submitted as Supplementary Files then reading the code might have made up for the missing critical details in the methods description.   I am not talking about the open source code used (Openbabel, fastcluster, etc) but the scripts so that I could see the work flow and what data was operated on.  I did appreciate the authors including input examples for some of the code run.  

2.3: 2) A related item to #1 is that the authors were not clear as to whether or not RLIF derived models were generated from all the 102 targets in DUD-E, the 11 targets presented or a new model was generated for each protein target.  This information is critical to assessing the applicability of the method and whether or not the results reported are goodness of fit which is important for understanding the quality of the model(s) but not the utility of the model(s).

2.4: 3) There are several sections in this paper that are either extremely weak or so poorly explained that I was unable to see how the examples validated the work or why they were included in the paper.  In particular section 3.5.  I apologize but I did not understand how this section provided novel insights and Figure 3.6 was rather frightening in that most of the fragment interactions provided little (or no) information yet were extremely highly weighted.  The Related work section in the Discussion could be included in the Introduction but has no place in the Discussion.

2.5: Page 2 line 54.  The sentence should read "Further, rather than looking" instead of "rather looking"

2.6: Page 4 lines 55 and 57.  O'Donnell was not one of the RECAP authors.  This was an implementation but not an implementation by one of the authors.

2.7: Page 8 line 20.  There is a formatting issue as I could not read the fastcluster.linkage command - it ran off the page.

2.8: Page 9 line 8.  "broadly robust good performance"  The inclusion of good is not appropriate.  I don't think the authors are trying to say that the performance was morally correct.

2.9: Page 10 Figure 3.1.  The caption needs to be moved to the upper right corner.  It is currently blocking the histogram of the Actives.

2.10: Page 11 lines 4 and 6.  The authors should include the data or examples of what they mean by reduction in same versus across all experiments in Supplementary Files.

2.11: Page 12 lines 3-13.  Please provide an explanation for the reasons why 11 (out of 102) were chosen and if there were specific reasons for each of the 11.  A justification of why they chose to use a different protein structure from the one used for DUD-E would be helpful.  It is not unreasonable to assume that the improvements in performance for HIVPR and HIVRT reported are largely or even solely the result of the change in protein structure.

2.12: Page 13 Figure 3.2. Should read Table 2 not 2.  Also the first 2 characters are not always the experiment label specifically not for HIVPR and HIVRT.

2.13: Page 13 lines 42 and 44.  The abbreviation AUROC comes before the explanation though is in the next sentence so this can probably be ignored.  I don't understand why the authors are making up a new abbreviation for AUC.  While AUC is not a specific as AUROC it has been widely used in the literature particularly statistics literature and is understood to mean Area Under Curve where the curve is a Receiver Operating Characteristic.  The authors need to check how they are calculating these values (see Table 3) the HIVRT AUC is not greater than the AMPC AUC based on the curves plotted in Figure 3.2.  I suspect the AUC for HIVRT should be around 0.6 or less. Because I'm guesstimating maybe as high as 0.7 but not 0.89 - 70% of the curve is worse than random. 

2.14: Page 14 line 45.  The numbers are identical Lesnik15 only reported 2 significant digits.

2.15: Page 15 lines 30-34.  This statement would be true is the authors were using percent correct predictions as the metric but they are using AUC.  If all classifications made were Decoy the AUC would be 0 not 0.5 or greater.

2.16: Page 15 lines 53-57 and 16 lines 40-45.  This is the Shannon Entropy equation.  Please either correctly identify it or provide a reference or both.

2.17: Page 17 lines 23-28.  What is the DUD-E training set? I can't find a description in the methods.  If the training set was the complete DUD-E data set and the authors are then using the model to predict for 11 targets that were part of the training set the AUC values shown in Figure 3.2 and Table 3 are only quality of the training fit (R in QSAR terms) and not true prediction capability of the model on untrained data (Q in QSAR terms).    I haven't read the Lesnik paper but comparing their results (if testing on training data) with the Mysinger results is not an appropriate comparison because the DOCK scoring function was not trained on the DUD-E data.

2.18: Page 21 line 52.  The authors need to let the reader know that FAAH stands for FightAIDS@Home found on page 24 line 47.

2.19: Page 24 line 53 and 55.  The Lipinski "Rule of five" has nothing to do with binding affinity or inhibition.  It is about increasing the probability of having an orally bioavailable compound.  It is more related to approval than the other two criteria mentioned.

2.20: Page 28-31 Table 4.  The authors should check that the RECAP fragmentation was implemented correctly.  There are a number of example of fragments where a urea was not fragmented.  I don’t remember if there was an exception for cyclic urea but I would guess not because formation is a good way to cyclize.  It has been too long since I read the RECAP paper so I don’t remember.

2.21: Last I would like to apologize for the tone this review was written in.   It is too demanding.  Please accept my apology.  If I had more time I would have rewritten it.

3.1: The authors present a nice approach to classify ligand docking modes. The manuscript is well written and should be published after minor revisions:

3.2: 1) Page 10 Fig 3.1 y-axes "total"
3.3: 2) Page 10 line 50 "more than one ligand"
3.4: 3) Page 21 line 3 "in terms of" what?