CT Grading System

Purpose 

The purpose of the CT-Grade scientific rating scale is to provide an unbiased evaluation of each CT article based on current published peer-reviewed research evidence in order to accurately educate CT users about conventional and integrative cancer therapeutics and related topics (See Editorial Process). A higher CT-Grade (Ex: 5 or 4) coincides with a higher level of research-based evidence from peer-reviewed journal articles.  The rating given by CT is determined by a stringent, peer-review process (See Editorial Process, Peer Review). 

CT understands that there are numerous methodologies and testimonials that are unpublished, but have been shown to be beneficial to cancer patients. The CT Grade system is implemented to provide a scientifically credible standard for reporting the peer-reviewed evidence for key topics in oncology and to educate the user about options in an unbiased fashion. In the event that CT reports on an unpublished methodology or testimonial, we are committed to presenting as much clinical detail and peer-reviewed literature reports as possible. This transparency will provide a clear picture to users about the evidence and also provide insights regarding further research efforts to understand the basis of benefit for the treatment/practice. It is our view that conventional and integrative oncology are not two separate, opposing factions, but one cohesive network of knowledge that must communicate effectively by adhering to the same standards of transparency, scientific methodology, data collection and rigorous peer-review in order to make strides in the eradication of cancer. We are committed to providing an unbiased source of information regarding the best options for care that are offered from conventional and integrative oncology. 

In summary, the CT-Grade Scientific Rating Scale does the following:  

• Systematically and critically reviews literature for evidence-based approaches to topics in oncology with ongoing monitoring and review of new literature
• Utilizes both external and internal peer-review of CT content
• Applies rigorous standards to report the most relevant and valid data
  • Employs unbiased reporting on the quality of reported data and gives more weight to higher quality data and less weight to lesser quality data 
• Reports on all available, applicable clinical treatments with a focus on clinically relevant data 

The CT-Grade Scientific Rating Scale is committed to the following: 

• In the event that a practice is unpublished or an outcome is from a patient testimonial perspective, the entire process will be reported as such and therefore not graded as “evidence” but reported in as much clinical detail as possible. The article will report on the findings of evidence-based research and the testimonial will be used to propel further scientific inquiry and as insight for users to promote discussions with their care providers. 
• Scientific data and ratings will not be gleaned from unpublished manufacturer-sponsored promotional material or other sources (such as promotional, non-scientific based websites).
• The articles will not employ bias for or against conventional or integrative oncology. 
• The ratings will not be adjusted due to advertising or sponsorship and any of the aforesaid partnerships will be fully disclosed. 

Specific Standards utilized during the CT-Grade Review Process 

The CT editorial team drafted the CT Grade Review Process utilizing the combined strengths of all editorial team members and referencing current standard rating scales utilized by well-established clinical and academic programs (See References). CT-Grade was then thoroughly vetted by professionals from clinical, research, and industry with scientific expertise in conventional and alternative cancer therapeutics (See Advisory Committee and Scientific Panel members). The overall goal of the CT-grade scientific rating system is to provide a thorough evaluation of all of the evidence (scientific and anecdotal) published on the CT website. The review process timeline varies in length depending on the body of research available on the subject.

Topic Selection 

The Advisory Committee is consulted on a quarterly basis to assist in topic selection for CT articles. Based upon these recommendations, a topic expert is invited (upon consultation from the CT Scientific Panel). Then, the CT Editorial Team works with each topic expert on an article based upon the author guidelines (insert link here) and the CT rating system. 

Topic Assignment to CT-Step Area 

The topic is then further assigned to the appropriate CT-Step Area (See 7 steps outline), based on review and recommendations from the Advisory Committee. 

Editorial Process

Information Acquisition  

The process begins with a thorough search of peer-reviewed journal publications for applicable information. The studies are then critically evaluated by both the CT Editorial Team and the Scientific Panel to determine the reliability and validity of the research based on rigorous principles based on scientific evidence. This process further informs identification and evaluation of new research evidence.  

CT sets itself apart from other similar publications by objectively and systematically finding the best information and reporting it in an unbiased and transparent manner. Additionally, CT strives to frequently update and re-evaluate content based on current, relevant research findings. The new data is evaluated, and integrated in light of previous findings. CT-Grade is also evaluated at this time and is adjusted accordingly, if applicable (See Adjustments). All information is stored in the CT database for future reference. This system gives users access to unbiased, valid, and current information regarding topics that are relevant to oncology treatment.  

Contributors

The CT team (Editorial, Advisory Committee, and Scientific Panel) is composed of individuals with expertise in conventional and integrative oncology as well as scientific journalism/writing. Experts in content area hold a Doctor of Medicine (MD), Doctor of Philosophy (PhD) or other applicable doctorates (PharmD, etc) with appropriate licensing, clinical experience, and/or expertise in evaluation of medical oncology research. Contributors hold active or previous faculty appointments/affiliations with recognized universities and/or academic/integrative medical centers. Contributors focus on validity of results and not personal bias, with the primary mission to objectively report the most relevant and valid conclusions from the research findings to the user. If conflicts of interest exist, full transparency is employed and the contributor is excluded from the CT-Grade for an article that particularly addresses specific topics that could invoke bias.  

Literature Criteria 

Identification of relevant data is achieved through systematic searches using specific key words relating to the topic utilizing resources such as PubMed, EBSCO, Google Scholar, and the NIH Clinical Trial database. In the event that a treatment modality is listed under several names (i.e. chemical, botanical), all search parameters are employed to further identify additional resources. The references cited by relevant articles and studies are also used to identify further resources that are not listed in the databases mentioned above. The CT network of experts, including the Advisory Committee and Scientific Panel, are consulted as sources for additional expertise regarding research in specific topic areas. Studies are selected that provide potentially relevant information regarding: 

• Clinical implementation (safety and effectiveness, lab test results) of a treatment 
• Pharmacology and/or mechanism of action 
• Contraindications
• Any other information that is relevant to health professionals 

Any literature that does not provide relevant, valid information is excluded and is assigned accordingly in the CT database. 

Significance 

Sources are screened for relevant data and validity is evaluated as outlined in the CT-Grading system. Significance is based on the ability of the study to address key issues that inform clinical practice, such as safety and effectiveness of the treatment modality. Furthermore, detailed and scientifically sound evidence regarding the mechanisms of action/pharmacology and all related topics must be included. 

Strength  

Significant research evidence is then assessed for strength of methodology/design and the ability of this design to produce valid results. At this point, both the level and quality of evidence are considered. 

Uniformity

New research data is compared to previously reported data to determine uniformity. Higher consistency yields higher confidence in the uniformity of the body of evidence, while inconsistent findings may be due to issues/differences regarding study design, patient/disease demographics, therapeutic formulation and application, etc. These inconsistencies translate to a lower level of reliability for the particular body of evidence. 

Information Workflow and Review

Following the outlined critical evaluation of relevant literature, CT works with a content expert recommended by the Advisory Committee (See Advisory Committee) to draft an article for publication on the CT website. This article is reviewed by members of the Scientific Panel (See Scientific Panel) using a multi-disciplinary and multi-pronged approach. The process is informed and standardized by the CT-Grade Scientific Rating Scale and a CT-Grade is assigned based on the consensus of the reviews from the Scientific Panel. The article is then published on the CT website, along with all peer-reviewed CT-Grade information and implications. This process is in place to ensure a practical and standardized means of evaluation that is consistent and provides practical, reliable, and relevant information to CT users. 

Eligibility Considerations for CT-Grade   

CT utilizes a peer-reviewed process and scientific rating scale that encompasses multiple critical aspects of research evidence on each selected topic in CT-7. The level, quality, significance, validity, efficacy, safety, and practical therapeutic implementation of the topic are key factors in the evaluation process. High levels of evidence with the above standards achieve high CT-Grade ratings. There are some cases where enough applicable evidence does not exist to make a clear rating, but the topic is promising. In these instances, CT clearly outlines the available information, but does not apply a CT-Grade. 

Author Guidelines 

Timeline for Publishing Rated Articles 

The peer-reviewed and rated article receives a final review by the CT Editorial team before being published on the CT website. The rating for all new articles on the website is provisional for 60 days, allowing for adjustments as needed. 

Grades and general criteria for each category 

General criteria for each grade category are listed below: 

5 – Excellent

• There is definitive evidence, strongly supported by numerous research studies, that the treatment/practice is beneficial. 

4 – Good

• There is some support by research evidence, supported by a number of research studies, that the treatment/practice is beneficial. 

3 – Low Quality

• The preliminary research regarding the treatment/practice is promising, but needs additional, thorough studies to address major flaws. 

2 – Very Low Quality 

• The research evidence that is available is very poor and therefore no scientific effect can be reported for this treatment/practice.

1 – Concerning practice

• The research evidence reports definitive evidence of harm due to the treatment/practice.

NR – No rating

• There are no published studies regarding this treatment/practice available to review according to the CT-Grade scientific rating scale. 

Specific standards for each grade 

Grade 5
Considerations:

• At least 2 rigorous (of sufficient sample size), randomized, controlled phase II/III clinical trials (RCT) have consistently shown definitive conclusions regarding the treatment modality and have been shown to be beneficial to patients in addition to the following:
  • Significant evidence supports the treatment modality (1 or more of the following):
    • In vitro and in vivo studies
    •  Non-randomized clinical trials
    • Nonquantitative systematic reviews
    • Lower quality RCT (phase I) 
    • Clinical cohort studies
    • Case-control studies
    • Historical control 
    • Epidemiological studies 
  • Sustained effect beyond a year of treatment (Length of study follow-up should be considered) when compared to control.
  • Meta-analyses may be considered as sufficient evidence on a case by case basis.
    
• All supporting evidence should be reported in published, peer-reviewed journal articles. Supporting evidence should inform and benefit current treatment practices with specific implementation instructions written by expert opinion are detailed through additional published books, treatment protocols, etc, where applicable. 
• Reported outcomes must be consistently applied across all subjects, in a valid, un-biased, and repeatable manner.  
• Clinical trials do not report any cases of significant, frequent, or severe harm caused by the treatment modality. 
• Additional studies are unlikely to change the grade and confidence in the efficacy of the treatment.

Grade 4
Considerations:  

• At least 1 rigorous (of sufficient sample size), randomized, controlled phase II or III clinical trial (RCT) has shown the treatment to be beneficial to patients with some limitations, OR:
  • Significant evidence supports the treatment modality (1 or more of the following):
    • In vitro and in vivo studies
    •  Non-randomized clinical trials
    • Nonquantitative systematic reviews
    • Lower quality RCT (phase I)
    • Clinical cohort studies
    • Case-control studies
    • Historical control 
    • Epidemiological studies 
  • Sustained effect beyond six months of treatment (Length of study follow-up should be considered) when compared to control. 
• All supporting evidence should be reported in published, peer-reviewed journal articles with equally definitive conclusions and a good overview of the literature. Any expert opinion given should be credible in light of the methods and outcomes cited in the studies. Supporting evidence should inform and benefit current treatment practices with specific implementation instructions detailed through additional published books, treatment protocols, etc, where applicable. 
• Reported outcomes must be consistently applied across all subjects, in a valid and repeatable manner.  These outcomes present a low to moderate risk of bias. 
• Clinical trials report show no significant, severe, or frequent cases of harm caused by the treatment modality.  
• Additional studies may change the grade and confidence in the efficacy of the treatment.  
• OR 
  • The level of evidence is the same as a rating of 5, however: 
    • Clinical trials report some cases of harm caused by the treatment modality in a subset of the patient population.

Grade 3
Considerations:

• A minimum of one study from the lists in 5/4 grade criteria with a control group in some form (untreated, placebo) has established the treatment’s efficacy over the control or: 
  •  the treatment is comparable to a practice with a 5 or 4 grade on the CT-grade scientific rating scale
  • the treatment is superior to a pre-existing comparison practice 
  • the treatment outcome exhibits significant limitations
• All supporting evidence should be reported in published, peer-reviewed journal articles. Supporting evidence should inform and benefit current treatment practices with specific implementation instructions detailed through additional published books, treatment protocols, etc, where applicable. 
• Reported outcomes must be consistently applied across all subjects, in a valid and repeatable manner.  
• If clinical trials exist, the outcomes do not report significant, frequent, or severe harm caused by the treatment modality. 
• Additional studies are very likely to change the grade and confidence in the efficacy of the treatment. The evidence presented has a moderate to high risk of bias.  
• Clinical trials report show no significant, severe, or frequent cases of harm caused by the treatment modality.  
• Additional studies may change the grade and confidence in the efficacy of the treatment.  
• OR 
  • The level of evidence is the same as a rating of 4 or 5, however: 
    • Clinical trials report some cases of severe/significant harm caused by the treatment modality in a subset of the patient population.

Grade  2
Considerations:

• Improved therapeutic/clinical outcomes have not been observed utilizing this practice in two or more randomized controlled trials (RCTs) when compared to current clinical practices.  
• In the case where multiple outcome studies have been reported, the weight of all supporting evidence (published in peer-reviewed journal articles, not systematic reviews, etc) does not support the benefit of implementing this practice. 
  • Example: If there have been four published RCTs and three showed the treatment did not have the desired effect, then the treatment/article/topic would be graded as a “2-Evidence Fails to Demonstrate Effect.”  
  • Studies exhibit severe limitations with indirect evidence
    • Inconsistent results
    • Low sample size
    • Inability to draw conclusions
  • Any benefit is very uncertain and based solely upon consensus or expert opinion that is not substantiated by published studies
• In the case that non-experimental studies exist, such as qualitative or meta-synthesis, systematic review, clinical practice guidelines, these should be reviewed based upon the low quality of research
• Specific implementation instructions are detailed through additional published books, treatment protocols, etc, where applicable. 

Grade 1
Considerations:

• In the event that multiple studies have been conducted, the cumulative evidence suggests that the practice has a negative effect upon a significant portion of the patient population and/or: 
  •  Case study data reports a risk of harm that:
    • May be caused by the treatment 
    • Is severe or frequent 
    • Outweighs the possible benefits of the treatment 
• Specific implementation instructions are detailed through additional published books, treatment protocols, etc, where applicable. 

NR (No Rating)
Considerations:

• There are no published, peer-reviewed journal articles in patients utilizing controls (untreated, placebo, etc) that establish the treatment’s benefit over control treatments or evidence proving any comparable or superior effect over current clinical practice. 

• The treatment is based on anecdotal evidence such as in vitro or animal research and testimonials.  The treatment is generally accepted in alternative/integrative cancer clinical practice as appropriate for patient treatment. 

• This treatment/protocol does not meet any other criteria for grading on the CT-Grade Scientific Rating Scale.

• There are no published reports of a risk of severe or frequent harm attributed to the treatment. 
• This treatment/protocol has specific implementation instructions detailed through published books, treatment protocols, etc. 

General description of review process  

Grade at a Glance

The CT Grade will be listed at the top of the article and a brief, “Grade at a Glance,” giving a brief summary of the reasoning behind the assigned grade and an overview of the article’s main points.

Advisory Committee 

The CT Advisory Committee is composed of  members with expertise representing a broad range of topics (relevant to the steps outlined in CT-7) in conventional as well as integrative oncology therapeutics and evidence-based research. These members serve on a 6-month rolling basis by invitation and are listed here (Insert link when we are ready for this. It should include their name, credentials, and area of expertise).  The main function of the Advisory Committee is outlined below: 

• Meet on a quarterly basis 
• Provide expertise with regard to topic selection for CT to review, rate, and publish on the website
• Recommend individuals with topic expertise to consult for CT articles
• Guide the review of the rating system as needed in terms of updated research on a topic on the CT website. 

Rating system review  

Adjustments to CT-Grades are continuously updated according to new, research-based evidence. The Advisory Committee directs the review of applicable peer-reviewed journal articles to provide the most current, credible grading changes, when applicable. All applicable CT articles related to a similar topic are updated according to these grading changes. 

Topic Selection  

The Advisory Committee is consulted on a quarterly basis to assist in topic selection for CT articles. Based upon these recommendations, a topic expert is invited to author an article (upon consultation from the CT Scientific Panel). Then, the CT Editorial Team works with each topic expert on an article based upon the author guidelines (insert link here) and the CT rating system. 

Scientific Panel  

The CT Scientific Panel is composed of members who are recognized as national leaders in conventional as well as integrative oncology therapeutics and possess expertise in evidence-based practice. Their collective expertise spans the topics outlined in the CT-7 steps. The panel assists in the following: 

• Identification of topic experts for CT articles 
• Peer review of articles and implementation of the CT-Grade Scientific Rating process 
• Offers guidance on the general scientific integrity of the content of CT as a whole

Peer Review and Assignment of Grade

The article is reviewed by members of the Scientific Panel (See Scientific Panel) using a multi-disciplinary and multi-pronged approach. The process is informed and standardized by the CT-Grade Scientific Rating Scale and a CT-Grade is assigned based on the consensus of the reviews from the Scientific Panel. The article is then published on the CT website, along with all peer-reviewed CT-Grade information and implications. This process is in place to ensure a practical and standardized means of evaluation that is consistent and provides practical, reliable, and relevant information to CT users.

Adjustments 

Re-review process

Adjustments to CT-Grades are continuously updated according to new, research-based evidence. CT conducts periodic reviews, with direction of the Advisory Committee, of applicable peer-reviewed journal articles to provide the most current, credible grading changes, when applicable. All applicable CT articles related to a similar topic are updated according to these grading changes. 

Re-grading  

When new research evidence is identified for the topic area, CT re-starts the rating process outlined above, under the direction of both the Advisory Committee and Scientific Panel, respectively. The re-grading updates are clearly noted in the “New Research at a Glance” section. 

The standards for the CT-Grade are objective, evidence-based ratings and represent many hours of peer-reviewed evaluation. As such, the ratings are updated based on the latest scientific data. Despite our best efforts in employing the objective, scientific approach utilized by CT-Grade, the CT Editorial team is aware that differing interpretations of scientific data may results in a difference of opinion regarding article rating. If there is scientific data that we have missed that suggests a change in rating, we invite you to contact the CT Editorial team. 

The following information will be helpful to consider prior to submission of this data for review:  

• The peer-reviewed process we employ at CT is based on unbiased evaluation of scientific evidence and ratings are applied accordingly. 
• We do not sell or endorse products or accept compensation in return for a product rating. 
• Any partnerships or advertising on CT are fully disclosed in the article and there is not oversight by vested parties in the rating of any related articles. 
• We are receptive to receiving new scientific information regarding products, ingredients, treatments, and patient outcomes. 
• Our CT Editorial Team, external Advisory Committee, and Scientific Panel will evaluate any new scientific evidence and adjust the rating of associated articles appropriately. 

New research at a glance 

Similar to “Grade at Glance,” any updates to the article should be noted in a summarized fashion with the date of update, highlighted additions/omissions, and any factors that impact a change in rating.

Disclosure

Inclusion of vested interest in company, product, clinic, treatment discussed in article

This statement should address any conflict of interest due to position, product development, etc on the part of the author. Details regarding this disclosure should be added at the beginning of the article, under the CT-Grade, as well as in the author profile. Ex: The author of this article <insert applicable title, etc. here> and has vested, financial interest in this <insert product, clinic, etc.>.