📖 ContentsChapter 11
Spine and Holistic Health · Chapter 11

Reading the Evidence

Every week a headline announces that some treatment "works" or "doesn't work," and every week patients walk into clinics carrying those headlines. A health professional who can read a study, spot its weak joints and put its numbers in perspective is far more useful than one who can only repeat what the latest article claimed. This chapter gives you the tools to judge research on spinal care and holistic therapies fairly: respecting what each tradition hopes to offer while asking, plainly, what the evidence actually shows.

By the end of this chapter you can

  • Distinguish experimental from observational study designs and place each on the hierarchy of evidence.
  • Explain how randomization, allocation concealment and blinding protect a trial from bias.
  • Define confounding, selection bias, information bias and regression to the mean, and recognize each in a study description.
  • Describe the placebo response and contextual effects, and explain why sham controls for manual and needling therapies are hard to design.
  • Calculate and interpret absolute risk reduction, relative risk reduction and the number needed to treat.
  • Interpret a confidence interval, a p value and an effect size, and separate statistical significance from the minimal clinically important difference.
  • Read a forest plot and explain how a systematic review and meta-analysis differ from a narrative review.
  • Explain how GRADE rates certainty of evidence and how clinical practice guidelines are built from it.
  • Summarize the current evidence and safety profile for spinal manipulation, acupuncture, massage, yoga, mindfulness, herbal products and homeopathy.

Section 1Study designs and levels of evidence

Why design matters more than the conclusion

A research paper is an argument. Its conclusion is only as strong as the design that produced it, so the first question a careful reader asks is not "what did they find?" but "how did they find it?" Two studies can reach opposite conclusions about the same treatment simply because one compared patients who chose the treatment with patients who did not, while the other assigned treatment by chance. Learning to name a study's design in the first minute of reading is the single most useful habit in evidence based practice.

Evidence based practice is usually described as the integration of three things: the best available research evidence, the clinician's expertise, and the patient's values and circumstances. Research evidence is one leg of that stool, not the whole stool. A strong trial does not tell you what an individual patient wants, and clinical experience cannot by itself tell you whether improvement came from the treatment or from time. The three legs work together.

The basic split: did the investigator assign the exposure?

Every study design can be sorted with one question: did the researchers decide who received the intervention? If they did, the study is experimental. If they simply watched what people did and what happened to them, the study is observational. The distinction matters because when people choose their own treatments, the choosers usually differ from the non-choosers in ways that also affect outcomes. People who seek out a yoga class may already exercise more, sleep better and smoke less. Assignment by the investigator, and especially assignment by chance, breaks that link.

Experimental designs

The randomized controlled trial (RCT) assigns participants to two or more groups by a chance process, such as a computer generated sequence. Because chance decides group membership, both known and unknown characteristics (age, severity, motivation, genetics) tend to balance between groups, especially in larger trials. Any difference in outcome can then be attributed to the intervention with much more confidence. Variations include:

A non-randomized controlled trial (sometimes called quasi-experimental) assigns treatment by some rule other than chance, such as alternating days or clinic location. It is better than no comparison but more vulnerable to imbalance between groups.

Observational designs

Observational studies are not second class. Many questions cannot ethically or practically be answered with a trial: you cannot randomize people to smoke, to suffer childhood adversity or to sustain a whiplash injury. Rare harms, such as a stroke after neck manipulation, are almost always studied observationally, because no trial could enroll enough people to see them.

Did investigators assign it? yesno by chanceby rule comparisonnone Experimental Observational Randomized trial Non-randomized trial Analytic Case report, case series Cohort exposure to outcome Case control outcome to exposure Cross-sectional one point in time Ask first: who decided who got the exposure?
Figure 11.1 A decision tree for naming a study design. The first fork, whether investigators assigned the exposure, separates experiments from observation; among observational studies, the direction of inquiry in time separates cohort from case control designs.

The hierarchy of evidence

Because some designs resist bias better than others, researchers arrange them in a hierarchy of evidence, often drawn as a pyramid. Near the top sit systematic reviews and meta-analyses of well conducted RCTs, then individual RCTs, then cohort studies, case control studies, case series and reports, and at the base, expert opinion, mechanistic reasoning and laboratory or animal research. The pyramid is a useful starting point, not a law. A large, careful cohort study can be more trustworthy than a tiny, poorly blinded RCT, and a meta-analysis of biased trials simply produces a precise estimate of a biased answer. Modern systems such as GRADE (discussed in Section 4) start trials higher and observational studies lower, then move each up or down based on how well it was actually done.

SR / MA RCTs Cohort studies Case control, cross-sectional Case reports, expert opinion, lab and animal research Less bias expected More bias expected SR = systematic review MA = meta-analysis Quality of the individual study can move it up or down a tier.
Figure 11.2 The traditional evidence pyramid. Designs near the top are less prone to bias on average, but the pyramid ranks designs, not individual studies: a sloppy trial can deserve less trust than a rigorous cohort.
DesignStarts withMain strengthMain weaknessTypical measure
Randomized controlled trialRandom assignment to treatmentBalances known and unknown confoundersCostly; may not generalize; poor for rare harmsRisk difference, mean difference
Prospective cohortExposed and unexposed peopleShows time order; measures incidenceConfounding; long and expensiveRelative risk
Retrospective cohortExisting records of a cohortFast and cheapDepends on record qualityRelative risk
Case controlPeople with and without the outcomeEfficient for rare outcomesRecall bias; choosing controls is hardOdds ratio
Cross-sectionalA population at one timeMeasures prevalence quicklyCannot show which came firstPrevalence, prevalence ratio
Case report or seriesOne or a few patientsGenerates hypotheses; signals harmsNo comparison groupDescription only

Efficacy, effectiveness and mechanism

Three different questions are often blurred. Efficacy asks whether a treatment produces a benefit under ideal conditions. Effectiveness asks whether it helps under ordinary clinical conditions with ordinary patients. Mechanism asks how it works. A therapy can be effective without anyone knowing its mechanism (many drugs were used long before their receptors were discovered), and a proposed mechanism can be plausible while the therapy still fails in trials. In spinal care, many explanations of how manipulation works have changed over the past century while the clinical question, does it help this patient with this problem, has remained the one that matters most to patients.

Study tip. To tell a cohort from a case control study, find the starting group. If the researchers started with people who had or did not have an exposure and waited to see outcomes, it is a cohort. If they started with people who already had or did not have the outcome and looked back at exposures, it is case control. "Cohort: cause first. Case control: cases first."

Reading the methods section first

Experienced readers often skip straight from the title to the methods. A quick checklist works for most clinical papers: Who was studied, and how were they recruited? What was the intervention, and what was it compared with? What outcome was measured, when, and by whom? How many people dropped out? This structure is often summarized as PICO: Population, Intervention, Comparison, Outcome. Framing your own clinical questions in PICO form ("In adults with acute low back pain, does spinal manipulation compared with usual medical care reduce disability at four weeks?") also makes literature searching far more efficient.

Key point

Study design determines how much a result can be trusted. Experimental designs, especially randomized trials, guard best against confounding; observational designs answer questions trials cannot, such as rare harms and exposures no one could ethically assign. The evidence pyramid ranks designs on average, but the quality of each individual study still has to be judged.

Section 2Bias, confounding and placebo

Error comes in two kinds

Every measurement contains error. Random error scatters results unpredictably around the true value; it shrinks as samples grow and is what confidence intervals describe. Systematic error, or bias, pushes results consistently in one direction, and no amount of extra sample size will remove it. A bathroom scale that reads two pounds heavy every time is biased; weighing yourself a thousand times only makes you more confident in the wrong number. Most of the work of appraising a study is the hunt for bias.

Selection bias

Selection bias occurs when the people who end up in the comparison groups differ in ways that affect the outcome. In a trial, the main protection is proper randomization plus allocation concealment: the person enrolling a patient must not know which group the next patient will join. If a clinician can predict the next assignment, he or she might, even unconsciously, steer sicker patients away from the experimental treatment. Sealed opaque envelopes or central computer randomization prevent this. Trials with inadequate allocation concealment have been shown to exaggerate treatment effects.

Selection bias also enters observational research. Patients who volunteer for studies, who attend every follow up or who seek out a particular therapy are rarely typical. The healthy user effect describes how people who adopt one healthy behavior tend to adopt many others, making any single habit look more protective than it is.

Information (measurement) bias

Information bias arises when exposure or outcome is measured differently between groups. Recall bias is the classic example in case control studies: people who have had a stroke search their memories harder for possible causes than healthy controls do. Detection bias occurs when one group is examined more closely than another. Observer bias arises when an assessor who knows a patient's group rates outcomes more favorably. Many outcomes in spinal care, such as pain, disability and global improvement, are patient reported outcomes, which makes them especially sensitive to expectation.

Blinding

Blinding (also called masking) keeps people from knowing who received which treatment. Up to five groups can be blinded: participants, the clinicians delivering care, outcome assessors, data analysts and the people writing the conclusions. "Double blind" traditionally means participants and investigators, but the term is used loosely, so good reports state exactly who was blinded. In drug trials an identical looking sugar pill makes blinding straightforward. In manual therapy, acupuncture, exercise and surgery it is far harder: the clinician always knows what he or she is doing, and patients often can tell whether a firm thrust was delivered. When participants cannot be blinded, trials can still blind outcome assessors and analysts, and they can ask participants afterward which group they believed they were in to test whether blinding held.

Attrition and analysis

Attrition bias arises when people drop out unevenly. If patients who are not improving quit the treatment group while improving patients stay, the remaining treatment group looks better than reality. The standard protection is intention to treat analysis: every randomized participant is analyzed in the group he or she was assigned to, whether or not that person completed treatment. This preserves the balance created by randomization. A per protocol analysis, which includes only people who followed the plan, can be reported alongside it but tends to overstate benefit. Loss to follow up above about 20 percent generally raises concern.

Confounding

A confounder is a third variable that is associated with the exposure, independently affects the outcome, and is not simply a step on the causal path between them. Suppose a cohort study finds that people who receive regular massage have fewer heart attacks. Income may be a confounder: higher income makes regular massage more affordable, and higher income is independently linked to lower cardiovascular risk through diet, health care access and many other routes. The massage may have no direct effect at all.

Researchers handle confounding by design (randomization, restriction to one subgroup, matching cases and controls on age and sex) and by analysis (stratification and multivariable adjustment). Adjustment can only remove confounding from variables that were measured, and measured well. Residual confounding from unmeasured or poorly measured factors is the main reason observational findings are treated cautiously. Only randomization balances confounders that no one thought to measure.

Confounder higher income Exposure regular massage Outcome fewer heart attacks Apparent link, may not be causal affects who gets massage affects risk by other routes
Figure 11.3 A confounder is linked to both the exposure and the outcome. Here income could explain why massage users have fewer heart attacks even if massage itself does nothing to the heart. Randomization breaks the left arrow.

Regression to the mean and natural history

Many spinal complaints fluctuate. People tend to seek care when symptoms are at their worst, and from a peak, the most likely direction is down. This statistical tendency for extreme measurements to be followed by more average ones is called regression to the mean. Combine it with the natural history of a condition (most acute low back pain improves substantially within several weeks whatever is done) and nearly any treatment started at a symptom peak will appear to work. This is why a control group is essential: only the difference between groups belongs to the treatment.

Study tip. When a patient or a testimonial says "I went in and the next week I was better," ask yourself three questions: Would it have improved anyway (natural history)? Was the person at a peak when care began (regression to the mean)? What else changed that week (co-interventions)? If any answer is yes, a before and after story cannot separate the treatment from the rest.

The placebo response and contextual effects

A placebo is an inert intervention made to resemble a real one. The placebo response is the total improvement seen in a placebo group, which includes natural history, regression to the mean and genuine psychobiological effects. The narrower placebo effect refers to that last component: real changes in brain and body triggered by the context of care. Brain imaging and pharmacology studies have shown that expectation of relief can activate endogenous opioid pathways (an effect partly reversed by the opioid blocker naloxone), engage prefrontal and brainstem pain modulating circuits, and reduce reported pain. Placebo effects are strongest for subjective, centrally processed outcomes like pain, nausea and fatigue, and weakest for objective measures such as tumor size or blood glucose.

The opposite, the nocebo effect, occurs when negative expectations produce real symptoms. Patients warned at length about side effects report more of them. In spinal care, alarming explanations ("your spine is degenerating," "your disc is out") can increase fear and disability, which is why careful language matters.

Researchers now often speak of contextual effects: the benefits that flow from the whole therapeutic encounter, including the clinician's warmth and confidence, a clear explanation, touch, the ritual of treatment, the setting, and the patient's expectations. These effects are real and are part of good care in every profession. They are, however, not specific to any one technique. A treatment's specific effect is the extra benefit beyond what the context provides, and estimating that specific effect is the purpose of a sham controlled trial.

Source of improvementWhat it isHow trials separate it
Natural historyThe condition improving on its own courseAny concurrent control group, even no treatment
Regression to the meanExtreme scores drifting back toward averageConcurrent control; repeated baseline measures
Contextual (placebo) effectsBenefits of expectation, attention and ritualSham or attention controlled comparison
Co-interventionsOther treatments taken at the same timeRecording and balancing other care
Specific effectBenefit of the active ingredient itselfDifference between active and credible sham

The problem of the sham

Designing a credible sham for a hands on therapy is one of the hardest problems in clinical research. A sham manipulation must feel like treatment without delivering the proposed active ingredient, yet no one fully agrees on what that ingredient is: the thrust, the cavitation, the position, the force, or the touch itself. Shams used in trials have included low force thrusts in neutral positions, inactive instrument devices and positioning without a thrust. If a sham is too inert, patients detect it and blinding fails, inflating the apparent effect of the real treatment. If a sham is too active, it may itself help, shrinking the difference. Acupuncture research faces the same dilemma with retractable non-penetrating needles and needling at "non-points," which may still stimulate skin and muscle. When a real therapy outperforms no treatment but does not outperform its sham, the honest conclusion is that much of the observed benefit appears to come from shared contextual factors rather than from the specific technique, not that patients did not improve.

Connection. Contextual effects are not a trick to be embarrassed about. Every clinician, whether a surgeon, physical therapist, chiropractor or acupuncturist, uses them. The ethical lines are honesty and proportion: practitioners can harness reassurance, explanation and a strong therapeutic alliance, but should not claim a specific effect that trials have not shown, and should not let patients pay for long treatment plans on the basis of those claims.

Other biases worth naming

Key point

Bias pushes results in one direction and cannot be fixed by a bigger sample. Randomization with concealed allocation, blinding wherever possible, intention to treat analysis and preregistered outcomes are the main defenses. Natural history, regression to the mean and contextual effects explain much of the improvement after any treatment, which is why a credible comparison group is indispensable.

Section 3Statistics for readers: effect size, confidence intervals, NNT

You do not need to calculate, but you do need to interpret

Most clinicians will never run a regression model, but every clinician has to read results sections. A small toolkit of ideas covers nearly everything: how big was the difference, how certain is it, and does it matter to a patient? This section works through each, using simple numbers you can check by hand.

Measures for yes or no outcomes

Many outcomes are dichotomous: a patient either recovered or did not, had a stroke or did not. Imagine a trial of 200 people with acute neck pain. In the control group of 100, 40 report being "much better" at four weeks. In the treatment group of 100, 60 report the same. The event rates are 0.40 in controls (the control event rate, CER) and 0.60 in the treated group (the experimental event rate, EER). From those two numbers come all the common measures:

Relative measures sound more impressive than absolute ones and are favored in press releases. "Cuts risk in half" can mean 2 in 10,000 down to 1 in 10,000. Always look for the absolute numbers. The NNT is especially useful because it carries the baseline risk inside it: the same RRR gives a small NNT when the outcome is common and a huge NNT when it is rare.

MeasureFormulaWorked example (CER 0.40, EER 0.60)No effect value
Absolute differenceEER minus CER0.20 (20 percentage points)0
Relative riskEER / CER1.51
Relative difference(EER minus CER) / CER0.50 (50 percent more)0
Number needed to treat1 / absolute difference5Infinite
Odds ratio(EER/(1 minus EER)) / (CER/(1 minus CER))1.5 / 0.667 = 2.251

The odds ratio (OR) compares odds rather than probabilities. Odds are the chance something happens divided by the chance it does not: a 0.60 probability equals odds of 1.5 (60 to 40). Case control studies can only calculate odds ratios, because the investigator chose how many cases and controls to include, so true risks are unknown. When an outcome is rare (below about 10 percent), the OR closely approximates the RR; when it is common, as in the example, the OR exaggerates the RR (2.25 versus 1.5).

Control: 40 of 100 better Treatment: 60 of 100 better better in either group extra 20 better only with treatment not better ARR 0.20 NNT = 1 / 0.20 = 5: treat 5 people for 1 extra good outcome
Figure 11.4 An icon array turns a risk difference into people. Most of those who improve in the treatment group (blue) would have improved anyway; only the gold dots are attributable to treatment, which is what the number needed to treat expresses.

Measures for continuous outcomes and effect size

Pain scores, disability questionnaires and range of motion are continuous outcomes. Trials report the mean difference (MD) between groups, for example 8 points on a 0 to 100 pain scale. When different trials use different scales, a meta-analysis converts results into a standardized mean difference (SMD), which expresses the difference in units of standard deviation. The best known version is Cohen's d. By widely used rules of thumb, an SMD of about 0.2 is small, 0.5 moderate and 0.8 large. These cut points are conventions, not laws; a "small" effect on mortality could be enormously important, while a "moderate" effect on a lab value might be trivial.

The question patients care about is whether a difference is big enough to feel. The minimal clinically important difference (MCID) is the smallest change in a score that patients perceive as meaningful. For 0 to 10 pain rating scales, a change of about 2 points or roughly 30 percent is commonly cited as meaningful for an individual. For the Oswestry Disability Index, values around 10 points are often quoted, and for the Roland Morris questionnaire (24 points), about 2 to 5 points. MCIDs vary with baseline severity and with the method used to derive them, so treat them as guides. Be careful too about the difference between an individual's meaningful change and a meaningful difference between group averages: a small average difference can hide the fact that some patients benefit a lot and others not at all.

The p value

A p value is the probability of seeing a result at least as extreme as the one observed if the null hypothesis (no true difference) were correct. A p value below 0.05 is conventionally called statistically significant. Three common misreadings cause trouble:

  1. A p value is not the probability that the null hypothesis is true, nor the probability that the result happened "by chance."
  2. A significant p value says nothing about the size or importance of an effect. In a trial of 5,000 people, a 1 point difference on a 100 point pain scale can be highly significant and clinically meaningless.
  3. A non-significant p value does not prove no effect. A small trial may simply be underpowered: too few participants to detect a real difference. "Absence of evidence is not evidence of absence."

Two error types frame the logic. A type I error (false positive, probability alpha, usually set at 0.05) is concluding a difference exists when it does not. A type II error (false negative, probability beta) is missing a real difference. Statistical power is 1 minus beta, the chance of detecting a true effect of a given size; trials are usually designed for 80 or 90 percent power. Testing many outcomes or subgroups multiplies the chance of a type I error: with 20 independent tests at the 0.05 level, about one false positive is expected by chance alone.

Confidence intervals

A 95 percent confidence interval (CI) gives a range of values for the true effect that are reasonably compatible with the data. Strictly, if the same study were repeated many times, 95 percent of the intervals calculated this way would contain the true value. In practice, read it as "the true effect is plausibly anywhere in this range." Confidence intervals are more informative than p values because they show both the size and the precision of an effect. Narrow intervals come from large samples and consistent data; wide intervals mean uncertainty.

Two reading rules cover most cases. First, if a 95 percent CI for a difference crosses zero (or for a ratio crosses 1), the result is not statistically significant at the 0.05 level. Second, compare the whole interval with the MCID. If the entire interval lies above the MCID, the effect is both real and important. If the interval is above zero but entirely below the MCID, the effect is probably real but too small to matter. If the interval spans from trivial to large, the study simply cannot tell.

-10010203040 Difference in pain (points on 0 to 100 scale), favors treatment to the right no effect MCID 15 A real, important B real, too small C not significant D too imprecise E favors control
Figure 11.5 Reading confidence intervals against two reference lines. Zero tells you whether a difference is statistically significant; the minimal clinically important difference (dashed, red) tells you whether it matters. Interval B is "significant" yet too small to help patients.

Diagnostic test statistics in brief

Examination chapters introduced sensitivity (the proportion of people with a condition who test positive) and specificity (the proportion without it who test negative). Research on orthopedic and neurologic tests also reports likelihood ratios: a positive likelihood ratio above about 10 or a negative likelihood ratio below about 0.1 shifts the probability of a diagnosis substantially, while values near 1 barely change it. Reliability, the agreement between two examiners or between two occasions, is reported with the kappa statistic for categorical findings (0 is chance agreement, 1 is perfect) and the intraclass correlation coefficient for continuous measures. Studies of motion palpation for segmental restriction have generally found low inter-examiner reliability, while tests involving pain provocation tend to agree somewhat better. Recognizing that a test is unreliable is itself useful clinical knowledge.

Study tip. Practice turning every relative claim into an NNT. Take "reduces risk by 25 percent": if the baseline risk is 20 percent, the ARR is 5 percentage points and the NNT is 20; if the baseline risk is 2 percent, the ARR is 0.5 percentage points and the NNT is 200. Same relative effect, ten times different meaning for patients.
Key point

Ask three questions of every result: how big (absolute difference, NNT, effect size), how certain (confidence interval), and does it matter (comparison with the minimal clinically important difference). A p value answers none of these by itself, and relative risks without baseline rates can mislead.

Section 4Systematic reviews and clinical guidelines

Narrative versus systematic reviews

A narrative review is an expert's summary of a topic. It can be a fine introduction, but the author chooses which studies to discuss, and an author who believes strongly in a therapy will tend, often unconsciously, to cite the studies that agree. A systematic review answers a focused question using explicit, reproducible methods that anyone could repeat:

  1. A protocol, ideally registered publicly before work starts, states the question (in PICO form), eligibility criteria and planned analyses.
  2. A comprehensive search covers several databases, trial registries and unpublished sources, so that the review is not limited to easily found, often positive, studies.
  3. Two reviewers independently screen studies and extract data, resolving disagreements by discussion.
  4. Each included study is assessed for risk of bias with a standard tool (for randomized trials, tools rate randomization, deviations from intended care, missing data, outcome measurement and selective reporting).
  5. Results are synthesized, statistically if the studies are similar enough, and the certainty of the evidence is rated.

Reporting standards (the PRISMA statement is the best known) require a flow diagram showing how many records were found, screened, excluded and included, so readers can see where studies were lost. The international Cochrane network produces many high quality systematic reviews in health care and publishes them with regular updates.

Meta-analysis and the forest plot

A meta-analysis is the statistical part of a systematic review: it combines results from several studies into a single pooled estimate, giving larger studies (with more precise results) more weight. Not every systematic review should include a meta-analysis; pooling studies of different patients, treatments or outcomes produces a number that answers no real question, sometimes called mixing apples and oranges.

Meta-analysis results are shown in a forest plot. Each row is one study, drawn as a square (the point estimate, with size proportional to weight) and a horizontal line (its confidence interval). A vertical line of no effect marks zero for differences or 1 for ratios. At the bottom, a diamond shows the pooled result: its center is the pooled estimate and its width is the pooled confidence interval. If the diamond does not touch the line of no effect, the pooled result is statistically significant.

StudyWeight Trial 18% Trial 212% Trial 330% Trial 410% Trial 540% Pooled100% -0.6-0.300.30.60.91.2 favors control favors treatment (SMD) SMD 0.23 (0.13 to 0.33) line of no effect
Figure 11.6 A forest plot. Each square is one trial's estimate (bigger square, more weight) with its confidence interval. Small trials scatter widely and several cross zero, yet the pooled diamond lies clearly to the right: a small but statistically reliable effect.

Heterogeneity

Heterogeneity describes how much study results differ beyond what chance would produce. It is quantified with the I2 statistic, the percentage of variability due to real differences between studies; roughly, 25 percent is low, 50 percent moderate and 75 percent high. High heterogeneity is a signal to ask why: different patient groups, different doses or techniques, different comparison treatments or different risk of bias. A fixed effect model assumes all studies estimate one true effect; a random effects model assumes true effects vary between studies and gives a wider, more cautious interval. Reviewers explore heterogeneity with prespecified subgroup analyses and sensitivity analyses (for example, rerunning the analysis with only low risk of bias trials). A common and important finding is that the apparent benefit of a therapy shrinks when only the best trials are included.

Publication bias and small study effects

If small negative studies go unpublished, a meta-analysis of published studies will overestimate benefit. A funnel plot graphs each study's effect against its precision. Without bias, the points form a symmetrical inverted funnel: large precise studies cluster near the true effect at the top, and small imprecise studies scatter evenly on both sides at the bottom. A missing lower corner, usually on the side of no effect, suggests that small unfavorable studies are missing. Asymmetry has other causes too, including genuine differences between small and large trials, so it is a warning sign rather than proof.

Rating certainty: the GRADE approach

The GRADE system (Grading of Recommendations, Assessment, Development and Evaluation) is used by most guideline groups worldwide. It rates the certainty of a body of evidence for each outcome in four levels: high, moderate, low and very low. Evidence from randomized trials starts as high and from observational studies as low. Five factors can lower certainty: risk of bias, inconsistency (unexplained heterogeneity), indirectness (the studied patients, treatments or outcomes differ from the question), imprecision (wide confidence intervals, few events) and publication bias. Three factors can raise observational evidence: a large effect, a dose response gradient, and situations where all plausible confounding would have reduced the observed effect.

GRADE levelMeaning in plain wordsTypical wording in reviews
HighFurther research is very unlikely to change our confidence in the estimate"Treatment X reduces..."
ModerateThe true effect is probably close to the estimate, but could differ substantially"Treatment X probably reduces..."
LowThe true effect may be substantially different"Treatment X may reduce..."
Very lowWe have very little confidence; the true effect is likely substantially different"We are uncertain whether treatment X..."
Study tip. Learn the GRADE downgrading factors with the phrase "Bias Is Inconsistent, Indirect, Imprecise and Published": risk of Bias, Inconsistency, Indirectness, Imprecision, Publication bias. Then notice the verbs in review conclusions: "reduces," "probably," "may" and "uncertain" map directly to the four levels.

Clinical practice guidelines

Clinical practice guidelines are recommendations, developed systematically, to help clinicians and patients make decisions. A trustworthy guideline is built on systematic reviews, by a panel that includes several professions and patient representatives, with conflicts of interest declared and managed, using an explicit method to move from evidence to recommendation, and with plans for updating. Under GRADE, a recommendation has both a direction (for or against) and a strength (strong or conditional, also called weak). Strength depends not only on certainty of evidence but on the balance of benefits and harms, patients' values, cost and feasibility. A strong recommendation can rest on moderate evidence when harms are small and benefits clear; a conditional recommendation means that the right choice varies with individual circumstances and should be shared with the patient.

The AGREE II instrument is commonly used to appraise guideline quality across domains such as scope, stakeholder involvement, rigor, clarity, applicability and editorial independence. Readers should notice when two guidelines disagree: often they asked slightly different questions, included different evidence or weighed costs and values differently.

Connection. Most current low back pain guidelines from North America, Europe and Australia converge on a similar message: reassure and educate, encourage staying active, avoid routine imaging without red flags, and prefer nonpharmacologic options such as exercise, heat, spinal manipulation, massage, acupuncture, yoga, tai chi and cognitive behavioral approaches before or alongside medicines, while discouraging routine opioids. This convergence is a good example of evidence moving practice across professions.
Key point

Systematic reviews use preplanned, reproducible methods to find and appraise all relevant studies, and meta-analyses pool their results in forest plots. Heterogeneity and publication bias must be examined, not ignored. GRADE rates how certain the evidence is, and guidelines combine that certainty with harms, values and costs to make strong or conditional recommendations.

Section 5What the evidence says about spinal manipulation and holistic therapies

Applying the tools fairly

The methods in this chapter apply to every intervention, whether a prescription drug, a surgery, an adjustment or an herbal tea. Fair appraisal means holding all of them to the same standard. The summaries below reflect the general direction of high quality systematic reviews and guidelines; individual reviews differ in detail, and new trials keep arriving, so treat these as a map, not a final verdict.

Spinal manipulation for low back pain

Spinal manipulative therapy (SMT), a high velocity, low amplitude thrust, is one of the most studied manual therapies, with dozens of randomized trials. For acute low back pain, systematic reviews have found moderate certainty evidence of modest improvements in pain and function over several weeks, on the order of 10 points on a 100 point pain scale, compared with control treatments. For chronic low back pain, large reviews pooling dozens of trials have found that SMT produces effects similar to other recommended treatments such as exercise, and better than some non-recommended comparisons, with low to moderate certainty. Against sham manipulation, differences are small and often below commonly used thresholds for clinical importance. The fair summary is that manipulation is a reasonable, guideline supported option for low back pain, roughly comparable to other recommended conservative care, rather than either a cure or a placebo.

Neck pain and headache

For neck pain, manipulation and mobilization, especially combined with exercise, show short term benefits in pain and function with low to moderate certainty, and they perform similarly to each other. For cervicogenic headache (headache referred from the neck), several trials suggest benefit. For migraine and tension type headache the evidence is more limited and inconsistent; some trials show small improvements, while sham controlled trials tend to show smaller differences. Exercise, sleep and trigger management remain central.

Conditions outside the musculoskeletal system

Historically, some chiropractic models proposed that correcting spinal subluxations could improve visceral and systemic health (Chapter 9 traces this history). Trials have tested manipulation for asthma, high blood pressure, infantile colic, ear infections, painful menstruation and other conditions. Reviews by international panels that included chiropractic researchers have concluded that there is no reliable evidence that SMT is effective for these non-musculoskeletal conditions. Much of the modern chiropractic profession accepts this finding and focuses care on musculoskeletal complaints, while some practitioners continue to hold broader traditional views. A student can respect the history of the vitalistic model and still be clear about what trials show.

Safety of manipulation

Mild, short lived side effects are common after manipulation: local soreness, stiffness, headache or tiredness, reported by a sizeable minority of patients and usually resolving within 24 to 48 hours. Serious adverse events are rare. The most discussed is cervical artery dissection (a tear in the wall of a vertebral or carotid artery) leading to stroke. Dissection can itself cause neck pain and headache before any stroke occurs, so patients with an early dissection may seek care for those symptoms. Case crossover studies found that the association between a recent chiropractic visit and vertebrobasilar stroke was similar to the association with a recent visit to a primary care physician, suggesting that much of the link reflects people with early dissection seeking care rather than manipulation causing it. A causal contribution in some cases cannot be excluded. Practical safety therefore rests on careful history and screening for red flags (sudden severe "worst ever" headache, new neck pain with neurological signs such as dizziness, double vision, slurred speech or difficulty swallowing) and on referring rather than manipulating when those are present. Lumbar manipulation has very rarely been associated with cauda equina syndrome and disc herniation.

Study tip. Keep three verdicts separate for any therapy: Does it beat no treatment? Does it beat a credible sham? Does it beat or match other recommended care? Many manual and holistic therapies answer yes, small or no, and "similar." Mixing those three questions is the most common source of overclaiming and of unfair dismissal alike.

Acupuncture

Acupuncture, from traditional East Asian medicine, inserts fine needles at defined points understood in its own framework to regulate the flow of qi along meridians. Large individual patient data meta-analyses of chronic pain (back, neck, osteoarthritis, chronic headache, shoulder pain) found that acupuncture outperformed both sham acupuncture and no acupuncture controls. The difference from no acupuncture was moderate; the difference from sham was small, though statistically reliable, suggesting that a substantial part of the benefit comes from contextual factors and that needling location may matter less than traditional theory proposes. With trained practitioners and sterile single use needles, serious adverse events (pneumothorax, infection, nerve injury) are rare; minor bruising and soreness are common.

Massage, yoga, tai chi and mindfulness

Massage shows short term benefits for low back and neck pain with low certainty, and it is very safe; caution applies to areas of suspected deep vein thrombosis, fragile skin and patients on anticoagulants receiving deep pressure. Yoga for chronic low back pain improves function and pain modestly compared with non-exercise controls, with effects similar to other forms of exercise and occasional minor strains. Tai chi has moderate to high quality evidence for reducing falls in older adults and helpful evidence for knee osteoarthritis and fibromyalgia symptoms. Mindfulness based stress reduction has been compared head to head with cognitive behavioral therapy for chronic low back pain and produced similar improvements, both better than usual care. These mind body approaches are covered more fully in Chapter 13.

Herbal products and supplements

Herbal medicine spans many traditions. Some products have reasonable trial evidence for particular uses (for example, St John's wort for mild to moderate depression), many have little or none, and natural does not mean harmless. Key safety issues are herb drug interactions (St John's wort induces liver enzymes and can reduce the effect of oral contraceptives, anticoagulants, transplant drugs and others; ginkgo and high dose fish oil may increase bleeding risk), direct toxicity (kava and some traditional products have been linked to liver injury) and product quality, since supplements are not tested before sale the way drugs are and some have been found contaminated or adulterated. Clinicians should always ask about supplements.

Homeopathy and energy therapies

Homeopathy is based on the principles that "like cures like" and that serial dilution and shaking increase potency; many remedies are diluted beyond the point where any molecule of the original substance is likely to remain. Comprehensive government sponsored reviews have found no reliable evidence that homeopathy is effective for any health condition beyond placebo. Energy therapies such as Reiki and therapeutic touch aim to channel or balance a biofield; trials are mostly small with high risk of bias, and better controlled studies have not shown effects beyond sham. These practices are usually physically harmless; their main risk is indirect, when they delay effective care for a serious condition.

vs usual orother care vs sham(specific effect) Seriousharms SMT, low back pain SMT, non-MSK conditions Acupuncture, chronic pain Yoga, chronic back pain Massage, back pain Homeopathy, any condition similar, modestsmallrare not shownnot shownrare modestsmallvery rare modestnot testablevery rare short termfew shamsvery rare none shownnone showndelayed care Green: favorable. Amber: small, uncertain or needs screening. Red: not supported. Gray: no fair test.
Figure 11.7 A simplified evidence map. Many therapies help compared with usual care or no treatment yet show only small specific effects against sham. Color describes the general direction of current systematic reviews, not a ruling on any individual patient.
TherapyBest supported usesCertainty (approximate)Main safety considerations
Spinal manipulationAcute and chronic low back pain; neck pain; cervicogenic headacheLow to moderateTransient soreness common; screen for dissection and cauda equina red flags
AcupunctureChronic back, neck, knee and headache painModerateSterile single use needles; rare pneumothorax
MassageShort term back and neck pain reliefLowAvoid deep pressure over suspected clots; anticoagulant use
Yoga and tai chiChronic low back pain; falls prevention; knee osteoarthritisModerateMinor strains; modify poses for osteoporosis and glaucoma
Mindfulness programsChronic pain coping, stress, anxietyModerateOccasional distress in trauma survivors; trained teachers
Herbal productsVaries by product; a few have positive trialsVery low to moderateDrug interactions, liver toxicity, contamination
HomeopathyNo condition shown beyond placeboHigh certainty of no specific effectDelay of effective care
Connection. Explaining evidence to a patient is a skill. A useful script is honest and specific: "For back pain like yours, manipulation helps about as much as exercise or other recommended care. Most people improve within weeks either way. Staying active is the most important thing, and if you notice numbness in the saddle area or trouble passing urine, seek care right away." It respects the patient's choice, sets realistic expectations and protects safety.
Key point

Spinal manipulation is a reasonable, guideline supported option for low back and neck pain with benefits similar to other recommended care, small specific effects over sham, common mild side effects and rare serious ones that careful screening helps prevent. Evidence does not support manipulation for non-musculoskeletal disease. Holistic therapies range from moderately supported (acupuncture, yoga, tai chi, mindfulness for chronic pain) to unsupported (homeopathy), and each deserves the same fair, specific appraisal.

The chapter in one breath

  • Study design sets the ceiling on trustworthiness: experiments assign exposure, observational studies watch it.
  • Randomized controlled trials balance known and unknown confounders; cohort, case control and cross-sectional studies answer questions trials cannot, including rare harms.
  • The evidence pyramid ranks designs on average, but individual study quality must still be judged.
  • Bias is systematic error; allocation concealment, blinding, intention to treat and trial registration are the main defenses.
  • A confounder is linked to both exposure and outcome; only randomization controls unmeasured confounders.
  • Natural history, regression to the mean and contextual effects make any treatment look effective without a control group.
  • The NNT is 1 divided by the absolute risk difference; relative risks without baseline rates mislead.
  • A confidence interval shows size and precision; compare it with zero for significance and with the MCID for importance.
  • A p value is not the probability that the null hypothesis is true and says nothing about effect size.
  • Systematic reviews use reproducible methods; forest plots show each trial and the pooled diamond; heterogeneity and publication bias need checking.
  • GRADE rates certainty high, moderate, low or very low, and guidelines make strong or conditional recommendations.
  • Spinal manipulation helps low back and neck pain about as much as other recommended care, is not supported for non-musculoskeletal disease, and requires red flag screening; holistic therapies range from moderately supported to unsupported.

Words worth knowing

TermWhat it means
Absolute risk reductionThe difference in event rates between control and treated groups, expressed in percentage points.
Allocation concealmentKeeping the upcoming group assignment hidden from those enrolling participants so they cannot influence who goes where.
BlindingKeeping participants, clinicians, assessors or analysts unaware of group assignment.
Case control studyAn observational study that starts with people who have and do not have an outcome and compares their past exposures.
Cohort studyAn observational study that follows exposed and unexposed people over time to compare outcomes.
Confidence intervalA range of values for the true effect that are reasonably compatible with the study data.
ConfounderA third factor associated with both exposure and outcome that can create a false or distorted association.
Contextual effectsBenefits arising from the whole care encounter, such as expectation, attention, touch and explanation.
Effect sizeA measure of how large a difference is, often in standard deviation units, as with Cohen's d.
Forest plotA graph showing each study's estimate and confidence interval with a pooled diamond at the bottom.
GRADEA widely used system for rating certainty of evidence as high, moderate, low or very low.
HeterogeneityVariation in results between studies beyond what chance explains, often measured with I squared.
Intention to treatAnalyzing every participant in the group to which he or she was randomized, regardless of adherence.
Meta-analysisStatistical pooling of results from several studies into a single weighted estimate.
Minimal clinically important differenceThe smallest change in an outcome score that patients perceive as meaningful.
Nocebo effectReal symptoms or worsening produced by negative expectations.
Number needed to treatThe number of patients who must be treated for one additional patient to benefit; 1 divided by the absolute risk difference.
Odds ratioThe ratio of the odds of an outcome in one group to the odds in another; the main measure from case control studies.
p valueThe probability of a result at least this extreme if there were truly no difference.
Placebo responseAll improvement seen in a placebo group, including natural history, regression to the mean and contextual effects.
Publication biasThe tendency for positive studies to be published more often, distorting the visible evidence.
Randomized controlled trialAn experiment in which participants are assigned to groups by chance and outcomes are compared.
Regression to the meanThe tendency of extreme measurements to be followed by values closer to average.
Relative riskThe risk of an outcome in one group divided by the risk in another.
ShamA simulated procedure designed to resemble a real treatment without its proposed active ingredient.
Systematic reviewA review that uses explicit, reproducible methods to find, appraise and synthesize all relevant studies on a question.
Type I errorConcluding that a difference exists when it does not; a false positive.
Type II errorFailing to detect a difference that really exists; a false negative.

Check yourself

Researchers identify 300 people who had a vertebrobasilar stroke and 1,200 similar people who did not, then review records for chiropractic and primary care visits in the previous month. Name the design and the measure of association it produces.
Case control study; odds ratio. It starts from the outcome and looks back at exposures. Because the investigators chose the numbers of cases and controls, true risks cannot be calculated, so the odds ratio is reported.
In a trial, 30 percent of control patients and 20 percent of treated patients develop chronic pain at one year. Calculate the ARR, the RRR and the NNT.
ARR 10 percentage points, RRR 33 percent, NNT 10. 0.30 minus 0.20 = 0.10; 0.10 / 0.30 = 0.33; 1 / 0.10 = 10 patients treated to prevent one case of chronic pain.
Why does concealing allocation matter even in a trial that uses a perfectly random sequence?
Because a known sequence can be subverted. If enrolling staff can see the next assignment, they may steer certain patients into or away from a group, recreating selection bias that randomization was meant to prevent.
A trial of 4,000 people reports that a new pain treatment lowered scores by 2 points on a 0 to 100 scale, p = 0.001. How should you interpret it?
Statistically significant but probably clinically unimportant. Large samples detect tiny differences. A 2 point change is far below typical MCIDs for pain, so the p value shows the effect is probably real, not that it matters.
A 95 percent confidence interval for the difference in disability scores is 1 to 22 points, and the MCID is 10. What can you conclude?
Probably a real benefit, but its importance is uncertain. The interval excludes zero, so the result is significant, but it spans from trivial (1) to clearly important (22). A larger or more precise study is needed.
Patients with severe acute back pain receive a novel treatment and their average pain falls by half in three weeks. Give two reasons this does not prove the treatment works.
Natural history and regression to the mean (also contextual effects). Acute back pain usually improves on its own, and people enrolled at their worst tend to drift toward average. Without a control group the treatment's share cannot be separated.
In a forest plot, three small trials cross the line of no effect but the pooled diamond does not. Is that contradictory?
No. Small trials are imprecise and individually underpowered. Pooling increases precision, so a consistent small effect can become statistically clear even when single trials were inconclusive.
Evidence from randomized trials on a therapy starts at high certainty in GRADE but ends at low. Name factors that could have caused the downgrade.
Risk of bias, inconsistency, indirectness, imprecision or publication bias. For example, unblinded trials with high dropout (bias) and wide confidence intervals from few participants (imprecision) could each lower certainty by one level.
Real acupuncture beats no acupuncture by a moderate amount but beats sham acupuncture by a small amount. What does that pattern suggest?
Much of the benefit is shared with the sham. Contextual factors such as expectation, attention and skin stimulation contribute substantially; the specific effect of correct needle placement appears small, though not zero.
Why do case crossover studies matter when discussing neck manipulation and vertebral artery stroke?
They help separate cause from care seeking. They found a similar association between stroke and recent primary care visits, suggesting that people with an early dissection seek care for neck pain and headache, so the association is not simply manipulation causing stroke. Screening for red flags remains essential.