How Do You Know If A Mental Health Treatment Really Works?

How do you know if a mental health treatment really works? Important information to consider before taking a psychiatric medication or herb for your mental health condition.

When It Comes to Your Mental Health ... It's Buyer Beware

"Take this herb!"

"Try this supplement!"

"Our pill is the best!"

"Listen to this tape on positive thinking and you'll recover from anything."

When it comes to mental health treatments, there's a lot of hype out there. So how do you know which treatments really work?

Psychiatric Medications and Scientific Evidence

Like clothes and cars, scientific evidence varies in quality. When you read a claim that a treatment works, it is a good idea to try to work out how good the evidence really is.

• Randomized controlled trials (RCTs): the best evidence

The randomized controlled trial is the Rolls Royce of scientific evidence. In an RCT, the people who volunteer to test out the treatment are randomly placed either in a treatment group (eg, given antidepressants) or a no treatment group (eg, given a sugar pill). A systematic review is a special unbiased method of identifying all relevant trials of a treatment and combining the results. The best possible evidence comes from a systematic review of all RCTs of a treatment. All FDA approved mental health medications must go through randomized controlled trials.

• Controlled trial, not randomized: the next best evidence

Sometimes scientists use controlled trials where volunteers are not randomly placed in groups. Suppose we give all the patients from a depression clinic in Miami a secret depression buster formula. At the same time, we give all the patients from a depression clinic in Chicago sugar pills. We find that the Miami patients recover more quickly than the Chicago patients. We might conclude that the depression buster formula works. We could well be right. However, we can't be sure. The difference between the two groups might reflect a difference in the clinics, a difference in the type of people who attend the clinics, or something different about the two cities. The non-randomized controlled trial is good evidence but not as good as the RCT.

• Before and after group study

Another type of evidence involves measuring health before and after treatment. If there is an improvement, we might conclude that a treatment works. The problem with this type of study is that we can't be sure that improvement is due to the treatment. The volunteers might have improved anyway. This type of study is not as good as a study with a control group.

• Little or no evidence

Sometimes people claim that a mental health treatment works on the basis of their personal or professional experience. For example, Mary Downtheroad tells her friends that pulling her ears three times each morning has changed her life. Now life is wonderful and she no longer becomes depressed. Mary believes that ear pulling has helped her but she cannot provide any scientific evidence to support her belief. Maybe trials in the future will prove her correct and perhaps they won't. This anecdotal information is the "skateboard" of scientific evidence - you can't tell if and when you will crash.

What else is important?

• Studies should involve enough people that we can be confident of the findings

The larger a study, the more likely we are to find an effect of treatment if it exists.

• The best studies are 'blind'

A blind study means that the people involved in the study do not know who is receiving the treatment and who is not. (In a single-blind study, the patients do not know if they have been given the active treatment or the placebo. In a double-blind study, neither the volunteers nor the people treating or assessing them know who is receiving the actual treatment). The advantage of a blind study is that the volunteers and researchers cannot consciously or unconsciously bias the results of the study.

• Findings should be tested for statistical significance

Sometimes differences occur by chance. There are special statistical methods for deciding if a difference between two groups (eg, one that receives a treatment and one that doesn't) is real. All good studies should report whether a finding is statistically significant.

• Findings should be meaningful

Sometimes a treatment can produce a real (statistical) effect but the effect is not very large. All other things being equal, a treatment that makes a large difference is better than a treatment that makes a small difference.