FAQs

Medical Literature

Most good papers have a table that shows how the cohorts or groups are alike in a number of areas. This allows the reader to see just how different they might be when compared to ask the question about how their differences matter. It also can show areas of bias that change the answer because some treatment or disease outcomes can be changed by the selection of a subtly different cohort. 

Statistical data is collected and put on a graph, even if the graph isn’t in the paper. When data is plotted with trial subjects it usually follows a “bell curve”, with most of the people getting one result, and as you move away from the middle, fewer and fewer people get more or less of a result. In other words, most people driving on the interstate are going 75. Fewer are going 70 or 80. Even fewer are going 65 and 80 and so on. It is rare for someone to go 100, and also rare for 40 (unless on the ramp!). The people going at 40 or 100 are on the “tails” of the curve. The majority going 65-80 are in the fat bell-shaped middle. 

If the question being studied really makes a difference, then the bell curve will move for the group getting that difference. How much movement is reflected by a value called the “p-value”, which gets very small when the difference is higher. If the difference is very low, then the item studied doesn’t matter much, and the p-value is high. Most people think that a p-value <0.05 makes the result "significant" or not due to a random event. This isn't always so but it is a good rule of thumb. Look for "confidence intervals" which aren't as black/white as a low p-value. These are very good because they average the statistical results and give a range of probable truths. You might see this as saying that drug A worked for 10% (95% CI -8 - +20). This means that it worked about 10% of the time, but to get 95% confidence it could be as low as -8% (they got worse!) to +20% (even better). Drug A might not be so hot. On the other hand, if it is drug B worked 65% (95% CI +62 - +80) then there is going to be a lot of people wanting that item! The bigger the trial, the smaller the spread of CI values. Note that this is a 95% CI - the tails are left out because those values are happening to the fewest numbers of people. The statistics will talk about the one-tailed or two-tailed test. If something can go up OR down, then a two-tailed test is needed because the people at the top AND the bottom of the curve can be moved around. They will also talk about paired tests. In general if you measure something once, and then measure it again later, you need a paired test because the results are probably related to each other. A good example is pulse rate; the guy with a pulse of 50 at the beginning will likely have a lower pulse later than the guy that starts at 90 - the data is paired. Sometimes the statistics seek to say that something caused something else, and a lot of information about correlation will be presented. An example of this is using statistics to say that drunk driving causes accidents, or "correlates" with them. An "r-value" will be presented but this isn't always used correctly. Be skeptical. Decide if it makes sense that it correlates. Look for other things to explain the finding.

The methods section holds the answers to possible bias, number of participants, and general performance of the research. Key items to appreciate are what kind of study was done and how many participants were involved. Were they followed for days, months or years, and did the item being studied warrant a different followup period? Sometimes the statistics used have to be examined, and I think that the more confusing the discussion about the statistics, the less impact the result will have because it takes a lot of math to show a minimal result, but very little math to show a clear result. You should also pay close attention to how people were screened, selected and rejected from the study – a lot of things that you might find important can be discarded in order to simplify the outcome or to prove a point that could not be proved if everyone were included. On the other hand, if the author uses no statistics at all and simply states that the numbers support the idea, then the proof has not really been given.

1.  Randomised Controlled Trial: a group of patients are signed up to help answer a question, and somehow they are randomly selected to get one treatment or another. This can be a choice between two or more different medications or surgery options, and may or may not include a placebo. This sort of trial is the most difficult to perform but the results are usually the best for seeing how good some treatment really is. The information is collected up front and on purpose in an organized way. 

2.  Cohort studies: two groups (each group is a “cohorts”) differ in exposure to the item being studied, and they are compared to each other to learn what happens when that item is changed. One good example is people who do and people who do not drink alcohol. Another is smokers and non-smokers. It is much easier to do a cohort study because the groups have self-selected themselves out over years and the data can be found by chart reviews without a lot of up-front planning. Sometimes the data comes from huge databases collected by the government. 

3.  Case-controlled studies: patients with a given condition are matched with another group that does not have that condition, and their differences are studied. These are also often gathered from databases and chart reviews to assess trends in large groups of people 

4.  Cross-sectional surveys: patients are given questionnaires and asked to rate things under consideration. Sometimes they are surveyed twice – one a the beginning of the study and again at the end. These studies help figure out how people change, but a bad questionnaire gives a bad answer. And since people are answering by their memory of things sometimes, the results can be imperfect.

5.  Case reports: an interesting thing has happened, and the doctor has submitted the case for the journal to report. Sometimes these report a series of cases that are similar.

You might not be able to avoid statistics. But sometimes authors show some other useful numbers you should look for. Let’s imagine 1000 people driving on the highway. In a construction area, suppose there are twenty accidents a day for those thousand people. In the non-construction area let there be five accidents. Let’s say the mileage of each stretch is the same so we have no “bias”.

1.  Relative risk: 20 accidents without construction and 5 with it. The risk of accident in the construction area is 20/1000 (2%). The risk of accident in the normal area is 5/1000 (0.5%). The relative risk is 20/5, or 2%/0.05% which equals 4. You are four times more likely to have an accident in the construction area.

2.  Absolute risk: If there is a 2% chance of accident in the construction area, and 0.5% risk otherwise, then the absolute difference is 2 – 0.5 = 1.5% and you could say that driving in the construction area raises your risk of accident by 1.5%. This doesn’t sound as bad as relative risk. It probably is the better way of looking at it, and isn’t as dramatic.

What if putting a cool new sensor in your car lowered your chance of accident in the construction area by 20%, so the numbers of accidents in construction are now 16 and in the clear are 5. This is like getting a treatment for a disease.

3.  Relative risk reduction: the new relative risk is 16/5 = 3.2, and the relative risk has lowered from (4-3.2)/4 = 20%. It isn’t a surprise that the number is 20%. We already said that the sensor made one group 20% better, a relative improvement.

4.  Absolute risk reduction: first find the absolute risk for the situation with the sensor, and it is (16-5)/1000 = 1.1%, so the absolute reduction is 1.5 – 1.1 = 0.4% and this is how much the sensor lowers accidents overall.

5.  Number needed to treat: this is the number of sensors we would have to install to prevent one accident. This is one of my favorite numbers because if the number is really high, then maybe it isn’t worth giving a medicine or treatment to all those patients. In our situation, we find that the number of sensors that have to be installed to prevent one accident among 1000 drivers is 1/(absolute risk reduction) = 25. Might be worth it? Only if you are one of the ones that was going to have an accident, and we don’t know that ahead of time do we? Such is statistics. Nobody can predict the future for an individual, but we can get an idea of what will happen to the herd.

All medical papers have similar formats and within a specific journal will tend to have a specific style, so few clues to the worth of the paper are had there. So we have to ask ourselves some basic questions at the start. Some good in-depth information on this is located in the book “How To Read A Paper” by Trisha Greenhalgh. She reminds us to look at the work of Sackett’s “Users guides to the medical literature” published in the Journal of the American Medical Association which were written in about 25 parts from 1993 to 2000. I’m leaning on Greenhalgh’s work here….

1. Why was the study done and what hypothesis were the authors testing? This is usually stated in the first paragraph or the abstract of the paper. 

2. What type of study was done? It is good to know if this paper was the research of the author, a review of other literature, a clinical trial with controls, etc.

3. Was a large enough group of people involved? A paper saying a medicine works but only trialed on 20 people might be suspect.

4. Was the study design appropriate to answer the question? A review cannot answer if a particular medicine works for a disease. A clinical trial cannot answer what all the treatment choices might be, etc.

5. Was there unreasonable bias in the paper? If the author owns the company of the device in the trial, you can wonder about the independence of the work. There are many other forms of bias.

A useful paper will have answers to these questions that all fit well.

There is a wide variety of information available via the Internet, and this is the best place for a patient to lookup questions. But because the variety is so wide, it is easy to get distracted or lost in the details. It is also possible to get one opinion and not know how it fits into the bigger picture, or why that opinion might or might not apply to a particular question. All journals published in recent years have been indexing their articles on Pubmed and this is a fantastic place to look up papers written by experts in the field. You will be able to see an abstract of the paper and the conclusions. Most of the details of the paper and what is good and bad about the research are buried in the actual paper you won’t be able to read. To read the actual paper you should contact your library and get help from them to receive it. It helps to be very specific when searching Pubmed because the articles will be very numerous otherwise. And it is up to you, dear reader, to figure out which ones are useful and which ones are not. If you want to learn more about the topic in general, add the word “review” to your search, and you will find articles that “review” the topic, and which are more likely to guide a good general understanding. 

After Pubmed, next best of the free searches is with Google. It is even harder there to figure out what is irrelevant, what is useless, and what is meant to make you buy something. Please read carefully there and bookmark things that look relevant. It is easier to search and bookmark and come back later to read it together than it is to just read as you go. 

Another excellent source of review literature is the Cochrane Library which seeks to gather information into the best practices and summaries of numerous problems. Excellent instructions on how to use this service are on their site.

Load More