One thing that I don't understand are people who claim to practice science but call on others to prove things even after they have done their experiments and released at least some of their data but they themselves the person attacking the science will not practice the scientific method to further prove or disprove what they are attacking. So my point here is that there are two types of people. Those who claim to practice the scientific method and those who really practice it.
Here's an example of why we use the scientific method, why we employ peer review and how failing to follow the practice leads to self fulfilling prophecies. It's based on a real example offered by a well known physicist. I'll change the scenario a bit. In the original the undergrad researcher is an Experimental Psych student. In my example she will be an undergrad researcher is studying Experimental Parapsychology.
The experimental design calls for her to run rats through a Skinner Box 10 times each and see how long it takes them to figure out the maze, get to the food shoot and pull the lever to release the food.
The parapsychological part of the design is that she will have another student sending out "thought waves" to the rats to help them figure out the maze. The proof of the pudding will be measured in how quickly they learn as compared to a control group who weren't offered thought waves. The control group experiment was run by a third single blind researcher, using the same Skinner Box, who was unaware that he was a control group or that another experiment would be run later. All times for each rat in each group are the average of their 10 runs through the box.
She gets her rats and starts running her experiment. Over time she discovers that the rats start learning the maze faster and faster the longer she runs the experiment. Rat 1 learned the maze in about the same time that the control group learned it. Rat 2 faster than 1, 3 faster than 2...Rat 20 learned it faster than Rat 19 and so on.
She applies a statistical model and the numbers indicate that by Rat 20 they were learning the maze far quicker than mere random chance. By Rat 100 the times level off but they are all still stupendously quick as compared to the control group.
This effect was not present at all in the control group. Variations in the learning curve times were all within the norm and had no particular pattern. She concludes that there is a paranormal effect. The rats start learning more quickly as the thought wave sender became more efficient over time (practice effect) at sending mental ques to the rats.
She's reviewed her design and (correctly) concludes that there was no cheating, all data was correctly logged, the math was all correct, no rats were used for more than one trial of 10 runs, it was always the same Skinner Box and nothing about the box was altered between groups or individual trials.
There's more to the story but the question right now is whether or not she has come to a valid conclusion based on her design and results?