• Sibbo@sopuli.xyzEnglish
    127·
    17 days ago

    P-hacking is the academically problematic practice of attempting to come up with a question for which the data offers a significant p-value (probability value), as opposed to correct scientific analysis in which a question is formulated clearly and then answered with data.

    It took a while to parse this comic, but with the explanation it’s probably much easier to understand for anyone who doesn’t know what P-hacking is.

    • Agent641@lemmy.worldEnglish
      30·
      17 days ago

      Reduce the sample size by increasing qualifying parameters until you find a dataset that matches your hypothesis in such a way that the research grant will be approved.

      • psycotica0@lemmy.caEnglish
        12·
        16 days ago

        Sometimes even worse, which is to collect a raft of data testing one hypothesis, and then realize it all came up empty, and so go looking for any data you can form a new hypothesis from that matches the data you already have.

    • Grail@multiverse.soulism.netEnglish
      16·
      17 days ago

      One thing you can use p-hacking for is that if you want to prove vaccines are bad, give a bunch of kids vaccines and measure 20 different vital indicators. Then theorise that the vital indicator which got worse was caused by the vaccines.

    • Windex007@lemmy.worldEnglish
      5·
      16 days ago

      Thanks for that. I’d never heard the term before.

      It sounds a little subjective though? Are there features that can be used to quantity how “P-Hacky” something is?

      I feel like a sports state of “a team tends to lose if thier top scoring player in the first quarter is injured before the end of the first half” has a lot of specific weirdness, but my intuition drives that this specifically could be a very legitimate observation.

      How do you draw the line?

      • 42firehawk@fedinsfw.appEnglish
        6·
        16 days ago

        Usually p hacking doesn’t come from 1 constraint, especially a well explained one, but instead comes from adding a couple or completely unexplained constraints (like a team losing more if their coaches wife is in one section of the stands or another) because at that point it’s decreasing the number of samples (times you have as a reference) to force a significant result.

        So usually for sports p hacking is stats about 1 team only, rather than a general stat about the sport. Preferably a restriction on the other team, then a follow up game based restriction so it seems plausible to the viewer.