Normative theories

eclipse 1919
Popper (rational reconstruction) - conjecture and refutation

Lakatos
Really p(D|H) < 1, hence
p(H|not-D) = p(D|H)*p(H) / p(D) > 0

Platt - strong inference (20 questions): p(D|H)=1 and p(D|not-H)=0

2 4 6 problem

Give examples to test your rule.

Modified problem:

1 2 3

Lodges

A patient has a .8 probability of having Chamber-of-Commerce disease and a .2 probability of Elk's disease. (He surely has one or the other.) A tetherscopic examination yields a positive result in 90% of patients with Chamber-of-Commerce disease and in 20% of patients without it (including those with some other disease). An intraocular smear yields a positive result in 90% of patients with Elk's disease and in 10% of patients without it. If you could do only one of these tests, which would it be? Why?

Z ray

A patient has a .8 probability of umphitis. A positive Z ray result would confirm the diagnosis, but a negative result would be inconclusive; if the result is negative, the probability would drop to .6. The treatment for umphitis is unpleasant, and you feel it is just as bad to give the treatment to a patient without the disease as to let a patient with the disease go untreated. If the Z ray were the only test you could do, should you do it? Why, or why not?

ET scan

A patient's presenting symptoms and history suggest a diagnosis of globoma, with about .8 probability. If it isn't globoma, it's either popitis or flapemia. Each disease has its own treatment, which is ineffective against the other two diseases. A test called the ET scan would certainly yield a positive result if the patient had popitis, and a negative result if she has flapemia. If the patient has globoma, a positive and negative result are equally likely. If the ET scan was the only test you could do, should you do it? Why, or why not?

Probability theory and hypothesis testing

Datum = dot or no dot
Hypothesis = red, blue, or green
You win $1.00 if you guess right.
Is the datum worth getting?

Red   Blue  Green
Urn 1
dot1384
no dot392412
Urn 2
dot0126
no dot    522010
Urn 3
dot13012
no dot39324

Information bias, ET scan

A patient's presenting symptoms and history suggest a diagnosis of globoma, with about .8 probability. If it isn't globoma, it's either popitis or flapemia. Each disease has its own treatment, which is ineffective against the other two diseases. A test called the ET scan would certainly yield a positive result if the patient had popitis, and a negative result if she has flapemia. If the patient has globoma, a positive and negative result are equally likely. If the ET scan was the only test you could do, should you do it? Why, or why not?

Test result Globoma Popitis Flapemia
Positive ET scan 40 10 0
Negative ET scan 40 0 10
All patients 80 10 10

Baron, Beattie, and Hershey

You open the refrigerator one night and discover that there is water on the floor. You think that the refrigerator has stopped working and the ice has melted.   p(H)=.85

Question 1. Had there been a large jug of water in the refrigerator?
p(D|H)=.1, p(D|~H)=.8, rating=10

Question 2. Was the refrigerator old?   p(D|H)=.7, p(D|~H)=.4, rating=25

Joint probabilities, p(H & D) = p(D|H) * p(H)

Question  Answer    H (.85)   ~H
1yes.085.12
no.765.03
2yes.595.06
no.255.09

Prescriptive heuristics

1. How likely is a yes answer, if I assume that my hypothesis is false?

2. Try to think of alternative hypotheses; then choose a test most likely to distinguish them - to make some less probable and others more probable.

Stop here

Z ray

A patient has a .8 probability of umphitis. A positive Z ray result would confirm the diagnosis, but a negative result would be inconclusive; if the result is negative, the probability would drop to .6. The treatment for umphitis is unpleasant, and you feel it is just as bad to give the treatment to a patient without the disease as to let a patient with the disease go untreated. If the Z ray were the only test you could do, should you do it? Why, or why not?

Test result Umphitis No umphitis
Positive Z ray 50 0
Negative Z ray 30 20
All patients 80 20

ET scan response

In this case, the interviewer (Baron) has asked why the test would be worth doing even if it cost $5 (this probe being used to negate the assumption that the test is free of both cost and risk). "Because at least to me, the added information is worth the money [section omitted]. [Interviewer: How is it gonna help treat it?] It will give added information. If you do have globoma, the test means nothing. So giving the test or not giving the test, there's no difference. If you do have one of the other diseases, however, the test will mean something. [What does it mean?] If the test is negative, given that you don't have globoma, you'll have flapemia. If the test is positive, given that you don't have globoma, you'll have Popitis. [So what are you gonna do then?] Well, it helps. It can't hurt. So then ... I mean You obviously look into it to see if there's globoma, and you try to take other tests whatever. [No, that's the only test you can do.] That's the only test you can do. Then you're in a difficult situation."

Another ET scan response

[after some initial discussion]. "I'd go ahead and do the ET scan if it were not expensive. [What if it cost $5?] Then I would do it. [Okay. Why?] I understand that ... Okay, a person coming in has a 20% chance of having popitis or flapemia, right? [Uh huh.] Now, if I couldn't do any other test, I would like to rule out as much of that 20% as I could. If I ruled out some of that, then the 80% would probably increase, right? So if I could rule out popitis... [Why would you want the 80% to increase?] I guess to increase the probability of the patient having that disease. [Okay.] If I understand this correctly, popitis and flapemia both present similarly. [All three present similarly, and the reason you think it's 80% is that globoma is just a lot more common.] So I could do the ET scan and rule out popitis, which, as you say, presents very similarly to globoma. I would have narrowed down my choices to globoma or flapemia."

Z ray response

[after discussion]. "Okay then my first choice would be to definitely do the Z-ray test. [How come?] Because of its minimal risk or cost to the patient, and ... its answer will help me decide whether or not to treat the patient for this disease. [Okay, how will it help you decide?] Well if the result is positive, then its obvious, then the patient has the disease and I treat them for it. However, if it's negative, then the probability is still greater that the person does have the disease, and I think that I would have to ... I would have to go with the probability. [So what would you do.] I would treat for the disease. [Okay. So why would you give the test, or would you?] Why would I give the test? [Urn hm.] I would give the test simply to help me decide, because the test can tell me one choice, one way if not the other. [How will it help you ...] I see, I see. If I give this test... Whether or not I give this test, according to what I just said, either way I would be treating for this disease. So, it ... yeah ... that makes the test kind of ineffectual, it makes it irrelevant."

Another Z ray response

"Yes, I think you definitely should do it, because ... it's gonna give you a conclusive result in 80%, and ... yeh ... you definitely should do it. [Okay, what does it mean to you to say that the two different kinds of mistakes are equally bad? Is that relevant?] I don't think so. [Okay, what would you do if the test is negative, and the probability is now .6?] The odds are still in your favor to give the treatment. [Now does that affect whether you should give the test?] Okay. Yeh. [So you wouldn't give it.] No. [Was there something you misunderstood that made you not see that the first time? or something you didn't look at?] Yeh, I didn't put the two together. No matter what, you're gonna do the treatment, and I took them apart, and I said, you do this test, then you definitely know, and that's ... definitely do it. But you're probably gonna do it anyway. The reason I said definitely do it is because it gives you a concrete positive result, but I didn't realize you're gonna probably do it anyway ....."