Mobile blogging

Okay! Now that I have an application on my android phone for blogging, there are no excuse for not updating this blog anymore!

ha...this is funny

http://astro.berkeley.edu/~mwhite/referee_funny.html

Probability as a Linking Function for Love

The mainstream thought in current psycholinguistics studies, or almost all computational studies in cognitive science, assumes that probability is an explanatory factor in cognition. Probability, estimated by various methods, can predict learning of word segmentation, speech duration, omission of phonetic details, and so on.

Recently I've become increasingly reluctant to attribute that much explanatory power to probability alone. A probability value is essentially just a point estimate. To make clear what I mean by this, let's consider an example of trigrams. Suppose we have learned in a fictitious world the love story between Romeo and Juliet is planned by God in the following way.

Romeo loves Juliet   3

Romeo loves Jane    2

Juliet loves Romeo   3

Juliet loves Andy    1

Juliet loves Jack   1

The numbers in the second column above indicate how many times the lover has said she loves the person being mentioned as the lovee. And we further assume in this world, nobody lies about romantic feelings, and that a person's affection for another is positively proportional to the number of times she indicated the feeling.

Then, we could ask, what is the probability that Romeo loves Juliet? To answer this question, we can look at the probability of that situation. First, we shall see there are a total of 5 instances of love expressions for Romeo. Second, the number of instances with Juliet is 3. By simple statistics, the reader can easily determine the probability of interest - 3/ 5 = 0.6.

For the probability of "romeo" preceded by "juliet likes", one could similarly find the answer to be 0.6.

So a rare occasion in love stories: Romeo loves Juliet just as much as Juliet loves Romeo. Good, no one gets hurt.

But let's pay a bit closer attention to the scenario. The probabilities of these two events are estimated from two different distributions (in each of which the probabilities of all events sum up to 1). Does 0.6 mean the same thing in both distributions? Or in other words, does an event of probability 0.6 tell you equally as much information in the first distribution as in the second?

No.

Yes?

Well. It is a matter of perspective. However, I am going to argue that saying "no" is better than the other. 

Let's look at the problem in terms of information theory. The first distribution has an entropy of 0.97 bits, and the second one has 1.37 bits. The information theory (communication theory) points out entropy is an indicator of choices - higher entropy corresponds to more choices. In other words, Juliet has more choices than Romeo in our hypothetical world. How much more? Juliet has 2.58 choices, and Romeo has only 1.97 choices.

This gets interesting. When Juliet apparently has more choices than Romeo, she loves him just as probably as he loves her. Does this make Juliet's love more valuable?

In a sense, yes. By how much? 0.61 choices, whatever that means.

Now let's look at another quantity. How many choices does each of them have besides the primary one (Juliet for Romeo, and Romeo for Juliet)? For Romeo, there's only one alternative left - Jane, which would have a probability of 1 if Juliet were not in the picture. Thus, the entropy of the alternatives is 0 for Romeo. For Juliet, the entropy of the alternatives is 1 bits.

This is even more interesting. If Romeo does not love Juliet, then he literally has no other choices but Jane, which makes the choice meaningless (choice with one option isn't really a choice). However, if Romeo is not an option for Juliet, she is better off by 1 bits, which are 2 choices.

Juliet is then doing a huge favor to Romeo by loving him with a chance of 60%! Because if they are mutually refusing to be in a relationship, Juliet is in a situation way better than Romeo's.

So, in general, the message to take away from this silly example is, Romeo should feel fortunate if Juliet ends up loving him, because her 60% is more valuable than his 60%.

Ok, seriously. 

Updates

It has been a while since I last updated this blog. But a random narcissist search of my name in google shows this blog has been indexed. Well...that's some motivation for a new post.

The first semester of my PhD life is done, which is relatively enjoyable. However, I am now more inclined to leave academia once I am done with graduate studies. I've always felt that professors are a noble job, and are people who contribute to the society significantly. Yet this contribute is rather indirect. And perhaps due to this indirect influence, many research projects are conducted with no specific intention of making social impact. I am rather dissatisfied with that.

The important reason why everyone should feel good about themselves

here is the reason:

http://www.scientificamerican.com/article.cfm?id=the-psychology-of-social

Bayesian Calculator

Aha! I've been looking for this kind of stuff for a long time. Glad someone made it:

http://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm

Infinity in Python

There are only two constants in the Python math module: pi and e. However, it is often useful to have positive and negative infinities as well. After a bit googling, there is a rather interesting way to get these values:

For positive infinity, just do:

float('inf')


For negative infinity, guess what -- float('-inf')


Somehow I find it very amusing.