Yes there are more complex probabilities that you can calculate. For example you can calculate the probability of reaching some state that is one or more transitions away given a starting state. You can find more info on some of other calculations you can do on Wikipedia.
That is a great intro. Thanks for putting it together. If you were to continue this to a Part #2, I would be interested in seeing the approach for determining if something was "abnormal". Is it considered abnormal to add a node to the chain, or to add a transition? Is detecting abnormal behavior more complicated than detecting new nodes or transitions?
Abnormal really means that it has a probability less than some threshold. So you might have a transition that has a probability of less then 0.1, depending on your situation, you might consider that abnormal even thought that transition does exist. Also a non existent transition would be considered "abnormal" because the probability is 0 in your model.
I've been thinking about doing a part 2 with text analysis and a cool use case would be to detect "abnormal" text from an author because he/she uses strings of words that have a low probability in the rest of their text.
11 comments
[ 3.3 ms ] story [ 34.5 ms ] threadAre there different or more efficient probability calculations that can be done other than the provided algorithm? It seems somewhat simple.
I've been thinking about doing a part 2 with text analysis and a cool use case would be to detect "abnormal" text from an author because he/she uses strings of words that have a low probability in the rest of their text.
Hope this helps and I'm glad you like the post.
"Markov chains explained visually": http://setosa.io/blog/2014/07/26/markov-chains/