Markov chains are super useful in statistics but it isn't obvious at first what problem they solve and how - some further reading that I found helpful:

https://twiecki.io/blog/2015/11/10/mcmc-sampling/

Note that the point of the markov chain is it's possible to compute relative probabilities between two given points in the posterior even when you don't have a closed form expression for the posterior.

Also, the reason behind separating the proposal distribution and the acceptance probability is that it's a convenient method to make the Markov process stationary, which isn't true in general. (Wikipedia page on MCMC is also useful here).

This is timely! I have an assignment on these coming up soon. Can anyone with knowledge about this explain something. From what I can tell, many matrix multiplications move vectors so they are more inline with eigenvectors if they exist. So Markov Chains are just a continual movement in this direction. Some examples that don't do this that I can think of are the Identity matrix and rotations.. Is there a way to test if a matrix will have this effect? Is it just testing for existence of eigenvectors?

What is the secret sauce that makes LLM better than a Markov chain?

previuos submissions

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

The relevance to me is that markov chains are a remarkable way to explain why LLMs are both useful and very unreliable.

You train on piece of text and then the output 'sounds' like that text it was trained despite being pure gibberish.

Past comments: https://news.ycombinator.com/item?id=17766358

markov chains are used for my favourite financial algorithm; the allocation of overhead costs in cost accounting. wish there was an easy way to visualise a model with 500 nodes