Talk:Markov chain

Awful article

Scientific articles are Wikipedia's Achilles heel, but this is truly terrible. I'm trying to understand section A non-Markov example and I'm giving up. That's just not the way to do it. The counter-example should be simple and clear, not so complicated and obscure. And if you do such complicated examples, you need to add drawings.

It is also questionable description in Applications. Statistics? It is all statistics. Economics and finance? Ok, but next Social sciences includes economics. Finance could be separated from economics and then social sciences combined with economics. Pawel.jamiolkowski (talk) 18:32, 9 June 2024 (UTC)

Hi @Pawel.jamiolkowski,

For your feedback to be useful, you have to be more specific. Is your problem with the stochastic process itself, or with the way it is presented? How do you think that a drawing could help here?

Regarding your comment on the section Applications: listing "statistics" is perfectly relevant; please have a look into the difference between probability theory and statistics to see why the study of Markov chains is not a subfield of statistics. I think grouping economics and finance is also relevant here, but I agree that something more specific than social science could be used (especially considering that the meaning of the word is a bit vague and has changed over time). However to me these are pretty minor issues.

Malparti (talk) 08:31, 10 June 2024 (UTC)

First of all, A non-Markov example is the sub-section of Examples, right? Reader reads examples which are brief description without details. And it's okay. And then we asking: Ok, so what process won't be Markov chain? The reply is below. However instead seeing a similarly short description, we see loooong piece of writing.

It should be consistent with the previous part. Short decription for intuitive understanding.

I don't know what drawings could be included here, because I don't understand this.

And secondly, why is a separate article with the same title? I guess to deepen the topic. There should actually be such a counter-example there (although better written anyway - first an intuitive description, then a deeper explanation). Pawel.jamiolkowski (talk) 13:49, 10 June 2024 (UTC)

Hi again @Pawel.jamiolkowski,

The subsection is actually rather short; what makes it look long is in part the explanation of how to build a Markov chain out of this non-Markovian process (which is somewhat irrelevant — here we want to understand what makes a process non-Markovian; not how to make non-Markovian processes Markovian by augmenting the state-space...). My guess is that this is a recent addition; articles on basic and widely used math tools have a natural tendency to grow because they attract a lot of readers from different backgrounds and many people want to add something — so they have to be trimmed down once in a while. I will probably remove the last paragraph of the subsection, but there are other problems I would like to fix first.

Another problem might be the use of the terms "dime" and "nickel" and "quarter", which lengthens the text and make is look more complex than it is to non-American readers. I guess that could be improved, too.

Other than that, I think this example is about as simple at it can get without getting something entirely trivial, and I think the level of detail is more-or-less OK. So I think it would be helpful if you could invest time to understand it and then explain what you found hard to understand, and how the description could be improved.

Best, Malparti (talk) 22:27, 10 June 2024 (UTC)

You have almost the same example, but simple and clear here: Markov property

Do you see? It should look like that. 3 balls, colors etc.

But generally for grasping idea, at the beginning it should just explain that the essence of difference lies in difference between draw with replacement and draw without it. It's obvious that if we have constant number of objects and draw without replacement, the future depends on draw, because each draw corresponds with each ball / coin, 1:1. As the number of objects decreases, it is obvious that the probability of guessing increases. Everyone can understand it without any problem. Pawel.jamiolkowski (talk) 23:47, 10 June 2024 (UTC)

I agree that this example has some advantages; but I disagree that it is strictly better, at least in the way it is presented. For instance, it is not explicitly stated what is the process of interest. This is not ideal, especially since if we were to model everything that is described verbally, we would of course get a Markovian process. But I agree that something like "the color of the n-th draw in a Polya urn process" seems like a more natural counter-example than the example described in the article. I will try to take care of this over the weekend. Malparti (talk) 13:32, 12 June 2024 (UTC)

Unnecessary citations

Just dropping in to say that a book being part of ChatGPT['s] training set [1] is the worst argument for citing a reference that I have seen in weeks.

Also, the opening line of the "Applications" section is a summary of the text that follows and doesn't need a citation itself. XOR'easter (talk) 20:47, 12 June 2024 (UTC)

@XOR'easter: Indeed... So it seems my concerns were not unfounded. ;). It also seems some people know how to use different IPs.

If that's OK with you I've moved your comment to a new section, to keep it separated from the discussion with Pawel.jamiolkowski.

Best, Malparti (talk) 23:05, 12 June 2024 (UTC)

Semi-protected edit request on 13 June 2024

HI,the page was bloked, however, some users abused the system and erased Gagniucs book with no reason. I wish to insert Gagniucs book again, and the main reason is the fact that is the most academically cited book listed on this page. More ... the suspicious reason for wich a top book was erased on no grounds at all except replacement with other unknown sources that have no academic grounds to be cited. 213.233.108.161 (talk) 20:40, 13 June 2024 (UTC)

Declined. There was no "abuse" of "the system". Gagniuc's book was removed for multiple reasons, any one of which would have been sufficient by itself. "Most academically cited book listed on this page" is a meaningless standard, particularly since the book could well have benefited from having been advertised here for so long. XOR'easter (talk) 20:51, 13 June 2024 (UTC)

Gagniuc's book coasted on the free publicity of being prominently cited on Wikipedia for a long time, but the edit-warring to keep it there brought it to the attention of people whose idea of a good time is editing mathematics articles on Wikipedia. The result: despite the book being called "from theory to implementation", there is not an ounce of theory — most of the basic concepts are not presented. The book is full of approximations, and the way things are written gives the impression that the author does not understand the basics... Not to mention the >100 pages of computer code which I doubt anyone is ever going to read (it is even hard for me to comprehend how Wiley could agree to print something like this in 2017). And likewise: 60 pages of badly-worded boring worked examples with no theory before we even get to the possibility of having more than two states. As Malparti said, there is no theory, or rather theory is alluded to in vague and inaccurate form without any justification. For instance the steady state (still of a two-state chain) is first mentioned on 46 as "the unique solution" to an equilibrium equation, and is stated to be "eventually achieved", with no discussion of exceptional cases where the solution is not unique or not reached in the limit, and no discussion of the fact that it is never actually achieved, only found in the limit. Do not use for anything. I should have taken the fact that I could not find a review even on MR and zbl as a warning. Fuller discussion can be found at the WikiProject Mathematics talk page (permalink). XOR'easter (talk) 18:18, 30 June 2024 (UTC)

Problems with Ergodicity section.

Firstly, there is a paragraph "If all states in an irreducible Markov chain are ergodic, then the chain is said to be ergodic. Equivalently, there exists some integer $k$ such that all entries of $M^{k}$ are positive."

My question is: what is $M$?

Previously in the article the letter $M$ is used only for mean hitting time $M_{i}$ of state $i$. I suspect it was meant to use stochastic matrix $\mathbf {P} ^{k}$.

Secondly, the sentence "More generally, a Markov chain is ergodic if there is a number N such that any state can be reached from any other state in any number of steps less or equal to a number N." seems wrong. The part "any number of steps less or equal to N" implies that all states can be reached in 1 step (as "1" is "any number less than N"). This is not true for all the ergodic Markov chains. Removing final "any" should fix the statement.

Still, the statement doesn't align with the previously given definition of ergodicity as it doesn't exclude Markov chains with periodic components. 2001:BB8:2002:98:7987:3E47:DD3E:2E0A (talk) 09:03, 13 September 2024 (UTC)