Wikipedia talk:Forum for Encyclopedic Standards/Archive 3

Archive 1

Archive 2

Archive 3

Archive 4

Archive 5

Archive 6

comments on "peer review" and PhD.s

I have a major problem with much of the discussion. It hinges on this: the peer-review process used by major journals, relying on PhD.s, is based on the notion of journals as publishing original research. Wikipedia is not a place for original research. Therefore the PhD. peer-review process is not a great model. I thiknk we need to be very careful about what peer review does and does not accomplish. But more importantly, I think we need to work through what is wrong with Wikipedia and what we hope to accomplish first, before deciding how to do it. For example, I don't care whether wikipedians editing my work are PhD.s or not (and I personally don't want to have to submit a CV for any reason to Wikipedia. I do not think it matters whether wikipedia articles are written by PhD.s or not, and I do not think it matters that wikipedia articles have been approved by PhD.s or not. What is important to me is that Wikipedia articles be based on serious research, which often involves reading articles and books by PhDs, or that have gone through peer review. I continue to like Wikipedia's somewhat anarchic and egalitarian culture. I don't want that to change. I just want editors to learn to rely on books in libraries, and books by serious scholars, over web-sites. I do believe this requires some policing and education -- but I do not think the journal system of peer review is either necessary or best-suited to these goals. Slrubenstein

Damn straight. I agree completely. john k 18:33, 17 Nov 2004 (UTC)

I'm not advocating a journal system of peer review or a system of article approval, but rather setting up an alternative to the Arbitration Committee to settle disputes based on editorial considerations as opposed to just process. This would not be any more of a change to Wiki's anarchic structure than the emergence of the conflict resolution process; this would just rationalize the way disputes are settled... Would you and Slrubenstein favor a more limited proposal along these lines? 172 20:46, 17 Nov 2004 (UTC)

I like the idea but still have mixed feelings. The arbitration committee's findings are (relatively) final -- I am not thrilled by that but in such an anarchic community as this, I see the need and good of that. Would this committee you suggest make "final" rulings? That's something I feel much less sure of. I understand the need for it (e.g. in my problem with CheeseDream the mediators and AC aren't supposed to jusge content). But what I love about Wikipedia is that the process truly matches the scientific value that the truth can never be known in its entirety, that knowledge is the product of an ongoing process of debate, speculation, testing, and re-considering. I wouldn't want to sacrifice this either. Belive me, I see the merit in your specific proposal. But I see that merit because I see a serious problem. When I reject "peer-rview" it is not to turn your proposal into a straw-man, it is merely to open up other possibilities. Slrubenstein

I don't expect editorial arbitration to stifle the process of open debate and reconsideration that makes producing articles on Wiki unique. I'd expect it to intervene in more clear cut areas, such standardizing German-Polish nomenclature, and technical terminology; and settling disputes of factual accuracy... I do understand your reservations, though. If this proposal is to go anywhere, I suppose that it'll have to come along with suggestions for limiting its jurisdiction. 172 21:11, 17 Nov 2004 (UTC)

I agree with much but not all of this. That is, casually produced material usually isn't the best source. But different sources, including personal knowledge and experience, are useful for different topics. For example, if I write on topics my experience that are not documented elsewhere, but not disputed by people of similar experience, I don't see that information as problematic. Maurreen 18:42, 17 Nov 2004 (UTC)

I am afraid you would have to be clearer/more specific about the content and context. Wikipedia is simply not a place for original research, or personal essays. Look, if you are part of the crew of a sailboat and know everything about how to rig a sloop or something, I suppose you can write an artricle on how to rig a sloop without citing other books as sources (although honestly, it might help readers if you cite books they can turn to for more information). I suppose I can think of other articles where your own experience may be enough. But I know oh so many cases where this is simply inappropriate. Slrubenstein

Can you give a few examples? In fact, I think you raise a point that I believe should be the work of people involved in this project. Rather than peer-review articles, I'd rather see this group as a kind of think-tank to address thorny issues in enforcing "verifiability" and other related standards. Maybe you could offer ten examples that represent a diverse range of kinds of articles, and we can discuss which one of them fails (is original research, personal essay, unverifiable) and which ones succede, or which ones are flawed but can be salvaged by real research. If we can come up with more specific rules of thumb on just this issue, I think we would be serving Wikipedia very well indeed! Slrubenstein

I think it's a bit of an effort/benefit tradeoff. If the article is non-contentious (ie nobody is contesting its accuracy), is the lack of sources that big a deal? I would say it's not. Sure, someone could spend hours tracking down an exact source for every single little claim. Or they could spend those hours writing another dozen articles. I know which one I think is a better use of time. Citing sources is important when it comes to articles that people can't agree on, but for the rest (the great majority of articles on wikipedia, I might add), it's probably not worth the time it takes for the benefit it gives.

Plus I agree it's often very difficult to find sources for a lot of things. For example, finding detailed information for albums released before the age of the internet is often impossible unless you happen to own a huge stack of music magazines. This leaves you with very little worthwhile you can put in such an article, unless you're willing to wing the sources requirement and write at least partially from personal knowledge or potentially unreliable (read random website) sources.

Which brings me to another point: random websites. Although not always reliable for factual information, I think they are a good way to measure some degree of "public opinion" in certain cases. We're aiming to capture the sum of all knowledge, and one of the wonders of the internet is that there is more variety human knowledge exposed than there ever has been before. We shouldn't discount it out of hand, it is a useful resource if you're willing to acknowledge the limitations. Shane King 07:01, Nov 18, 2004 (UTC)

Sourcing options

One example of using a nonpublished source: I used to work at HMX-1, a U.S. Marine helicopter squadron. I added the following to the article: "The presidential and VIP flights fall under 'whiteside.' 'Greenside' operations include support of Marine Corps Combat Development and operational test and evaluation, such as with the V-22 Osprey, a vertical take-off and landing aircraft."

If that information is not published anywhere, is somebody going to tell me I shouldn't have added it? Maurreen 07:58, 18 Nov 2004 (UTC)

The question is if it really was unpublished. Was this a standing order, a military operational procedure, etc., all of which are written down someplace. Most of these actually have a source, but you may need to go through extensive hoops to track down a reference. In the meantime, if the element is particularly relevant, a footnote of the timeframe you can personally report from and "reported by Name on date" is a method of citing yourself. This citation is, of course, ripe for abuse and is generally poor practice. - Amgine 17:13, 18 Nov 2004 (UTC)

"is somebody going to tell me I shouldn't have added it?" Yes, you shouldn't have added it. If it's not published, how is anyone else going to verify it? What if someone maliciously interchanges "whiteside" and "greenside" with the edit summary "Maurreen is ignorant"? How can anybody adjudicate that kind of a dispute? I personally accumulated lots of juicy stories about the GNU project, but won't put any of it in WP until it's been published elsewhere. Stan 05:13, 19 Nov 2004 (UTC)

It seems to me that a Talk: page would be the right place for this - write an entry of the form (in this case) "I worked in this unit during xxx-yyy and this is what I saw".

It's nice to have the ideal of always going back to something written, but alas, not everything worth covering in an encyclopaedia always gets written down. E.g. one I have personal experience with is very early Internet history (late 1970s); a lot of it didn't get written down, or written versions (e.g. old email from the late 70's) is now lost. For this reason you often see books (e.g. Hafner's book on the ARPANET) which rely heavily on interviews - and not just technology fields, anything where the participants are still alive to give "oral history". Nobody is saying "don't use this book as a source because it relies on interviews", that would be silly. Why do we have to have a higher standard than that? Personal memories of participants are an accepted source in academic circles. Yes, I know it's gone through an extra step of review in being published, but at the same time, fundamentally the origin of the data is still human memories.

Also, not everything worth covering is going to get turned into a book. Do we only cover topics that got turned into books? Yes, it can get close to the line of no original research, but if you read that page, it's actually talking about e.g. new scientific theories, not documenting history. -- Noel (talk) 14:36, 19 Nov 2004 (UTC)

I followed Noel's advice. Thanks. Maurreen 16:56, 19 Nov 2004 (UTC)

The book that publishes the interviews is the original research; it's exactly the sort of thing that history professors do to get tenure. The step of review and publication is to a degree artificial, but it's the only thing that stands between WP and the armies of kooks who would like to put all their mumblings online here. In practice, we do accept plausible-sounding websites in addition to books and news articles, although I look for some kind of credential, such as a real name that is attested elsewhere as an expert in the topic, etc. See primary source, etc, for our own material on all this (which I note is itself unsourced, tsk tsk :-) ). Stan 20:22, 19 Nov 2004 (UTC)

This is great

I've been pushing a lot lately to help get the articles cited better with sources. I especially feel all featured articles should have good references. You may be surprised to find out that many, if not most, featured articles currently have no proper references. I am making a list of them and will post that somewhere soon. I feel all wikipedia articles should be properly researched and cited, so the featured articles are a good place to start. See the ongoing discussion at Wikipedia_talk:Featured_article_removal_candidates. (Currently I seem to be losing :) - Taxman 18:49, Nov 17, 2004 (UTC)

Well, I agree we need better references/sourcing/etc! Noel (talk) 16:35, 19 Nov 2004 (UTC)

Responding to Neutrality's suggested template

A couple days ago Neutrality posted a template that warns the reader that an article has no sources. I like this idea very much--it really should be a no-no to post stuff on the Wikipedia that isn't backed by sources, and this template might give people the right idea. I have two little suggestions and one bigger one:

• No need to say "this important article"--all articles should have sources
• No need to mention this Committee; all can source
• Figure out some way to gain prior community assent before posting this template all over the place, otherwise people will feel bullied.

Opus33 01:54, 18 Nov 2004 (UTC)

Isn't it usually kind of obvious when an article entirely lacks sources? I think that's why people would feel bullied - the reaction would be "well, duh". Also, something painfully obvious, like the title of a album with the scan of its cover right there, is sort of "self-sourcing", if you will. I think a better way to phrase would simply be "please add more references", because many article have one or two dead links as references :-), and we want for those is "more and better". Stan 04:54, 18 Nov 2004 (UTC)

The article isn't meant to be put all over the wiki. It's only meant for very important articles that don't have sources, such as England and Blackshirts. [[User:Neutrality|Neutrality (hopefully!)]] 05:01, Nov 18, 2004 (UTC)

I think the template has potential but it's probably premature right now. Maurreen 07:44, 18 Nov 2004 (UTC)

Based on discussion on the Village Pump when WP:Bias tried adding templates to articles about being (for example) overly US-centric, something like this will be a lot more welcome on talk pages than in the articles themselves. -- Jmabel | Talk 07:59, Nov 18, 2004 (UTC)

Yes, the Talk: page is the right place for it. That's where the editors communicate about the article, and it is the editors who need to know that it needs sources. The only reason to put it on the article would be to warn readers - and we already have a number of standard warnings that would be more appropriate if there is any concern about the contents. (Not to mention that until we get some sort of "known good version" system working, any page could always contain vandalism anyway...) Noel (talk) 14:44, 19 Nov 2004 (UTC)

Is it time for a cabal?

In a sense, what is being proposed here -- and I don't necessarily think it is a bad idea -- is to actually form the cabal some of us are often accused of belonging to. However, unlike your classic cabal, I recommend that if we do this we should be very careful to operate openly, and in particular if a group of us operating as a team take on particular articles, we should be very clear that we are operating as a team, and we should be systematic about having our communications about that article in an open forum such as a talk page, not by private back channels like email. -- Jmabel | Talk 08:07, Nov 18, 2004 (UTC)

Oooh! I've never been in a cabal before. How exciting! :) Maurreen 08:31, 18 Nov 2004 (UTC)

I don't agree about forming a cabal, unless you mean something like a common interest group with open membership. Slrubenstein

Exactly. I was just joking about how many of the people who are often accused of being a "cabal" have signed on to this already. -- Jmabel | Talk 21:52, Nov 18, 2004 (UTC)

My apologies. All my good humor is being sapped on another page. However, it seems that after a period of growth Wikipedia is at a time of transition and I think now more than ever we need to be careful to make clear to everyone what some basic principles are, including open membership and (as I know you did take pains to emphasize) transparency. Slrubenstein

Possible test cases

I'd like to suggest that rather than indulge at great length in hypotheticals, we should set up some (self-organizing) teams of about four people; each team would take on a few articles and try to bring them up to what we consider acceptable standards, then look at each other's work and see what we can learn from it.

I believe that appropriate candidates for these first experiments would be articles that are believed to be at least generally accurate, are not subjects of current active controversies, but are currently under-sourced. We should certainly try for articles in different subject-matter areas (because many of the teams would presumably organize by subject-matter area), and within each should probably try for articles of various lengths.

I have some more concrete ideas along these lines, but before I take time to detail this, I want to make sure that at least a few people think this is an interesting enough suggestion to be worth my fleshing it out. -- Jmabel | Talk 08:18, Nov 18, 2004 (UTC)

Taxman is compiling this list of featured articles that are insufficiently referenced. Would this list form a suitable test corpus? There are featured articles on many different topics, and it would be good to make sure that they all have proper references. -- ALoan (Talk) 14:50, 18 Nov 2004 (UTC)

Might be a good place to start (although I wonder at the notion that it is a problem, for example, that Leet gives only online references). I'd also like to add to the pot:

1. A few articles that look basically on the mark, but totally (or almost totally) lack references of any sort. For example, I just translated the bulk of History of Catalonia from the Spanish, which gave no references.
2. A few article from popular-culture areas -- especially foreign-language popular-culture areas -- which I think are going to be some of the toughest things to get non-web references for. For example, to name two major Romanian bands, where are we going to find references, especially academic references, for gangster rappers B.U.G. Mafia or for Spitalul de Urgenţă (sort of a Romanian Pogues)? My suspeicion is that we are going to have to have different standards in these areas, and we might as well know it up front. -- Jmabel | Talk 19:09, Nov 18, 2004 (UTC)

Same players, different roles

Trying to raise an article to encyclopedic standards is going to involve taking on a "role" for a period of time, at least with respect to a particular article, that may be distinct from other roles one plays in editing Wikipedia. I'm sure that most, or all, of us involved in this conversation are serious about turning Wikipedia into a better reference work; probably most, or all, of us feel we are doing that by most of our edits.

Still, I think this is something distinct. It requires (for example) willingness to acknowledge the inconvenient fact as readily as the convenient one, and to strengthen the citations for even positions we disagree with (something that few Wikipedians normally do: normally we rely on a mildly adversarial system to solve that). It almost certainly is not a role that any of us will easily take in an area where we have strong feelings. For example, I was recently chewed out by someone -- not a very likable someone, but not a fool -- for editing the article about the college I attended as an undergrad. Frankly, if people didn't ever write about the college they went to, I suspect we'd have rather few articles about colleges, so I'm unapologetic, but I would not be the best person to judge that article dispassionately. Similarly, with all due respect, I would doubt that Jayjg or Xed, who I think are generally good contributors to Wikipedia, could take a dispassionate approach to the article Arab-Israeli Conflict. Their roles there are already staked out. I'm sure there are other areas where they can be much more objective. (Jayjg, Xed, I hope you don't feel I'm singling you out in a negative way: I am trying to pick an example from people who are already signed up on this project.) -- Jmabel | Talk 08:40, Nov 18, 2004 (UTC)

Hmm. While my approach may not be dispassionate, my edits are NPOV. Is dispassion required? Jayjg 23:17, 18 Nov 2004 (UTC)

For this task, probably so. And I agree that you try to be NPOV. Again, I think it's fine that you edit those articles, but I think you are too engaged to be an even-handed arbiter, which is what we seem to be talking about here. I believe that you are too engaged to give the same scrutiny to a new addition that tends to be favorable toward Israel that you give to one that tends to be critical of it. And, again, I didn't particularly mean to single you out, I was just trying to pick an example I was familiar with pertaining to people within this project, because I felt that the example I could point to about myself was too narrow to illustrate the problem. I'm sure someone else could identify a broader area where I tend to be other than evenhanded. I think the nature of this is that one tends to be unaware when one is doing it, and it's more obvious to others than to oneself. -- Jmabel | Talk 23:43, Nov 18, 2004 (UTC)

An example of one of Jayjg's 'NPOV' edits "The Jews against Occupation spokesman went on to rant about Israel": [1]. - Xed 23:54, 18 Nov 2004 (UTC)

Oh, that was just a tiny joke during an edit war. I knew you would revert it, that was during your "stalk Jayjg's edits, and whenever he's in a dispute with anyone, revert him" phase. And you did revert it, four minutes later. Sheesh, where is everyone's sense of humour? Jayjg 00:06, 19 Nov 2004 (UTC)

Clearly you find vandalizing pages hilarious. So hilarious that you reverted it right back to "The Jews against Occupation spokesman went on to rant about Israel". - Xed 00:35, 19 Nov 2004 (UTC)

Oh Xed, you laughed yourself. Chill out. Jayjg 02:55, 19 Nov 2004 (UTC)

Deletion

Because it is un-Wikilike and does not appear to have consensus, I am deleting the following paragraph. Maurreen 08:45, 18 Nov 2004 (UTC)

We believe that the objective of Wikpedia must be to produce a viable online sourcebook worthy someday of a citation in a scholarly journal. We are calling on organizations governing Wikipedia (such as the Wikimedia Foundation and the Wikipedia:Arbitration Committee) to adopt policies that cease privileging process over product and cease privileging editors over readers. We are calling on Wikipedia to adopt a structure that protects serious editors and prevents problem users from sabotaging their work. In practice this means that Wikipedia's structure must change so that certain subjects have some kind of editorial oversight by a review panel of expert writers.

I'm with Maurreen here. On the other hand, I'd love to see a process to identify and make available "released" versions of articles, which have been through a review process, just like happens with open-source software. -- Jmabel | Talk 08:54, Nov 18, 2004 (UTC)

I agree with much of the paragraph. I don't know how realistic it is to expect Wikipedia articles in general to be worthy of citation in a scholarly journal (though there's no reason individual articles should never be worthy) but I strongly agree that serious editors should be protected from users sabotaging their work. I feel we do need a panel of expert writers who can be called upon in case of disagreement, and who will be in a position to judge the quality of the articles, not just mediate between the different personalities. Slim 10:06, Nov 18, 2004 (UTC)

I would be ecstatic if a scholar just used WP to get background on an area tangent to his/her specialty, then read and cited the references found in those articles. Stan 05:20, 19 Nov 2004 (UTC)

While I don't agree with each and every sentence/idea in the deleted para, I'm in sync with the general concepts, and with most of it. Wikipedia needs to be something that people can rely on; we need to recognize that content is more imporant (in the end) than even-handedness, and that not all editors of an article have equal levels of knowledge/etc about a subject, and that really knowledgeable contributors are valuable and we need to keep them; and that to reach these two goals will probably require some changes, which will inevitably have to be rooted in the Wikimedia foundation. So I wish we'd try to fix any problems with that para, rather than simply deleting it. Noel (talk) 16:43, 19 Nov 2004 (UTC)

I interpret the paragraph as seeking to change the nature of Wikipedia. An add-on system could be acceptable. Maurreen 17:09, 19 Nov 2004 (UTC)

Purpose or mission statement

Do we want to develop a consensus on a purpose/mission statement like the above? I think most of us are interested in developing Wikipedia into a respected resource worthy of citation at the personal, business, and educational levels, although I do not think there is consensus for developing Wikipedia into an academic research resource tool except for background material. It's the elements regarding when and how an article is protected from change which are in question here, and not the end results of this (desired/opposed) process.

I think it would be good for the group to develop such a statement. But I don't see any consensus on protecting articles. Maurreen 18:03, 18 Nov 2004 (UTC)

Foreign-language wikipedias

Is anyone aware of good work in these directions in Wikipedias in other languages? In my experience, I've encountered higher standards on sourcing and fact-checking in the German-language Wikipedia and a level at least comparable to ours in the French-language Wikipedia; in others, my experiences have varied and sometimes been disappointing. I've had people view it as an insult when I ask for their sources. -- Jmabel | Talk 08:51, Nov 18, 2004 (UTC)