First published on ethnographymatters.net.

Last month on Ethnography Matters, we started a monthly thematic focus where each of the EM contributing editors would elicit posts about a particular theme. I kicked us off with the theme entitled ‘The Openness Edition’ where we investigated what openness means for the ethnographic community. I ended up editing some wonderful posts on the topic of openness last month – from Rachelle Annechino’s great post questioning what “informed consent” means in health research, to Jenna Burrell’s post about openaccess journals related to ethnography and Sarah Kendzior’s stimulating piece about by legitimacy and place of Internet research by anthropologists. We also had two really wonderful pieces sharing methods for more open, transparent research by Juliano Spyer (YouTube “video tags” as an open survey tool) and by Jeff Hall, Elizabeth Gin and An Xiao in their inspiring piece about how they facilitated story-building exercises with Homeless Youth in Boyle Heights (complete with PDF instructions!) Below is the editorial that I wrote at the beginning of the month where I try to tease out some of the complexities of my own relationship with the open access/open content movement. Comments welcome!

On Saturday the 12th of January, almost a month ago, I woke to news of Aaron Swartz’s death the previous day. In the days that followed, I experienced the mixed emotions that accompany such horrific moments: sadness for him and the pain he must have gone through in struggling with depression and anxiety, anger at those who had waged an exaggerated legal campaign against him, uncertainty as I posted about his death on Facebook and felt like I was trying to claim some part of him and his story, and finally resolution that I needed to clarify my own policy on open access.

I had worked passionately for open access in my previous life, helping educational institutions and foundations design open access policy, pushing for open government data and railing against those who didn’t ‘get’ why closing access to publicly-funded information was outdated and unsustainable. But nearing the end of my work with Creative Commons and its international offshoot, iCommons, I became jaded by the internal politics of the open content movement, and embarrassed by my previous zealousness. I started to realize that open access was definitely not revolutionising access to education in the majority of the world, and that the passion that myself and others had felt about pushing forward the openness agenda was becoming sinister as any criticism was met with aggressive denial, as definitions of openness became ever narrower and technologically defined, and as we seemed to get further and further from the goals that we started with.

In the wake of Aaron’s death, and the renewed calls by the open access community for academics to take a stand, I felt that I needed to resolve these feelings and to define my own perspective on the issue. Thinking about the openness of your research can be like going down a rabbit hole because if you’re attempting maximum accessibility for all people at all times, any open access policy looks incomplete. Open access definitions tend to be restricted to a particular medium (digital, online) and a particular definition of free (free of charge and free from most copyright licensing conditions) (see Peter Suber’s great introduction to open access here).But access, especially when it concerns the study of people and communities, needs to follow a multiply layered, contextual approach if it is to fulfill the ultimate goals of an open research process. What about accessibility standards for differently abled? What about the fact that Arabic speakers can’t read your final report about them? What about the fact that not everyone can understand academic lingo or appreciate layperson explanations? Researchers’ open access policies don’t generally cover these equally valid issues that necessarily close your work off to particular audiences.

When the ethnomatters team met on Skype a few days ago to talk about this month’s theme, we reflected on what we thought about the current debate and what we might do to contribute to it. We all agreed that, although we deeply respect academics and researchers who have written personal open access policy statements boycotting closed access publications, our own view is that the open access debate shouldn’t be limited to the intellectual property rights and/or price of final results of research.

This website is testament to the principles that we think are important: we started it so that we would have an avenue to make our research results more accessible to audiences, but perhaps more importantly so that we could have a conversation with other researchers about the changing role of ethnography. In other words, the fact that Ethnography Matters is free and openly licensed is less important than the fact that we are starting new conversations accessible to audiences outside (and in addition to) those who read academic papers – in closed or open access journals. Information, to us, is valuable not just as a neatly packaged product for journals, but as the signal of larger conversations and the enactment of social relationships.

Janet Vertesi and Paul Dourish’s 2011 CSCW paper is really instructive here. Vertesi spent three and a half years embedded with two NASA-led spacecraft teams both working in the field of Planetary Science and examined the differences between the two team’s data sharing principles and practices. She noticed that Paris team members shared raw data internally across all members of their mission and saw their data as an inherently shared resource, while Helen members did not automatically share internally and were on the whole more selective about where their data traveled. By examining the context of data production within the Paris and Helen teams, Vertesi and Dourish concluded that how the data is crafted and acquired (rather than distributed or used) is critical to understanding their members’ different data sharing values. They explained that, with changes in expectations of data sharing within NASA and the broader scientific community, there has been an imposition of Paris-like rules for data-sharing uponHelen, producing culture clashes, confusion and frustration.

Rather than ascribing the sharing regimes to generosity or selfishness, it became important to see data in its “interactional context” where data is not merely an end product, a commodity, but that data “enacts social relationships”; it is one of the interactional elements with which social relationships are performed. Vertesi and Dourish argue that the imposition of data-sharing systems that do not respect the context of production results from “a troublesome process of commodification”, where the interactional and social regimes that produced the data are obscured. In conclusion, the authors call for thinking about “data sharing” within a broader “data economy” that focuses on production as well as use and exchange, so that we might clarify the value of data within local organizational cultures.

Vertesi and Dourish’s paper resonates because it acknowledges the different stages of data sharing in research, rather than merely the end product, and ascribes different data sharing principles to local organizational cultures (rather than different research fields). It reminds us that data has deep sociotechnical origins and that those origins are essential to understanding different cultures of data sharing. ‘Data’ or academic papers or blog posts should not be merely seen as a ‘objects’ that are primed, packaged, ready for transfer, but rather as a speech acts within broader conversations, each of which are shared and ascribed with different value within particular cultures.

For the Ethnography Matters team, it is important for us to support and apply accessibility as a principle to each stage of our research and that strategies to enhance access would necessarily look different according to each of the groups (or conversations) we hope to affect with our work. For Tricia, it is important to share photographs of her fieldwith her research participants and her friends and colleagues back home. For Jenna, it is important that she write an accessible summary of each of her journal articles for Ethnography Matters or her blog. For me, it’s important to enable my research participants to speak back to the data about them. All of us recognize the importance of open access to our research materials, but we think of access somewhat differently from the way that it is traditionally defined.

What do you think ethnographers should do about open access? What does openness look like to ethnographic researchers or to any researchers who study technology? Jenna has written a thoughtful post that has attracted a lot of great feedback on open access journals relating to ethnography. Do you have other suggestions on thinking about data sharing, open access and broader accessibility issues? We’d love to hear from you. Contact us or share your comments below.

Pic: ‘Open window’ by Sharon Hall Shipp. CC-BY-NC on Flickr

Last month’s Wired magazine showed an infographic with a headline that read: ‘History’s most influential people, ranked by Wikipedia reach’ with a group of 20 men arranged in hierarchical order — from Jesus at number 1 to Stalin at number 20. Curious, I wondered how ‘influence’ and ‘Wikipedia reach’ was being decided. According to the article, ‘Rankings (were) based on parameters such as the number of language editions in which that person has a page, and the number of people known to speak those languages’. What really surprised me was not the particular arrangement of figures on this page but the conclusions that were being drawn from it.

According to the piece, César Hidalgo, head of the Media Lab’s Macro Connections group, who researched the data, made the following claims about the data gathered from Wikipedia:

a) “It shows you how the world perceives your own national culture.“

b) “It’s a socio-cultural mirror.“

c) “We use historical characters as proxies for culture.“

And finally, perhaps most surprising is this final line in the story:

Using this quantitative approach, Hidalgo is now testing hypotheses such as whether cultural development is structured or random. “Can you have a Steve Jobs in a country that has not generated enough science or technology?” he wonders. “Ultimately we want to know how culture assembles itself.”

It is difficult to comment on the particular method used by this study because there is little more than the diagram and a few paragraphs of analysis, and the journalist may have misquoted him, but I wanted to draw attention to the statements being made because I think it represents the growing phenomenon of big data analysts using Wikipedia data to make assumptions about ‘culture’.

National culture? 

Hidalgo claims that Wikipedia can show you ‘how the world perceives your own national culture’. But Wikipedia is actually pretty bad at showing national differences because of two key reasons: a) Wikipedia divides projects by language, and languages are necessarily cross national (Portuguese, for example, is spoken by people from two different continents; Arabic Wikipedia has contributions by people on three continents), and b) Different Wikipedia language editions are not necessarily being edited by people from the place traditionally associated with that language.

In 2011/12, I did a study of Swahili Wikipedia and learned, surprisingly, that the majority of edits were being made, not by East Africans where Swahili is a lingua franca but by Europeans, with only one of the three Swahili Wikipedia bureaucrats (Muddyb from Tanzania) a native Swahili speaker. Long-time Swahili Wikipedia editor and bureaucrat from Tanzania, Muddyb, complained about the lack of local contributions to  the encyclopedia on his blog:

The most active users who are contribute on the Swahili Wikipedia is Wazungu (the White people) who coming from various areas (but mostly from Germany). Kipala is the only user who change the Swahili Wikipedia from the lower level to the highest level.  Kipala was the one who took it from the 140 to the 1,000 mark – and make it the first African Wikipedia to reach over the 1,000.

The notion of home/first languages

Similarly, there is a common misconception that editors will edit the encyclopedia in their home/first language rather than a “foreign” language. The Makmende case study showed how Kenyans (who generally speak a native, regional language like Kikuyu or Maasai as well as the “official languages” of Swahili and English) preferred to write the article in English than in Swahili (where the article still does not exist). In interviews, I learned that the majority of Kenyan Wikipedia Chapter members choose to edit in English and that more Kenyans actually read English Wikipedia than Swahili Wikipedia, despite the fact that Swahili Wikipedia is one of the largest African Wikipedias with almost 23,000 articles (Zachte, 2011) (1). In a country where the national language is Kiswahili and the official languages are Kiswahili and English, Swahili Wikipedia gets less than 1% of Wikimedia readership, while English Wikipedia receives 89.6% of all Wikimedia requests. The largest readership of Swahili Wikipedia comes from Tanzania at only 20%, followed closely by the United States at 16.5%, Kenya at 6.6% and Germany at 5.6%. Swahili Wikipedia is actually being read either by East African expats living abroad and/or by Westerners with ties to the region.

Readership and editing of Wikipedia language versions in Kenya and Tanzania as at 31 October 2011 (Source: http://stats.wikimedia.org/)

There are many reasons why this is the case – the most important is the fact that language in Kenya (as in many places on the continent) is intimately connected with an individual’s access to education, employment, and political participation (Simpson, 2008). Even though Swahili was chosen as the national language in Kenya after independence, the colonial experience had a large impact on the way that languages were perceived, since being fluent in a colonial language ultimately meant access to the best positions during colonial times (Simpson, 2008). The legacy of this (and as Simpson argues, a characteristic of African traditional society) is that different languages are used for different situations in modern day Kenya: English for government, business and science and technology discussions, Swahili (or a version, like Sheng) for everyday conversation.

So, even though many East Africans (like Muddyb) complain about the lack of local support for Swahili, there are very important reasons why people choose to edit and read the English version instead. But this also means that looking at Wikipedia data about Swahili cannot ‘show you how the world perceives your own national culture’ because Wikipedia editors of particular language versions are often not even native speakers of the language which is supposedly reflecting this culture.

The role of individuals 

The second problem with using ‘Wikipedia as a proxy for culture’ is that some language versions are very small, and that individuals can significantly skew the data. Take this example below of Mark Graham’s visualization of Swahili Wikipedia.

Wikipedia editors are made up of a particular slice of the population and this tends to reflect the kinds of articles that are representation (and not represented) on Wikipedia. Metafilter has a great discussion about the infographic and I really loved what user ‘eotvos’ had to say about why someone like Vasco da Gamma or Muhammad doesn’t show up in the graphic:

Once again, the subject as has the potential to be a lot more interesting than either the analysis or the article. While this doesn’t say much at all about “influence” or national reach or whatever else the authors suggest, comparisons of wikipedia databases in multiple language could be fascinating.

Consider Copronymus’ lovely example of Vasco da Gamma and Muhammad above. I suspect most of us have a reasonably good idea of what it means to have a wikipedia page about a person in English, or in Spanish. But, how ought we to think about wikipedia articles in languages with less reach?

To pick an example, what does it mean to have a wikipedia page in Nahuatl? The number of literate Nahuatl speakers with Internet access who don’t also speak one of the big 10 languages may not be zero, but it’s vanishingly close to zero. People who create wikipedia pages in Nahuatl are clearly doing something other than informing the monolingual Nahuatl-speaking public about Vasco da Gama. What are they up to, and what can we learn about them from their editorial choices?

Instead of measuring Vasco da Gama’s influence among Nahuatl speakers, these data probably say something about the canon of world knowledge as it is understood by a very small group of tech-savvy language activists. A detailed analysis of that could be really exciting.

November 27, 2012

Indeed. Wikipedia data about different language encyclopedias has interesting things to show us about the culture of Wikipedians themselves. The ways in which that data points to yearning for friendships across the seas, to access to what some see as greener pastures, to the things that they want people who are watching or reading to see and the things that they see should be part of the global English corpus. But Wikipedia is no proxy for national culture.

In the next post, I’ll talk about the particular mechanics of editing on Wikipedia that shape what ends up being represented.

Footnotes:

(1) Afrikaans and Swahili Wikipedias tend to periodically switch with one another as being the largest African Wikipedia. See Ian Gilfillan for more on this.

(2) Muddyb explains his editing of articles on Tanzania and Norway as an expression of friendship towards Wikipedians from those countries. See http://muddybtz.blog.com/2011/08/14/this-time-for-norway/

Featured image: Planisphæri cœleste Celestial map from the 17th century, by the Dutch cartographer Frederik de Wit. Public domain from Wikipedia.

Ward Cunningham, inventor of the wiki, at the first WikiSym in 2005 which was co-located with ACM OOPSLA in San Diego, California. Pic by Peter Kaminski CC BY on Flickr.

There has been much reflecting and soul-searching about the future of WikiSym in the past year (and probably before that as well). Many felt that the conference was becoming dominated by Wikipedia research and that it needed to grow to encompass more research in the open source, open data and open content realm. I felt that the conference needed to attract more social scientists and qualitative researchers in order to reach more detailed understanding of Wikipedia is being integrated into everyday life.

Despite the negatives, everyone felt that WikiSym was and still is the best place for people who do research about Wikipedia and other wikis to gather and that there was a lot of promise in broadening our mandate. This is why I feel so excited about co-chairing a new dedicated Wikipedia track at next year’s WikiSym in Hong Kong along with Mark Graham, also at the Oxford Internet Institute. And that’s why I was also happy that Dirk Riehle, veteren of WikiSym, is at the helm again next year, leading an effort to redesign the event around a changing research landscape.

There are a few key differences to next year’s event:

1. WikiSym 2013 will be held jointly with a new conference called ‘OpenSym’ and the entire event will consist of four tracks dedicated to different research trajectories:

• Open collaboration (wikis, social media, etc.) research (WikiSym 2013), chaired by Jude Yew of National University of Singapore
• Wikipedia research (WikiSym 2013), chaired jointly by myself and Mark Graham of the Oxford Internet Institute at the University of Oxford
• Free, libre, and open source software research (OpenSym 2013), chaired jointly by Jesus M. Gonzalez-Barahona and Gregorio Robles of Universidad Rey Juan Carlos
• Open access, data, and government research (OpenSym 2013), chaired by Anne Fitzgerald of Queensland University of Technology

This means that Mark and I can focus on getting the very best of Wikipedia research to WikiSym and in thinking hard about what is missing and what needs to be encouraged in the years to come.

2. WikiSym 2013 will once again, after a hiatus of two years, be co-lated with Wikimania, Wikipedia’s annual community conference, which will enable us to benefit from the input of some of the amazing people who congregate for Wikimania and will hopefully strengthen the already-valuable input that we always have from experienced Wikipedians commenting on current research.

3. WikiSym 2013 will hopefully attract more social scientists and more qualitative researchers who can lend some new in-depth perspectives to Wikipedia research. Because Wikipedia is such a wonderful source of big data about the world, we tend to see a lot of big picture perspectives of the encyclopedia.But if we want this to be an interdisciplinary gathering and if we want to complement big data perspectives with more in-depth understandings of social and historical features of Wikipedia, we will need to attract a greater diversity of researchers. For this reason, we’re thinking of ways to enable social scientists to use the conference as a way to discuss their ideas in a way that doesn’t prevent them from publishing in journal publications. We’ve realised that some social scientists don’t attend WikiSym because they have different publishing requirements to the computer scientists regarding publishing in proceedings, (preferring to publish in journals rather than in conference proceedings) and so we have a new paper format called ‘presentation paper’ in which only the abstract of the paper presented at the event will be published as part of the proceedings (even though the entire paper must be submitted for review).

Those are three key changes to WikiSym 2013. We’ve engaged with the Wikipedia research community and our amazing committee to highlight specific types of Wikipedia research that we’re interested in seeing more of. In particular, questions like:

• What do particular articles or groups or articles tell us about the norms, governance and architecture of Wikipedia and its impact on media, politics and the social sphere? How is information on Wikipedia being shaped by the materiality of Wikipedia infrastructure?
• What is the impact of all/some of Wikipedia’s 211 language editions having on achieving the project’s goal to represent the “sum of all human knowledge”? Do smaller language editions follow the same development path as larger language editions? Can different representations in different languages tell us anything about cultural, national or regional differences?
• What are the gendered dimensions of Wikipedia editing? How are issues around power, knowledge and representation drawn into focus by gender, geography and other gaps and imbalances in Wikipedia editing?
• What skills/competencies/connections/world views are required to become an empowered member of the Wikimedia community? What would a Wikipedia literate person look like? How are those skills/competencies/connections/world views obtained and enacted?
• Does Wikipedia enact an open source of authoritative knowledge that impacts learning in formal and informal settings? For instance, how do students employ Wikipedia as a covert/overt source in their papers or as a generative site for problem formulation? Or how is Wikipedia being used as a serendipitous experience of knowledge acquisition? What methods can be employed to understand these varied utilizations?
• What is the effect of outreach initiatives involving the growing institutionalisation of Wikipedia activities? As galleries, libraries, archives and museums hire Wikipedians-in-residence to digitize, showcase and/or represent their collections, is Wikipedia able to fill some its key knowledge gaps? Or are there unintended effects of this institutionalization of knowledge?
• What are the methodological challenges to studying Wikipedia? How are researchers engaging with innovative methodologies to solve some of these problems? How are other researchers using traditional or well-established methods to study Wikipedia?
• How are wiki projects other than Wikipedia evolving? What are the benefits to studying other wiki projects and can comparisons and generalisations be made from our observations of these systems?
• How does information contained in Wikipedia shape our understanding of broader social, economic, and political practices and processes? What theoretical frameworks in social, economic, legal and other relevant theoretical traditions can be applied to enrich the academic discourse on Wikipedia?

Led by the amazing committee of people who will be evaluating papers including:

Megan Finn
Affiliation: Microsoft Research, New England
Home page URL: http://research.microsoft.com/en-us/people/megfin/

Stuart Geiger
Affiliation: UC-Berkeley School of Information
Home page URL: http://www.stuartgeiger.com

Brent Hecht
Affiliation: Department of Computer Science and Engineering, University of Minnesota
Home page URL:http://www.brenthecht.com

Brian Keegan
Affiliation: Northeastern University
Home page URL:www.brianckeegan.com

Wen Lin
Affiliation: Newcastle University
Home page URL: http://www.ncl.ac.uk/gps/staff/profile/wen.lin

Felipe Ortega
Affiliation: Researcher, Dept. of Statistics and Operations Research, University Rey Juan Carlos.
Home page URL:http://felipeortega.net

Dan Perkel
Affiliation: IDEO
Home page URL: http://blogs.ischool.berkeley.edu/dperkel/

Joseph Reagle
Affiliation: Northeastern University
Home page URL: http://reagle.org/joseph/

Jodi Schneider
Affiliation: DERI, NUI Galway
Home page URL: http://jodischneider.com/jodi.html

Monica Stephens
Affiliation: Humboldt State University
Home page URL: https://sites.google.com/a/email.arizona.edu/stephens/

Dario Taraborelli
Affiliation: Wikimedia Foundation
Home page URL: http://nitens.org/taraborelli

Robert West
Affiliation: Computer Science Department, Stanford University
Home page URL: http://ai.stanford.edu/~west1/

Matthew W. Wilson
Affiliation: Department of Geography, University of Kentucky
Home page URL: http://matthew-w-wilson.com

Taha Yasseri
Affiliation: Oxford Internet Institute, University of Oxford
Home page URL: http://www.oii.ox.ac.uk/people/yasseri/

Matthew Zook
Affiliation: University of Kentucky
Home page URL: http://zook.info

I’m confident that we’ll be building something really great with this wonderful group of some of the world’s most accomplished and/or up-and-coming information/Web and/or Wikipedia researchers. I’m hoping that this will stand us in good stead to see a much more diverse and extensive panel next year highlighting what is missing from Wikipedia research. I’d also personally love to see some of the great ethnographies of Wikipedia in a panel at the conference, and to see more research on Wikipedia in developing countries. I’ll be searching for people to encourage to attend so if you know of anyone, please let me know! More soon

Mark Graham talks about the stated goal of Wikipedia to become the “sum of all human knowledge” while Ahmed Medat waits to translate into Arabic

It was the end of the final day of our workshop on the outskirts of Cairo and we were all feeling that curious mixture of inspiration, energy and exhaustion that follows those meetings where a world of ideas and people and things are thrown together in a concentrated few days. Mark Graham asked each of us if we’d like to say a few parting words and the participants spoke about how they enjoyed meeting Wikipedians from so many places in the Middle East, that they were happy to come to an event with academics and that they were excited about doing something to make a change in the real world. The majority of participants spoke in English – what was for many of them a third or fourth language – while some had their Arabic translated on the fly by other participants.

I was surprised when we got round to Mohamed Amarochan, Wikipedian, Mozilla hacker and blogger from Morocco, when he said that he would like to speak in Arabic. I knew that Mohamed had a really good command of English because I’d spent a fascinating ride with him from the airport on the way to the workshop where we commiserated with one another about visa hardships. When he chose to speak in Arabic and allowed others to translate into English, I realized that Mohammed was making an important statement about how small decisions like which language you choose to speak in a conversation like this one has big consequences.

As Clive Holes writes, ‘How we speak is an important part of who we are: in a sense, speech is the oral counterpart of how we dress. Both are intimately linked to our sense of self, and of how we prsent ourselves to, and are seen by, others.’ (Holes, 2011)

Language was a luminous thread running through the two-day workshop, sparking some pretty intense debates among participants who didn’t always agree with one another about what the real problems were or what the perfect strategy was in improving the coverage of Arab topics on Wikipedia. A particularly interesting issue was around the validity of the Masri (Egyptian Arabic) version of the encyclopedia and whether it “divided the ranks” of an already small effort, whether it was an initiative dominated by people from outside the Middle East (with some hinting that there was perhaps some ulterior motive behind the initiative) or even whether it was a legitimate language in its own right or merely a dialect of Arabic and should thus not have been provided the status of an independent Wikipedia. Many participants said that they felt “offended” by the existence of the Masry Wikipedia which they saw as a “politically-motivated fork filled with bots and hoaxes” – political in the sense that its existence came into being because of issues around identity and power, rather than practical considerations such as how many people would actually edit it. Others said that it was up to any community who speak a language to create their own Wikipedia and that since the user base was small, it didn’t really have an impact on the Arabic Wikipedia project.

After I wrote the first version of this post and sent it to participants, Egyptian Wikipedian, Ahmed Shawky Mohammedin sent me a well-researched eight-page document of  comments and notes. He said that he had spent the entire day poring over it, and although he was incredibly respectful and kind, he said that and felt that I didn’t completely understand the status of Egyptian Arabic in Egypt. I read further and realized that the situation is a lot more complex than I had originally understood it to be and that there is even disagreement within the linguistic community itself about the interaction between Egyptian Arabic and the so-called “Classical Arabic” of the region – so much so that people within Egypt often feel that the languages (and/or dialect) has been misunderstood (see Suleiman in Simpson, 2008).

It turns out that Egyptian Arabic is spoken natively by about 50+ million Egyptians and as a second language by most of the remaining 24 million Egyptians in several regional dialects, as well as by immigrant Egyptian communities in the Middle East, Europe, North America, Australia and South East Asia (see UCLA Language Materials, the Ethnologue and Omniglot) giving some credence to the claim that it the Wikipedia Masry is perhaps being driven by people outside of the region. Egyptian Arabic is also a signal of the regional power of Egypt and the mobility of Egyptian professionals through the MENA region. Egyptian Arabic is said to have become a lingua franca in other parts of Arabic-speaking world because of the proliferation of Egyptian films and other popular media throughout the region, and because of the high numbers of Egyptian teachers and professors who helped set up the education systems in other Arab countries.

Despite the widespread speaking of Egyptian Arabic throughout the region, it is still predominantly a spoken rather than written language. Yasir Suleiman (Suleiman in Simpson, 2008) writes about the complexities of talking about Arabic and Egyptian Arabic in Egypt, preferring to use the terms al-lugha al-’arabiyya al-fusha (eloquent Arabic language) and al-’ammiyya (the colloquial, referred in previous paragraphs as Egyptian Arabic). Suleiman writes that many people outside of Egypt see ‘ammiyya as ‘the Egyptian language’ but that for most Egyptians ‘ammiyya is not a language but a kind of dialect that is ‘denied the  status of language in common parlance’ and that is often incorrectly described by locals as ‘having no grammar’ even though it is the mother tongue of Egyptians and the one through which they are socialized.

Participants at the workshop brainstorming ideas about whether it is important to have Arab topics represented on Wikipedia

Suleiman writes about the historical role of language in terms of Egyptian and later Arab nationalism. Nationalists during different periods of Egypt’s history have used fusha (Classical Arabic/standard) and ‘ammiyya (Egyptian Arabic/colloquial) as a vehicle to assert the ambitions of Egypt but that this has changed over time. In the 1920s, the so-called ‘territorial nationalists’ embarked on a vernacularization campaign to promote ‘ammiyya ’as the only authentic and legitimate voice of Egypt’ but Suleiman writes that the shift from Egyptian nationalism to pan-Arabism in the 1930s which emphasized the linguistic community not the speech community and the native language not the mother tongue (i.e. standard Arabic not the colloquial Egyptian Arabic). He writes that new debates are around conflicts involving ideologies of tradition and change or authenticity and foreignization/Westernization (taghrib) as Arabic struggles with the need to modernize and thus to import and localize concepts from the West.

Interestingly, the book in which Suleiman’s chapter appears was published in 2008, the year in which Masri Wikipedia was founded, the same year that Wikimania, the annual Wikimedia community conference, was hosted in Alexandria, Egypt. Wikipedia Masri started as a project in the Wikipedia Incubator in April 2008 and in November the same year, it became an official Wikipedia. Masri Wikipedia was mired in controversy from the beginning of its existence with some users establishing Facegroup groups to encourage vandalism against the encyclopedia. It seems that opposition was fueled by the elevation of the colloquial Masri to an encyclopedia, what is by many accounts considered to be the highest form of written “truth”, very different from the popular films and other media that Masri is traditionally used for. Others saw Masri Wikipedia as an affront against the attempts of Arabic Wikipedia to unify efforts across the Arab world.

Looking at the statistics of usage of Masri Wikipedia, it seems that the narrative about classical and colloquial Arabic in Egypt may not be as settled as Suleiman presents it in his chapter on Egyptian language and nationalism. Masri Wikipedia is still a small encyclopedia compared to Arabic as you can see in the table below:

Data derived from stats.wikimedia.org as at 29 October, 2012

But despite small editor numbers, Masri Wikipedia users are on the rise, and given the language’s historical ties to nationalism and democracy movements, there are few concrete indications that it is dying anytime soon (despite many Arabic Wikipedians’ statements to the contrary).

Erik Zachte, stats.wikimedia.org as at October 31, 2012 

Getting a more nuanced understanding of issues around language dynamics has a big impact on the way in which Wikipedias in different parts of the world are evaluated and nurtured. The Western world tends to think of language in static numerical terms. Appropriate languages for particular communications are often chosen according to statistics about the number of language speakers, rather than understanding the role of different languages used in different social and educational contexts.

In Tunisia, for example, almost everyone speaks Tunisian Arabic natively, and although there has been an attempt in recent years to Arabise facets of life that were dominated by French during and after the colonial period, including the teaching of scientific subjects, French is still widely used in business, in natural sciences, medicine and intellectual domains (Wikipedia). Although the people of Tunisia are almost certainly “counted” as native Arabic speakers, the reality is therefore a lot more complex. Many participants said that Arabic Wikipedia was much stronger in topics like history, religion and literature, but still pretty weak in maths, science and technology. Some said that there was a perception among some young people in the region that Arabic was “backwards” and that there was a problem that language standardization bodies take a while to find an Arabic translation for new words like “blog”.

Now I look at the statistics to the left in a whole new light. Comparing the number of editors according to the number of language “speakers” is actually an inappropriate measure of interest by a community in editing Wikipedia and it reflects a very Western framework of assessing participation. Arabic is the national language of twenty countries and the native language of approximately 300 million people, and Egyptian Arabic is the mother tongue of over 50 million people in Egypt, but editors may still choose to edit some articles in English or French and may vehemently disagree about which local language/dialect is appropriate for encyclopedic writing. The impact of this is that participation by people in the MENA (and many other developing nations) is dependent to a large extent on the ways in which language is tied to different spheres of knowledge.

Of course, there is another layer to this discussion and that was the assumed preference for English at this wonderful workshop about Arab topics on Wikipedia. Although there are almost 300 language versions of Wikipedia, the project is still framed and debated in the particular values and needs of the English-speaking world, and this bias is carried through to face to face settings. Wikipedia is certainly doing a great job of including a lot of people in its quest to become ‘the sum of all human knowledge’ but it would be a mistake to think that the language in which its goals are framed has no impact on the intentions, motivations and desires of its participants.

When the workshop was over, I remarked to Mohamed Amarochan that I was glad that he had chosen to speak Arabic there. He said, “Yes! It’s an Arabic workshop so I should speak Arabic.” Well said, Mohamed. Well said

Reference:

Suleiman, Y. (2008). ‘Egypt: From Egyptian to Pan-Arab Nationalism’. In A. Simpson (ed), ‘Language and National Identity in Africa‘. Oxford: Oxford University Press.

Huge thanks to the participants of the workshop in particular to Ahmed Shawky Mohammedin for his extensive feedback on an earlier version of this post. The workshop entitled ‘Representation on Wikipedia in the Middle East and North Africa’ was a collaboration between the University of Oxford, the American University of Sharjah and University Tunis ElManar. It was funded by the International Development Research Centre in Canada and the John Fell Awards Scheme at the University of Oxford. The organising team included Dr Ilhem Allagui, Dr Ali Frihda, Dr Mark Graham, Dr Bernie Hogan, Ahmed Medhat, Clarence Singleton and myself. For more information on the results of the research, see the Oxford Internet Institute’s website here. 

Download as PDF

Abstract: Wikipedia is, in many ways, the poster child of the Internet Age. It has been singled out as the ultimate working example of the collaborative power of the Internet (Shirky, Tapscott) and what Yochai Benkler calls ‘commons-based peer production’ to describe how the Internet has created radical new opportunities for how we make and exchange information, knowledge, and culture (Benkler, 2009). Part of its popularity comes from its power to influence and inform. As the sixth largest website in the world, with over million users and 90,000 active editors, Wikipedia is becoming one of the most influential reference works in history.

For every broad statement about Wikipedia, however, there are examples on the ground that hint at an alternative reality. The ideal that commentators (many of whom are not involved in editing the encyclopaedia on a daily basis) project is of a unified group of rational, detached, individual editors building a neutral, free encyclopaedia that is “the sum of all human knowledge”. But the organic nature of the encyclopaedia, its culture, politics and architecture have produced and continue to produce an encyclopaedia in which particular tactics, identities and relationships, many of which are in defiance of original rules, often prevail over reasoned and rational dialogue. Wikipedia still has a number of “dark spots”: from uneven geographies of articles written about places (Graham, 2011), to low numbers of female contributors (Lam et al, 2011) and vastly different levels of quality (Duguid, 2006). But there are other dark spots too – spots within the encyclopaedia itself: knowledges that are silenced, perspectives that are marginalised and people that are banned.

Who wins and who loses in this open environment? How do culture, politics, regulations, architecture and identity influence who wins or loses? And what does this mean for the way we think about online collaboration, its power and pitfalls?

I hope to answer these questions in an ethnographic study of Wikipedia’s marginalised knowledges, its deleted pages and banned users. Ethnography, a rich bouquet of methods that stresses the importance of theory grounded in the everyday experience of users, offers an opportunity for methodological innovation and a new lens for looking at Wikipedia’s missing pieces. Using Burrell’s “field site as network” (2009) I will construct a field site that retains coherence even though it traverses multiple language editions and places. Beginning with the story of banned editor of Hebrew, English and Arabic versions, “drork”, I will go on to using “trace ethnography” (Geiger and Ribes, 2011) to analyse and visualize trends in deleted articles, history pages, banned users, related talk pages, bots and mailing lists, and iteratively zoom back in to the ground-level reality to interview other banned users and those involved in deleted pages discussions.

In doing so, I hope to shed light on what I believe is one of the most important questions of our time: does new technology offer an opportunity for people to collaboratively develop mutually accommodating truths? What new power relations are being built around these new knowledges and this new “visibility” (Tkacz, 2007) of marginalised knowledges that Wikipedia architecture represents? Understanding where Wikipedia is failing, who it bans and what it deletes as Wikipedians who edit multiple versions of the encyclopedia navigate its rules, architecture, norms and market to build pages together is a critical piece in that puzzle.

Rationale: In the introduction to “Critical Point of View: A Wikipedia Reader” published earlier this year, Geert Lovink and Nathaniel Tkacz reflect on Wikipedia’s tenth year anniversary celebrations declaring that, although there might be new voices that comment on Wikipedia, the terms of debate about the encyclopaedia is still very narrow (Lovink and Tkacz, 2011, p.10). They write that at this junction in Wikipedia’s history what is missing is an informed, radical critique from the inside. ‘What does Wikipedia research look like when the focus is no longer on the novelties of (open) collaboration or on whether Wikipedia is trustworthy and accurate?’ (Lovink and Tkacz, 2011, p.11)

Much of the research on Wikipedia has, until fairly recently, focused on the encyclopaedia as a novel phenomenon – a surprising project in which thousands of volunteers contribute their labour for free and in which anyone can edit a page and contribute to its growth. The question seemed to be: why does Wikipedia work? Or what Yochai Benkler asks: ‘Why can fifty thousand [1] volunteers successfully coauthor Wikipedia, the most serious online alternative to the Encyclopedia Britannica, and then turn around and give it away for free?’ (Benkler, 2007, p.5)

But Wikipedia is no longer novel: it has become deeply ingrained into everyday life. And while the question about why Wikipedia works seems to have been answered in the light of such novelty, the question about why and where it doesn’t work seems to have been less well analysed.

The entry point to this research investigating Wikipedia’s dark spots is the story about a banned editor, drork. drork is one of the thousands of editors that Wikipedia loses each month, a number that is not being replaced as rapidly as it was in earlier years. In November, 2009, Felipe Ortega published his Ph.D thesis in which he found that the English Wikipedia had lost 49,000 editors during the first three months of 2009 as opposed to 4,900 editors during the same period in 2008 [2] prompting calls that Wikipedia may be in decline. Recent numbers have confirmed this slow decline but we still know very little about why Wikipedia is losing more editors than it is gaining.

A long-time Wikipedia editor of the English, Hebrew and Arabic Wikipedias on Middle East topics, one of the founding members and previous board member of Wikimedia Israel, drork was for many years a model Wikipedian. A linguist by training, drork epitomized many of the ideals of Wikipedia: he was committed to the ideals of the encyclopedia, his language skills enabled him to edit across three different language versions in an area rife with disputes, he was transparent and open in his editing (drork is one of the few editors who uses his full name on his user page) and played a strong leadership role beyond editing, speaking to the media about his experiences editing Middle East topics and assisting with outreach projects to bring offline Wikipedia to countries in Africa.

After about seven years editing, drork has come to believe that Wikipedia is no longer concerned with the “truth” in the tradition of the Enlightenment that it had originally purported to serve. Losing patience with the edit wars, cabals, ganging up and victimization, drork’s passion and impatience finally got the better of him. In early 2010, in a series of Wikipedia “trials”, he was banned from editing for 24 hours on the English encyclopedia and then six months for disruptive behaviour. Unable to keep from what he admits had become an obsession[3], drork kept editing under a number of pseudonyms (what Wikipedians call “sockpuppets”) and was repeatedly banned until his recent lifetime ban.

drork’s story is a rich point of entry for researching how consensus is reached and where it seems never to be reached, how deadlocks are exacerbated by the norms and architecture of Wikipedia as well as what kinds of tactics are used to win in articles covering political disputes. His story also sheds light on a very different reality from the picture of the objective, detached, individual editor to show how important identity, place and tactics are to who succeeds in establishing the predominant version of an article on Wikipedia.

This research will be novel not only because it centers on the grounded stories of its editors. It will be novel also because I will traverse a number of different language versions in trying to understand power relations between the different encyclopaedias (most of the previous studies have focused on one version of the encyclopaedia or dealt with the main versions separately). It will be novel because of its focus on deleted articles, marginalised knowledges and banned users, an area that, to my knowledge, has not been studied in depth.

Methodology: At this year’s annual WikiSym, Wikipedia researcher, Cliff Lampe critiqued current research initiatives. ‘We’re studying sites, we should be studying people. Wikipedians didn’t come to Wikipedia without baggage. Their cultural and ethnic background is important but the problem is that it’s messy and it’s hard to get that data.[4]’

Many have used statistics and high-level visualizations to try to understand Wikipedia’s missing pieces and people (Ortega, Chi) but there are still no comprehensive on-the-ground studies about why they leave and why articles are deleted. With much of the current research trying to obtain a coherent, all-encompassing view of the encyclopedia, many of the details and divergences have been lost. Up until fairly recently, research on Wikipedia has assessed and analysed the encyclopaedia as a single, monolithic community that behaves in certain ways, is motivated by specific variables, and acts according to rules set out in its policies.

But on closer inspection, Wikipedia is more of a city than a single community. Like a city, there is constant migration from some areas to highly populated ones: while overall growth of editors is stagnating, editors editing trending news topics is on the rise, being attributed to a large portion of all traffic and edits to Wikipedia[5]. Like a city, there are places where rules are strictly enforced and places where one can treat a red light as a yield sign: where some editors are banned because of verbal abuse but where others who do the same go about unabated. Like a city, Wikipedia needs to be understood in terms of where principles are flourishing and producing and where editors are struggling to keep a clean house, or where all the sane people have left and militia is holding down the neighbourhood.

Understanding what Wikipedia deletes and who it bans is critical to understanding what it takes to win and have your edits prevail on Wikipedia. In my recent work in this area, I have written about the experiences of Kenyan Wikipedians attempting to write an article about a local superhero in ‘The Missing Wikipedians’ (Ford, 2011) and being continuously reverted. More recently, I undertook a study of deleted articles on the English Wikipedia with Stuart Geiger, in which we found that the vast majority of the hundreds of articles that are deleted by Wikipedia administrators are not spam, vandalism or “patent nonsense” but rather articles which could be considered encyclopedic but do not fit the project’s standards (Geiger and Ford, 2011). The next step is to drill down into understanding, through a comprehensive on-the-ground study, why Wikipedians leave and why articles are deleted.

I would begin this project by setting the boundaries of the field site and defining the subjects and objects of research. But constructing a field-site becomes problematic when studying online communities like Wikipedia where subjects are not conveniently co-located in a consistent physical space. How does one study multi-sited online communities while still taking account of the powerful role of place in defining how technologies are adopted, rejected, absorbed, reflected and shaped?

Some of the most interesting new methods to solve this methodological conundrum have emerged from the field of ethnography which Jenna Burrell defines as “a complex of epistemological framings, methodological techniques, and writing practices (that) has spread into many domains and disciplines beyond its roots in cultural anthropology.” (Burrell, 2009, p.181) In order to reconcile such spatial complexities, Burrell ‘conceived of (her) field site as a network composed of fixed and moving points including space, people and objects’ (Burrell, 2009, p.189).

According to Burrell, there are two key advantages to conceptualizing the field site in this way. It enables one to develop unconventional understandings of social practices because it is a structure that can be constructed using the observable connections performed by participants. And secondly, the “field site as network” produces a continuous space that does not presume proximity or even spatiality in a physical sense.

‘Continuity does not imply homogeneity or unity; it implies connection. The continuity of a network is evident in the way that one point can (through one or more steps) connect to any other point. In a “field site as network,” the point of origin, the destination(s), the space between, and what moves or is carried along these paths is of interest.’ (Burrell, 2009, p.190)

Using Burrell’s field site as network and Marcus’s idea to “follow the person” (Marcus, 1998) I will begin the research by conceptualizing of my field site as a network composed of fixed and moving points including spaces, people and objects and Burrell suggests. I will use grounded theory, an iterative methodology that emphasizes the generation of theory from data in the process of conducting research, as a way of staying close to the lived experience of the community, building levels of abstraction directly from the data, developing theories iteratively and then gathering further data to check and refine the emerging analytic categories (Charmaz, 2006).

The story of banned editor, drork is my starting point. From there, I will conduct (further) interviews with him and those he has interacted with, using “trace ethnography” (Geiger and Riber, 2011) to analyze the pages that played a part in his banning, moving on to analyze and visualize the corpus of deleted pages on Wikipedia and interviewing other banned editors and editors involved in deleted pages.

The people under study include those who have been banned or who have left Wikipedia as well as those involved in deletion discussions (AFDs) and speedily deleted pages (CSDs). Spaces include those in which banned editors have been engaged (talk pages from articles related to the Middle East, arbitration and sockpuppet investigations pages) and those that have been deleted (AFD discussions, CDSs, talk pages). Objects include the traces left behind by bots involved in administrative tasks (Geiger, 2011) as well as on Twitter as people discuss what is deleted. I am particularly interested, here, in new thinking around the particular “agency” of non-humans on Wikipedia and their role in directing what is not seen.

Conclusion: Wikipedia, by creating a new knowledge form, is also creating new politics. In a 2007 article entitled ‘Power, visibility, Wikipedia’, Nathaniel Tkacz argues that a new power relation, that of “visibility” is at play in Wikipedia. Drawing from Geleuze’s (2006) reading of the work of Michel Foucault, Tkacz shows how the edit and history functions within Wikipedia’s architecture reveal what is being silenced, and how the discussion pages become ‘a place for marginalized knowledges’ (Tkacz, 2007, p.14).

‘The task that lies ahead,’ he writes, ‘is to map the relations of power in greater detail and thus provide a new diagram of power to match this new visibility.’ (Tkacz, 2007, p.17) Understanding what Wikipedia silences is critical to understanding these new power relations and answering this question has important implications for questions we still have about peer produced or crowdsourced information. What types of personalities and identities prevail in structureless (Freeman, 1970) organisations? What is the role of architecture, identity, norms and regulations in these new relations of power? And what does this mean for the way we think about online collaboration, its power and pitfalls?

I believe that my current research and rich experience as an “insider” working in the open content and open source software communities for the past decade stands me in good stead to try to answer these questions and to present ground breaking research that enables us to move beyond statements about the novelty of peer production towards a new generation of research that has Wikipedia users and other knowledge stakeholders at their center.

Bibliography

Benkler, Y. (2007). The Wealth of Networks: How Social Production Transforms Markets and Freedom (p. 528). Yale University Press. Retrieved from http://www.amazon.com/Wealth-Networks-Production-Transforms-Markets/dp/0300125771

Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide through Qualitative Analysis (1st ed.). Sage Publications Ltd.

Duguid, P. (2006). Limits of self-organization: Peer production and “laws of quality.” First Monday, 11(10). Retrieved from http://frodo.lib.uic.edu/ojsjournals/index.php/fm/article/view/1405

Ford, H. (2011) ‘The Missing Wikipedians’ in Geert Lovink and Nathaniel Tkacz (eds), Critical Point of View: A Wikpedia Reader, Amsterdam: Institute of Network Cultures, 2011. ISBN: 978-90-78146-13-1.

Freeman, J. (1970). The tyranny of structurelessness. Retrieved from http://www.midiaindependente.org/media/2001/07/203242.pdf

Geiger, R.S. (2011). ‘The lives of bots’ in Geert Lovink and Nathaniel Tkacz (eds), Critical Point of View: A Wikpedia Reader, Amsterdam: Institute of Network Cultures, 2011. ISBN: 978-90-78146-13-1.

Geiger, R. S., & Ford, H. (2011). Participation in Wikipedia’s Article Deletion Processes. WikiSym 7th international symposium on wikis and open collaboration.

Geiger, R.S., & Ribes, D. (2011). Trace Ethnography: Following Coordination Through Documentary Practices. In Proceedings of the 44th Annual Hawaii International Conference on Systems Sciences.  Retrieved from http://www.stuartgeiger.com/trace-ethnography-hicss-geiger-ribes.pdf

Lih, A. (2009). The Wikipedia Revolution: How a Bunch of Nobodies Created the World’s Greatest Encyclopedia [Hardcover] (p. 272). Hyperion. Retrieved from http://www.amazon.com/Wikipedia-Revolution-Nobodies-Greatest-Encyclopedia/dp/1401303714

Ortega, F. (2007). Quantitative analysis of the Wikipedia community of users. Proceedings of the 2007, 75-86. New York, New York, USA: ACM Press. doi:10.1145/1296951.1296960

Reagle, J. (2010). Good Faith Collaboration: The Culture of Wikipedia (p. 264). The MIT Press. Retrieved from http://www.amazon.com/Good-Faith-Collaboration-Foundations-Information/dp/0262014475/ref=sr_1_1?s=books&ie=UTF8&qid=1321268259&sr=1-1

Tapscott, D. (2010). Wikinomics: How Mass Collaboration Changes Everything (p. 368). Portfolio Trade; Expanded edition. Retrieved from http://www.amazon.com/Wikinomics-Mass-Collaboration-Changes-Everything/dp/B004J8HXOA/ref=pd_sim_b_7

Tkacz, Nathaniel. (2007) Power, Visibility, Wikipedia [online]. Southern Review: Communication, Politics & Culture, Vol. 40, No. 2, 2007: 5-19. Retrieved from http://search.informit.com.au/documentSummary;dn=946871107175453;res=IELHSS. ISSN: 0038-4526.

Wikimedia project at a glance. (n.d.). Retrieved November 14, 2011, from http://stats.wikimedia.org/EN/SummaryEN.htm

This was my itinerary from June to October:

San Francisco – Johannesburg (with family) – Cape Town (with Liv) – Johannesburg – Rome (with Steph)- Falerone (with Steph and James and Jon) – Naples – Ravello - Vescovado di Murlo (with Sarah and Eric and Ellie and Helena) – Rome – Washington D.C. (for Wikimania) – Rome – Tel Aviv (with Elad) – Jerusalem – Tiberias – Ashdod – Tel Aviv – Berlin (with Vicky and Alex) - Münster (with Judy and Meinfred) – Baden-Baden – Berlin – Linz (for WikiSym) – Johannesburg – Exeter (with mom) – Padstow – Penzance – Torquay – Oxford – Painswick – Oxford (me, just me)

So many adventures were had. It wasn’t easy (it’s no surprise that the word ‘travel’ comes from the word ‘travail’, to toil, or labor) but I was surprised at how I felt like I could do this forever – wander from one place to the next, visiting friends and peeking in on their lives. Because of the visa insanity and the fact that I need a lobotomy, I didn’t have a camera (not even my iPhone!) for most of the trip. I really wanted to capture everything and so I drew a lot. This, below, was one of my favorite moments:

I was sitting on a stone staircase in Jerusalem’s Mamilla Ave overlooking the street below. It was about 10pm but the street was really busy, with families and groups of young people hanging out and celebrating the summer night. I got out my new notebook and pencil when I saw this Hasidic artist guy drawing cartoons of three beautiful young European women. I was sitting behind him drawing him drawing them and I kept thinking what a wonderful moment it was for those girls – how they would think back to the time they got a street artist to capture them on a hot summer night in Jerusalem.

I also drew a lot of pictures of my feet because people seem to move about too much!

I learned a lot about myself on the trip (as one does). I learned that I want to live in a place without electric fences and barbed wire, in a small(ish) town where people know and look out for each other, and where a short drive gets me out into the countryside where I can walk in the mud and swim in the river and climb to the top of a mountain and look down. I have a picture of it in my head. Oxford will do very well for now, I think. I have found three amazing people who I hope will become my friends, I’ve joined the Mountain Club, started learning Arabic, I go running through the park in the mornings and have blocked off time to write and read everyday. If I can keep this up, it’s going to be an incredible year.

On the first day of WikiSym last week, as we started preparing for the open space track and the crowd was being petitioned for new sessions over lunch, I suddenly thought that it might be a good idea for researchers who used ethnographic methods to get together to talk about the challenges we were facing and the successes we were having. So I took the mic and asked how many people used ethnographic methods in their research. After a few raised their hands, I announced that lunch would be spent talking about ethnography for those who were interested. Almost a dozen people – many of whom are big data analysts - came to listen and talk at a small Greek restaurant in the center of Linz. I was impressed that so many quantitative researchers came to listen and try to understand how they might integrate ethnographic methods into their research. It made me excited about the potential of ethnographic research methods in this community, but by the end of the conference, I was worried about the assumptions on which much of the research on Wikipedia is based, and at what this means for the way that we understand Wikipedia in the world. 

WikiSym (Wiki Symposium) is the annual meeting of researchers, practitioners and wiki engineers to talk about everything to do with wikis and open collaboration. Founded by the father of the wiki, Ward Cunningham and others, the conference started off as a place where wiki engineers would gather to advance the field. Seven years later, WikiSym is dominated by big data quantitative analyses of English Wikipedia.

Some participants were worried about the movement away from engineering topics (like designing better wiki platforms), while others were worried about the fact that Wikipedia (and its platform, MediaWiki) dominates the proceedings, leaving other equally valuable sites like Wikia and platforms like TikiWiki under-studied.

So, in the spirit of the times, I drew up a few rough analyses of papers presented.

It would be interesting to look at this for other years to see whether the recent Big Data trend is having an impact on Wikipedia research and whether research related to Wikipedia (rather than other open collaboration communities) is on the rise. One thing I did notice was that the demo track was a lot larger this year than the previous two years. Hopefully that is a good sign for the future because it is here that research is put into practice through the design of alternative tools. A good example is Jodi Schneider’s research on Wikipedia deletions that she then used to conceptualize alternative interfaces  that would simplify the process and help to ensure that each article would be dealt with more fairly.

Talking about ethnography?

I am still intrigued by the fact that so many quantitative analysts wanted to know about ethnography during our open space session. We started the session with those who had done ethnographic work talking about their experiences: Stuart Geiger talked about his ethnographic work on Wikipedia bots, Isis Amelie Hjorth talked about her ethnographic enquiry into Wreckamovie, the collaborative movie outfit from Finland and Paško Bilić discussed how he studied breaking news stories on Wikipedia. Others wanted to know how you even begin to do ethnographic research on Wikipedia when editors are a) anonymous and b) located all around the world. One participant said, “I’m faced with 3 million edits (in my dataset) and I have to say something about them. How do I even begin?”

I think this comment points to the key problem that quantitative researchers are facing. Many are recognizing that after they have done the social network analysis of a particular category of articles or come to the conclusion that a particular type of editor is dominating, they start asking “why?” And understanding the “why?” requires them to delve into a grounded understanding of what is happening at the so-called “small data” level, leaving them flailing in questions about how such a small sample can say anything about a world that they have described with thousands, sometimes millions of datapoints.

As a response to such questions, Stuart Geiger referred participants to George Marcus’s work on “multisited ethnography” that uses a question-based approach to studying a particular community and I told them about Jenna Burrell’s very useful paper, “The Field Site as a Network: A Strategy for Locating Ethnographic Research” but I think the key problem was that researchers often are thinking of qualitative research merely as the same dataset “with interviews”. It was a great conversation with some really interesting comments and questions, even if some participants left saying with a smile: “It was a good discussion but I still believe that numbers tell the truth.”

The sample

If I learned anything at WikiSym this year it was how critical it is to be clear about how the sample is selected. As Jenna wrote, when I sent her an initial draft of this post:

Regardless of research approach, I always think HOW the sample was selected is the most important question…always.  The issue of sampling frame…or even in big data approaches where you aren’t necessarily sampling, what you can say/claim stems from what is inside vs. outside of your dataset (whether its a sample or a complete enumeration).

What worried me about a lot of papers dealing with “culture” and “emotion” and “historical sense making” is the over-reliance on studying only the large language Wikipedias and then using this data to make broad claims about human behavior without taking note of the limitations. Below is a list of the top 15 language editions that were often used:

English, German, French, Dutch, Italian, Polish, Spanish, Russian, Japanese, Portuguese, Swedish, Chinese, Vietnamese, Ukrainian, Catalan

Thus, leaving out more than half the world and two of the world’s continents – admittedly better than many of the papers that analysed only English Wikipedia (as opposed to any of the other 284 language versions) and often didn’t even express that in the title or abstract.

On the bright side, although Brent Hecht has also chosen only the biggest languages for his Omnipedia project, his drive to find an alternative method of comparing different language Wikipedias without using English as the “template” against which all other encyclopedias are judged is a really noteworthy accomplishment.

When I became frustrated at the fact that researchers were making broad claims about “culture” when they were studying only the large language Wikipedias (generally located in the developed world), I heard things like: “This is just the beginning. We’ll get to those in time.” (Or more often: “This is just the beginning. I’m not going to do that research because the datasets are too small but I’m happy that you will be.”) And it frustrated me because whenever “further research” was mentioned, small language encyclopedias were not, nor were questions about minority groups on Wikipedia.

This is not, in the end, an ethical question. It is a question about good research. I don’t believe we can truly understand  Wikipedia’s role in society or make the kind of broad claims that we’re seeing if we only look at a small section of the data. Researchers often complain that they can’t do broader, Wikipedians “in-the-world” kind of research because it’s too hard – Wikipedia doesn’t make users’ locations accessible unless they are anonymous (1) and there is too much data to analyse. But this perspective comes from a very particular way of seeing the world as some researchers feel that they have to analyse the same amount of data qualitatively that they analysed from a quantitative perspective when good qualitative research is much more about selecting the right kind of data, getting a grounded, participant driven perspective on the world, and truly understanding what Wikipedia means in the context of the outside world.

What aren’t we measuring?

I was surprised that no one in the panel entitled “What aren’t we measuring?” mentioned the obvious bias towards English, developing world and large language groups. The panel was interesting, however, especially since it elicited questions about how to measure Wikipedia “activity” and the fact that such activity is often equated with a small sliver of Wikipedia activity, leaving out things like offline Wikipedia activities including training and advocacy. A lot of this seems to have to do with what data is available and what tools we have to analyze it.

Which leads me to the question:

To what extent do you believe that the available tools are influencing what kinds of questions we are asking and how we are choosing to answer those questions?

Surely, if the questions were really about cultural and language differences and different ways of seeing the world we would choose a sample of languages from cultures that seemed very far apart rather than a sample of languages from cultures that are very close together? The answer is usually that the dataset for those “far-away” languages is too small,  too disorganised, too uncategorized. This seems to be a perfect example of the tools determining the research questions rather than the other way around.

In his keynote, Jimmy Wales focused on the need to focus more on African Wikipedias. He said that the Wikimedia Foundation is focusing on India and Brazil in their Global Development Program, but that African countries would be next. I always so appreciate Jimmy’s personal interest in the continent and I know that he is only one person on the Wikimedia Foundation Board, but what concerns me is that, although the Global Development Program does amazing work, very little of it is research. And sound, grounded research to understand more about the country and languages in focus would go a long way to avoiding the kinds of problems that have occurred in past – problems like rampant copyright infringement in the India pilot which might have been helped by a deeper understanding of intellectual property questions in India. A contextual understanding of copyright in developing countries and ways to frame this problem is going to be a really important research topic in the coming years as Wikipedia grows. Without it, and despite the incredible staff in the program who have learned how important it is for them to work with locals and get a better understanding of the context before they start major advocacy work, Wikipedia might be seen as just another Western Aid project and not the global project promoting tolerance and understanding that many would hope it to be. If the research on Wikipedia continues to be done on major languages and developing countries, we won’t get any closer to this kind of understanding.

The value of participant observation

The great thing about WikiSym is that there are Wikipedians in the audience who call researchers out when their findings do not equate with their own observations. Another question that came up in the conference was what it means to base your quantitative analysis in the grounded experience of Wikipedians. Being a participant in Wikipedia is one way of solving this problem: you’re more likely to recognize when your findings are incorrect if you’ve had direct experience in the behaviors that you’re counting. As Stuart said after the ethnography meetup: “We need to collaborate and keep each other accountable!”

So that’s what I’m doing – or at least trying to – with this post. When I’ve been criticized for things I hadn’t noticed or thought about in the past, it’s made my research better. And I hope that my role as critic here is somehow constructive. One researcher caught me after the conference to say that there are plenty of conferences I could go to where it wouldn’t be left to me to keep complaining about these issues. It got me thinking: where does ethnography belong? Or more accurately perhaps: Where do critical approaches that put people at the center rather than the tools belong?

Looking forward

In the closing session last year, I remember saying “I have been to a lot of conferences lately and I don’t feel like I belong. But I feel like I belong here.” People come to WikiSym because it’s the place to be if you’re doing Wikipedia work. In the words of conference chair, Cliff Lampe said, “WikiSym is the place we come where we know we don’t have to explain ourselves. Where people just “get it”.

But at the end of this year’s conference, as we stood in a circle and shared our parting thoughts, I didn’t feel like people were getting it. I said, “I was really happy and really angry and really sad and really excited…” That’s one way of saying that my initial enthusiasm has been tempered this year by my feeling like such an outsider, like the old grouch who keeps thinking that Wikipedia research has a bias towards asking very particular questions using very particular tools without a recognition of the limitations of that lens.

I’m still committed to WikiSym – at least for another year – and next year I hope to organize a panel that reflects on the type of research that gets done for WikiSym and outlines what kind of bias that represents. Because in the end I love what wikis represent: the ability to change something if there are enough people behind you. That’s one thing that has stuck with WikiSym and this year Chris Lampe did an amazing job at putting the wiki way into practice. It’s still one of the only academic conferences that I know of where there is a parallel open space track. And I guess that’s one of the ways that keeps WikiSym-ers like me coming back.

(1) anonymous users’ IP addresses are their usernames so you can look up where they resolve to.
(2) this is an interesting observation since it presumes that ethnography and counting are mutually exclusive when that is often not the case.

Fail Whale by Flickr CC BY NC SA

Stuart Geiger and I will be presenting our paper about Wikipedia literacy in Linz, Austria for WikiSym 2012 (link below). It’s in the short paper series in which we introduce the concept of of “trace literacy”, a multi-faceted theory of literacy that sheds light on what new knowledges and organizational forms are required to improve participation in Wikipedia’s communities. The paper focuses on three short case studies about the misunderstandings resulting from article deletions in the past year and relate them to three key problems that literacy practitioner and scholar, Richard Darville outlined in his English literacy research. Two of the case studies are from interviews that we did with Kenyan Wikipedians, and the other concerns the Haymarket affair article controversy. Literacy, we believe, has a lot more to do with users being able to understand the complex traces left by experienced editors and how, where and when to argue their case, than simply learning how MediaWiki syntax works.

“Writing up rather than writing down”: Becoming Wikipedia Literate H. Ford and S. Geiger, WikiSym ’12, Aug 27–29, 2012, Linz, Austria

Almost a year ago, I was hired by Ushahidi to work as an ethnographic researcher on a project to understand how Wikipedians managed sources during breaking news events. Ushahidi cares a great deal about this kind of work because of a new project called SwiftRiver that seeks to collect and enable the collaborative curation of streams of data from the real time web about a particular issue or event. If another Haiti earthquake happened, for example, would there be a way for us to filter out the irrelevant, the misinformation and build a stream of relevant, meaningful and accurate content about what was happening for those who needed it? And on Wikipedia’s side, could the same tools be used to help editors curate a stream of relevant sources as a team rather than individuals?

Original designs for voting a source up or down in order to determine “veracity”

When we first started thinking about the problem of filtering the web, we naturally thought of a ranking system which would rank sources according to their reliability or veracity. The algorithm would consider a variety of variables involved in determining accuracy as well as whether sources have been chosen, voted up or down by users in the past, and eventually be able to suggest sources according to the subject at hand. My job would be to determine what those variables are i.e. what were editors looking at when deciding whether to use a source or not?

I started the research by talking to as many people as possible. Originally I was expecting that I would be able to conduct 10-20 interviews as the focus of the research, finding out how those editors went about managing sources individually and collaboratively. The initial interviews enabled me to hone my interview guide. One of my key informants urged me to ask questions about sources not cited as well as those cited, leading me to one of the key findings of the report (that the citation is often not the actual source of information and is often provided in order to appease editors who may complain about sources located outside the accepted Western media sphere). But I soon realized that the editors with whom I spoke came from such a wide variety of experience, work areas and subjects that I needed to restrict my focus to a particular article in order to get a comprehensive picture of how editors were working. I chose the 2011 Egyptian revolution article because I wanted a globally relevant breaking news event that would have editors from different parts of the world working together on an issue with local expertise located in a language other than English.

Using Kathy Charmaz’s grounded theory method, I chose to focus editing activity (in the form of talk pages, edits, statistics and interviews with editors) from the 25th of January, 2011 when the article was first created (within hours of the first protests in Tahrir Square), to the 12th of February when Mubarak resigned and the article changed its name from ’2011 Egyptian protests’ to ’2011 Egyptian revolution’. After reviewing the big picture analyses of the article using Wikipedia statistics of top editors, and locations of anonymous editors etc, I started work with an initial coding of the actions taking place in the text, asking the question ‘What is happening here?’

I then developed a more limited codebook using the most frequent/significant codes and proceeded to compare different events with the same code (looking up relevant edits of the article in order to get the full story), and to look for tacit assumptions that the actions left out. I did all of this coding in Evernote because it seemed the easiest (and cheapest) way of importing large amounts of textual and multimedia data from the web, but it wasn’t ideal because talk pages, when imported, need to be re-formatted and I ended up using a single column to code data in the first column since putting each conversation on the talk page in a cell would be too time-consuming.

Screenshot of my Evernote desktop showing initial coding

I then moved to writing a series of thematic notes on what I was seeing, trying to understand, through writing, what the common actions might mean. I finally moved to the report writing, bringing together what I believed were the most salient themes into a description and analysis of what was happening according to the two key questions that the study was trying to ask i.e. How do Wikipedia editors, working together, often geographically distributed and far from where an event is taking place, piece together what is happening on the ground and then present it in a reliable way? And: how could this process be improved?

Ethnographymatters has a great post by Tricia Wang that talks about how ethnographers contribute (often invisible) value to organizations by showing what shouldn’t be built, rather than necessarily improving a product that already has a host of assumptions built into it. And so it was with this research project that I realized early on that a ranking system conceptualized this way would be inappropriate – for the single reason that along with characteristics for determining whether a source is accurate or not (such as whether the author has a history of presenting accurate news article), there are a number of important variables that are independent of the source itself. On Wikipedia, these include variables such as the number of secondary sources in the article (Wikipedia policy calls for editors to use a majority of secondary sources), whether the article is based on a breaking news story (in which case the majority of sources might have to be primary, eyewitness sources), or whether the source is notable in the context of the article (misinformation can also be relevant if it is widely reported and significant to the course of events as Judith Miller’s NYT stories were for the Iraq War).

This means that you could have an algorithm for determining how accurate the source has been in the past, but whether you make use of the source or not depends on factors relevant to the context of the article that have little to do with the reliability of the source itself.

Another key finding recommending against source ranking is that Wikipedia’s authority originates from its requirement that each potentially disputed phrase is backed up by reliable sources that can be checked by readers, whereas source ranking necessarily requires that the calculation be invisible in order to prevent gaming. It is already a source of potential weakness that Wikipedia citations are not the original source of information (since editors often choose citations that will be deemed more acceptable to other editors) so further hiding how sources are chosen would disrupt this important value. On the other hand, having editors provide a rationale behind the choice of particular sources, as well as showing the variety of sources rather than those chosen because of loading time constraints may be useful – especially since these discussions do often take place on talk pages but are practically invisible because they are difficult to find.

Analysing the talk pages of the 2011 Egyptian revolution article case study enabled me to understand how Wikipedia editors set about the task of discovering, choosing, verifying, summarizing, adding information and editing the article. It became clear through the rather painstaking study of hundreds of talk pages as to how editors were:

a)     storing discovered articles either using their own editor domains by putting relevant articles into categories or by alerting other editors to breaking news on the talk page,

b)    choosing sources by finding at least two independent sources that corroborated what was being reported but then removing some of the citations as the page became too heavy to load,

c)     verifying sources by finding sources to corroborate what was being reported, by checking what the summarized sources contained, and/or by waiting to see whether other sources corroborated what was being reported,

d)    summarizing by taking screenshots of videos and inserting captions (for multimedia) or by choosing the most important events of each day for a growing timeline (for text),

e)     adding text to the article by choosing how to reflect the source within the article’s categories and providing citation information, and

f)     editing disputing the way that editors reflected information from various sources and replacing primary sources with secondary sources over time.

It was important to discover the work process that editors were following because any tool that assisted with source management would have to accord as closely as possible with the way that editors like to do things on Wikipedia. Since the process is managed by volunteers and since volunteers decide which tools to use, this becomes really critical to the acceptance of new tools.

Sources work process for breaking news

After developing a typology of sources and isolating different types of Wikipedia source work, I made two sets of recommendations as follows:

1. The first would be to for designers to experiment with exposing variables that are important for determining the relevance and reliability of individual sources as well as the reliability of the article as a whole.

2. The second would be to provide a trail of documentation by replicating the work process that editors follow (somewhat haphazardly at the moment) so that each source is provided with an independent space for exposition and verification, and so that editors can collect breaking news sources collectively.

Regarding a ranking system for sources, I’d argue that a descriptive repository of major media sources from different countries would be incredibly beneficial but that a system for determining which sources are ranked highest according to usage would yield really limited results (we know, for example, that the BBC is the most used source on Wikipedia by a high margin, but that doesn’t necessarily help editors in choosing a source for a breaking news story). Exposing the variables used to determine relevancy (rather than adding them up in invisible amounts to come up with a magical number) and showing the progression of sources over time offers some opportunities for innovation. But this requires developers to think out of the box in terms of what sources (beyond static texts) look like, where such sources and expertise are located and how trust is garnered in the age of Twitter. The full report provides details of the recommendations and the findings and is available on Scribd here.

There are a variety of independent variables that determine whether or not sources are used

This is my first comprehensive ethnographic project and one of the things I’ve noticed having done other design and research projects using different methodologies is that, although the process can seem painstaking and it can prove difficult to articulate the hundreds of small noticings into findings that are actionable and meaningful to designers, getting close to the lived experience of editors is extremely important and valuable work that is rare in Wikipedia research. I realize now how before I actually studied an article in detail, I knew very little about how Wikipedia works in practice. And this is only the beginning!