I like the idea of trusted networks of people working together (actually, it seems that this technique has been used for decades in the activism community) but I have some problems with the term that has been “coined”. I guess I will be called a “muggle” but I am willing to take the plunge because a) I have never been called a “muggle” and I would like to know what it feels like and b) the “crowdsourcing” term is one I feel is worthy of a duel.

Firstly, I don’t agree with the way that Meier likens “crowdsourcing” work like Ushahidi to statistical methods. I see why he’s trying to make the comparison (to prove crowdsourcing’s value, perhaps?) but I think that it is inaccurate and actually de-values the work involved in building an Ushahidi instance. Working on an Ushahidi deployment is not the same as answering a question through statistical methods. With statistical methods, a researcher (or group of researchers) tries to answer a question or test a hypothesis. ‘Do the majority of Hispanic Americans want Obama to win a second term?’ for example. Or ‘What do Kenyans think is the best place to go on holiday?’

But Ushahidi has never been about gaining a statistically significant understanding of a question or hypothesis. It has been designed as a way for a group of concerned citizens to provide a platform for people to report on what was happening to them or around them. Sure, in many cases, we can get a general feel about the mood of a place by looking at reports, but the lack of a single question (and the power differential between those asking and those being asked), the prevalence of unstructured reports and the skewed distribution of reporters towards those most likely to reply using the technology (or attempting to game the system) make the differences much greater than the similarities.

The other problem is that the term lacks a useful definition. Meier seems to suggest that the “bounded” part refers to the fact that the work is not completely open and is limited to a network of trusted individuals. More useful would be to understand under what conditions and for what types of work different levels of openness are useful, because no crowdsourcing project is entirely “unbounded”. Meier says that he ‘introduced the concept of bounded crowdsourcing to the field of crisis mapping in response to concerns over the reliability of crowd sourced information.’ But if this means that “crowdsourced” information is unreliable, then it would be useful to understand how and when it is unreliable.

If we take the very diverse types of work required of an Ushahidi deployment, we might say that they include the need to customize the design, build the channels (sms short codes, twitter hashtags, etc), designate the themes, advertise the map, curate the reports, verify the reports, find related media reports, among others. Once we’ve broken down the different types of work, we can then decide what level of openness is required for each of these job types. I certainly don’t want to restrict the advertising of my map to the world, so I want to keep that as “unbounded” as possible. I want to ensure that there are enough people with some “ownership” of the map to keep them supporting and talking about it, so I want to give them some jobs that keep them involved. Tagging reports as “verified” is probably a more sensitive activity because it requires a set of transparent rulesets and is one of the key ways that others come to trust the map or not. So I want to ensure that trusted people, or at least those over whom I have some recourse, do this type of work. I also want to get feedback on themes and hashtags to keep it close to the people, since in the end, a map is only as good as the network that supports it. Now if I have different levels of openness for different areas of work, is my project an example of “bounded” or “unbounded” crowdsourcing?

Although I am always in favor of adding new words to the English language, I feel that the term “unbounded crowdsourcing” is unhelpful in leading us towards any greater understanding of the nuances of online work like this. Actually, I’m always surprised at the use of the term “crowdsourcing” over “peer production” in the crisis mapping community since crowdsourcing implies monetary or commercial incentivized work rather than the non-monetary incentives that characterised peer production projects like Wikipedia (see an expanded definition + examples here). I can’t imagine anyone ever “coining” the term “unbounded peer production” (but I seem to be continually surprised, so I should completely discount it from happening) and I think that this is indicative of the problems with the term.

So, yes, if we’re talking about different ways of improving the reliability of information produced on the Ushahidi platform, I’m excited to learn more about using trusted networks. I just think that if a term is being coined, it should be one that advances our understanding of what the theory is here. Is it that: if you restrict the numbers of people who can take part in writing reports, you get a more reliable result? Where do you restrict? What kind of work should be open? What do we mean by open? Automatic acceptance of Twitter reports with a certain hashtag? Or an email address that you can use to request membership? Is there a certain number that you should limit a team to (as the Skype example suggests)?

This “muggle” thinks that the term doesn’t get us any further towards understanding these (really important) questions. The “muggle” will now squeeze her eyes shut and duck.

Cross-posted from blog.ushahidi.com

As Ushahidi ethnographer, my job is to do on-the-ground research on users’ experience with our technology in particular contexts. Something that we’ve been thinking about a great deal as we develop SwiftRiver is the process of verification, the ways in which technology and society work together to create useful, trustworthy and actionable information, as well as where the technology in particular contexts might be failing.

With over 20,000 installations of Ushahidi and Crowdmap since January, 2009, Ushahidi has been used in a number of different contexts – from earthquake support in Haiti, to reports of sexism in Egypt, to election monitoring in the Sudan. In each of these cases, a map is publicized and individuals are encouraged to send reports to it. The process of verifying information reported by the crowd has taken on a variety of different forms depending on the needs and affordances of the environment and the community supporting it.

The memo I just published on scribd introduces the concept of verification, how it has evolved at Ushahidi and in sample deployments, alternative ways of thinking about verification and some suggestions for further research. Its goal is to inform developers and designers as they develop the next generation of Ushahidi and SwiftRiver software to meet the needs of our users rather than prescribing what should be done.

Ushahidi support for verification has until now been limited to a fairly simple backend categorisation system by which administrators tag reports as “verified” or “unverified”. But this is proving unmanageable for large quantities of data and may not be the most effective way of portraying the nuanced levels of verification that can practically be achieved with crowdsourced data.

What research needs to be done to test verification alternatives so that Ushahidi and Crowdmap deployers are provided with due diligence tools that can advance trust in their deployments? Can we do this in a way that doesn’t add any new barriers to entry to those who need to have their voice heard on Ushahidi? How can we ensure that this solution is as close as possible to the needs, incentive systems and motivations of deployers and users? What is the next step for Ushahidi verification?

Wikipedia articles on breaking news stories dominate page views on the world’s sixth-largest website. Perhaps more importantly, these articles drive the most significant editor contribution — especially among new editors.

In the first three months of this year, English Wikipedia articles with the most contributors were the 2011 Tucson shooting, the 2011 Egyptian revolution and the 2011 Tōhoku earthquake and tsunami articles with 460, 405 and 785 editors contributing to the growth of the article respectively.

Interestingly, a number of Wikipedia policies discourage writing articles on breaking news. One of Wikipedia’s 42 policies, titled “What Wikipedia is not” (or WP:NOT), highlights that the site is, above all, an encyclopedia, not a newspaper (Wikipedia:NotNewspaper). The policy states that although the encyclopedia needs to include current and up-to-date information as well as standalone articles on “significant current events,” not all verifiable events are suitable for inclusion in Wikipedia.

Wikipedia articles are not journalism

According to the policy, “Wikipedia should not offer first-hand news reports on breaking stories” because “Wikipedia is not a primary source.” The encyclopedia has a tenuous relationship with primary sources. Policy states that primary sources, “accounts written by people who are directly involved in an event, offering an insider’s view of an event” are (mostly) inappropriate because Wikipedia strives to represent a “Neutral Point of View” (NPOV), and primary sources can be misused to reflect a fringe theory as mainstream. NPOV is one of the five pillars of Wikipedia and frames to a large degree what is allowed into the encyclopedia and what is left out.

News reports on a breaking news story require that Wikipedians use primary sources to update the rapidly evolving articles on issues like death counts after an earthquake. While journalists are able to use primary sources to make a judgment on the death count at the time of publishing and then do the same using new sources when they write successive stories, Wikipedians must do the same collectively and iteratively as new versions are created every few seconds.

In the Japanese earthquake article, this challenge resulted in contradictory facts about the height of the tsunami and the death tolls in the same article, prompting one editor (“Dcoetzee”) to create templates for the number of missing and the dead casualties that could be edited once with changes immediately reflected in every part of the page (see Keegan, Gergle and Contractor’s Hot off the Wiki: Dynamics, Practices, and Structures in Wikipedia’s Coverage of the Tohoku Catastrophes).

Wikipedia articles are not news reports

The barrier to entry into Wikipedia articles is notability: Subjects must be notable enough to create enduring articles on the encyclopedia. According to policy, while news reporting covers announcements, sports news or celebrities, the fact that something is “in the news” is not a sufficient basis for inclusion in the encyclopaedia. Notability is difficult, perhaps impossible to predict directly after an event, and can result in historical events being described in purely modern terms or an article being created about something noteworthy at a particular time which later might not meet notability requirements.

Wikipedians call this “recentism” and have a tag to make it transparent to readers that the article might be skewed towards “recent perspectives.” In an essay on “recentism,” Wikipedians describe the phenomenon as “writing or editing without a long-term, historical view, thereby inflating the importance of a topic that has received recent public attention.”

Both the “Wikipedia articles are not: Journalism” and “Wikipedia articles are not: News reports” policies recommend moving timely news subjects to WikiNews, a sister project to Wikipedia that allows use of primary sources and is intended to be a primary source. But WikiNews has suffered from a low contributor base and disagreement among contributors about the best way to build the news portal.

In September, a large portion of the Wikinews contributor base announced on the Foundation-l mailing list that they had forked the project and started OpenGlobe” after becoming deeply dissatisfied with Wikinews.”

Wikipedia articles are not who’s who

The third item of “Wikipedia:NotNewspaper” explains that, even when an event is notable, individuals involved in it may not be. This policy speaks to the need for enduring articles that will still be notable in the years after the event. While newspapers are often concerned with explaining events through the people affected by such events, Wikipedia wishes to take the long-term view, attempting to avoid cases that give undue weight to the person or event and thus conflict with NPOV.

 

The first rough draft of history?

It took just 11 minutes for the Japanese Wikipedia to create an article after the 9.0-magnitude undersea megathrust earthquake occurred off the coast of Japan on March 11. Twenty-one minutes later, the English Wikipedia article was created, and although the wire services reported the earthquake within minutes, The New York Times did not file a full story until more than three hours after the earthquake hit.

Despite the distinct discouragement of reporting on current news item for reasons mentioned above, Wikipedia has become the site of major activity around large news events like this one. The ability of anyone to edit the encyclopedia and the lack of any restrictions on editing articles, as well as the fact that notability is a relative concept, means that Wikipedia policy cannot stop the hundreds of editors who flock to the encyclopedia driven by a single purpose to work on a particular page.

But if Wikipedia and not the news media is the first rough draft of history, what does this mean for Neutral Point of View? If Wikipedians are evaluating and synthesizing primary sources rather than sources who have already evaluated the importance of an event, is Wikipedia at the risk of becoming subjective? Consensus may be more easily achieved when the event is a natural disaster, but when it’s a war or a revolution and the editors’ motivations are different, then the same architectural flexibilities can lock articles into disagreement.

Wikipedia may be a reluctant journalist, but its influence on the media landscape is unmistakable.

I arrived in Nairobi last night after an absence of about five years. As I left the plane through the walkway, I took a deep breath and inhaled the familiar southern African smell that I always miss so much living in America. I walked through to customs and baggage claim and to my taxi and hotel and became aware of all the things I was noticing: my slight frustration at the absence of instructions about which line to stand in at the immigration hall; the fact that there was not enough room for my place of birth in the immigration paperwork; the fact that, in stark contrast to the Amsterdam Schiphol Airport that I had come from, this airport seems not to have changed in a decade or so.

I noticed how long we had to wait for our bags to come through, the nationalities of the people coming here, how closely they stood next to one another. And my driver, patiently waiting for me, familiar sign in hand. On the car ride to the hotel, I looked at billboards and noticed what was being advertised and who was being represented, the state of repair of the roads and the roadside flowers and how people drive and the smells of food and industry and bodies.

I thought: Is this the collection of noticings that constitutes a place? And if what defines a place is its signposts, its boundaries, the taken-for-granted ways of doing things, the expected and the unexpected, what are the equivalents in online spaces? How do we know that we have left one space and arrived at another? How does the experience of outsiders (or n00bs) differ from that of locals?

This new way of thinking about social media (new for me, at least) came about when I was asked to speak at a conference about the ‘crucial role of social media’ in the Middle East and elsewhere. Buried in the description of the session was the question: ‘Does what happened in the London Riots diminish the power of social media?’ As I thought about what to say and what was expected of me, it struck me that the problem with the current way questions around social media are framed is that they require defining technological artefacts as good or bad, when it might be more appropriate to talk about technology as a place where good and bad things can, and do, happen.

If we frame social media as places, we can understand more fully the role of people in those places, rather than talking about the technical characteristics of Facebook or Wikipedia as determining a particular type of behaviour. Looking only at the “bad” privacy features of Facebook, for example, we are tempted to assume that “privacy is dead” because of the “forced sharing” that is happening through changes in the technology. But this view fails to represent the ways that people self-censor or move to more intimate spaces in order to protect their privacy, something I noticed in my study of privacy in an educational context, for example.

Framing social media as places enables us to realise how we move between platforms (for example, Facebook and Google+) not only because of the new shiny gadgets we find there, but because of the people who inhabit those spaces. It is the flow of people and practices that defines the place as much as it is its landscape and architectural features. Facebook, for example, is defined by particular boundaries (my page, your page, a photograph that belongs to a particular group), taken-for-granted ways of doing things that define deviance and compliance among particular groups (don’t friend your teacher, don’t send too many updates and flood your friends’ streams, don’t tag drunk pictures of friends) and artefacts (the activity stream, wall and photo albums) that, taken together, define the place.

It seems kind of obvious when you think about it, and it isn’t a new way of thinking about technology: we’ve been talking about going online and migrating from different operating systems for a while. But the fact that we’re surprised that Google+ isn’t currently teeming with people, or that more Kenyans aren’t contributing to Swahili Wikipedia, or that women make up such a small percentage of Wikipedia edits suggests that we are thinking too much of social media as things rather than as places. If we thought about Google+ as a big, shiny, new complex, we’d begin to understand that people won’t necessarily move there just because the technology is better when few of their friends are there.

The key aspect that we miss in thinking of social sites as technological artefacts is that we tend to ignore culture and power – two really big and slippery aspects of what makes certain types of people have certain types of conversations in particular online spaces, and of what defines who feels welcome or unwelcome to participate. It has caused us to define Wikipedia or Facebook at a level of granularity that isn’t deep enough to really get an understanding of what is happening there, where the power is located and how we might engineer to encourage particular creations and conversations. This is not just about understanding the affordances of the software. In order to understand Wikipedia collaboration, I can’t only look at the MediaWiki software – in the same way that to understand Kenya, I couldn’t just read about its legal framework or look at the statistics about the country. Being there, experiencing how people to speak to me, noticing what the signposts say and what they leave out, is part of the necessarily long journey toward a full understanding of the place.

Perhaps most importantly, it is the culture of a place that will dominate my decision to come back or not. And this, in its essence, is at the heart of what every online community seeks, and is the same reason why it’s so hard to control. The government of Kenya can build better roads and speak on television about being welcoming to tourists, for example, but the majority of the experience of being in Kenya as a tourist or a local, is outside of government control. Culture, we find out, is a mysterious mix of so many different qualities in varying proportions that act together to define a place. Understanding culture is probably more art than science, but however we learn about it, it’s an important part of what makes us stay in some places to become loyal nationalists or merely return as tourists.

Rachelle Annechino and I are recent graduates of the School of Information at UC Berkeley. We met one cold, sunny summer day in August (only in San Francisco!) when I arrived to find friends to learn Python with. Rachelle and I meet to co-work and chat at Brown Couch Café in Oakland where we talk about fascinating bits and pieces from our lives and work. For her final project, Rachelle and her project partner, Yo-Shang Cheng interviewed San Francisco residents and asked them to draw pictures of their internal images or “mental maps” of the neighborhoods they lived in and of the city as a whole. They then visualized these mental maps according to concepts like ‘corridors’ (where are the hearts of each neighbourhood?), ‘barriers’ (is it really that close? It’s not always as simple as it looks getting from one neighbourhood to another in San Francisco) and ‘boundaries’ (what neighbourhood are you in? according to whom?). Rachelle is simply one of the most insightful, brilliant people I know. And she rocks at Python – which makes her a good friend to have.

I met Jenna Burrell when Rachelle and I took her Qualitative Research Methods class last year. Jenna has been doing research on Internet use in Ghana for the past decade or so and was one of the most inspiring teachers that I had at the I School. Jenna’s forthcoming book ‘Invisible Users: Youth in the Internet Cafes of Urban Ghana’ is an incredibly rich contribution to our understanding of African Internet culture. Mostly when I think of Jenna, I think of the fact that while I was in Accra speaking in staid conference rooms during the Africa preparatory conference for the World Summit on the Information Society in 2005, Jenna was also out talking to young Ghanaians in Internet cafes and in the streets who were disconnected from a discussion which was ostensibly about them. Jenna is an incredible mentor and her writing about ‘The Fieldsite as a Network’ has been so helpful in thinking about how to ‘do’ digital ethnography. She continues to push the boundaries of the discipline and ask important questions about how digital technologies might become part of the grassroots, self-organizing efforts of populations marginalized from the global economy.

Tricia Wang was introduced to me in one of Jenna’s classes when Jofish Kaye suggested I read about the work she had done on Internet censorship in China. I looked her up and just knew we would be friends. Tricia’s critique of the Google China debacle and her calling for Google to employ more ethnographers in order to better understand the Chinese internet culture was so powerful, and her PhD work on migrant workers is inspiring to say the least. As I write this, Tricia is in China doing her fieldwork, sleeping in Internet cafes and accompanying migrant workers as they move through the city. She’s trying to understand how the use of technology changes how people interact with the physical city, a concept she calls’ digital urbanism on the margins’: migrants’ urban lives mediated through communications technologies like mobile phones and computers in Internet cafes.

And then there’s me, the budding ethnographer, finding herself lucky to know these incredible people and looking forward to the little journey we’re going to go on at this site. Ethnography Matters will be a place where we can share what we’re reading and writing about, how we’re thinking about ethnography, and hopefully giving a little insight for others who are thinking about a career in ethnography into what this even means today. We’ll have others join us in the future, and if you’re interested in contributing, please let us know. We’re looking forward to walking around in your shoes too!

Our key findings include:

1. About half of all deleted articles from June ’07 to Jan ’11 were unilaterally deleted by administrators via the CSD process.
2. Surprisingly, spam, vandalism and patent nonsense make up only 8.00%, 5.69% and 5.36% of CSDs respectively, while the more subjective ‘No indication of importance’ makes up 38.47% of all CSD criteria.
3. With some outliers, AfD discussions have few participants, and those participants are overwhelmingly regulars to the process. 74% of all AfDs are made up entirely of users who have previously participated in an AfD, and 18% of all AfDs only have one newcomer. You can read more on the PDFs below but there’s also a lot of great research by other authors at WikiSym and on from the Wikimedia Foundation’s Summer of Research program.

Poster PDF |  Participation in Wikipedia’s Article Deletion Processes (WikiSym accepted poster research) PDF

This morning I read a wonderful article by Tamar Ashuri from Ben-Gurion University for an upcoming edition of the journal New Media and Society entitled ‘(Web)sites of memory and the rise of moral mnemonic agents‘. Ashuri looked at how two websites set up by Israelis – one to monitor human rights of Palestinians at Israeli checkpoints; the other to collect testimonies of Israeli soldiers who served in the Occupied Territories – act as agents of collective memory. Ashuri argues that digital networked technologies is challenging the mechanisms that society employs to deny memories of immoral acts and how the online archives created by moral witnesses become a space of living memory and a sphere of moral engagement.

Ashuri explains that ‘collective memory is a social necessity; neither an individual nor a society can do without it.’ She quotes from Kansteiner (2002: 180) to describe how collective memory is different from history:

Collective memory is not history, though sometimes made from similar material [...]. It can take hold of historically and socially remote events but it often privileges the interest of the contemporary. It is as much a result of conscious manipulation as unconscious absorption and it is always mediated.

Ashuri describes how Avishai Margalit distinguishes between “common memory” (a group of people who recall a certain episode that each of them experienced) and “shared memory” (which requires communication). Shared memory is is not just an aggregate of individual memories because it requires those who remember the episode to come together to create one version (or at least a few version) through an active presentation and retelling of a story that Margalit terms ‘a division of mnemonic labor’ (2002: 52). Margalit wrote that whereas in traditional society there was a direct line from the people to their priest, storyteller or shaman, shared memory in modern society ‘travels from person to person through institutions, such as archives, and through communal mnemonic devices, such as monuments and the names of streets’ (2002: 52). Ushuri posits a new term “joint memory” to describe a new type of memory that is a ‘compilation of personal histories made public for the public’ (2011: 4). She argues that digital networked technology is challenging the exclusive role of professional mnemonic agents designated by the church, state, monarchy etc.

Significantly, joint memory is not motivated by personal interests – the desire to tell an interesting story or reveal new information – but is driven by a social purpose: Witnesses who add their recollections to an accessible and shareable compilation of memories attempt to expose events that the default collective (such as the nation) denies or wishes to forget.

I don’t agree that such reporting is not motivated at all by personal interests but I do agree with the fact that the social/moral purpose of witnessing is really critical here. Ashuri builds on Margalit’s conception of a moral witness whose testimony is ‘essentially driven by a moral purpose. It reflects hope for the witness to be a social agent who, in testifying to his or her harsh experience, transforms (passive) addressees into active audiences’. She says that what is happening now is slightly different because the moral witness now performs memories of suffering experienced in a public space.

In my conceptualisation, the (moral) mnemonic agent is the one who recalls his or her memories regarding events in whcih others have suffered and by that act of witnessing renders this suffering visible and hence difficult to marginalize or deny. The moral aspect of this act, in my estimation, derives from the content of the mnemonic text (testimony about suffering inflicted by evil) and from its objective (calling on the audience to shed their observer garb and re-enact the experience of the harsh realities). (p5)

I think this is really useful way to think about how websites like Ushahidi as well as engagement on social media sites like Twitter are acting as platforms for this kind of performance and the communication of suffering, and how this is one way of looking at how collective narratives about the world are being wrested from those who traditionally controlled this in the Middle East. Whether it is reporting on human rights violations in Saudi Arabia, harassment of women in Egypt on Harassmap  or reports of arrests and casualties in Syria I think that looking at the maps through the lens of moral witnessing and Ashuri’s “joint memory” could be a wonderful entry point for re-thinking Ushahidi’s role and effectiveness in the world.

The big questions that I’m trying to answer in this research are:

1. What debates are Wikipedians having around sources and what does this say about how Wikipedians understand the verifiability policy?
2. What is the effect of the technical features and affordances of current wiki tools on issues of quality (for example, the ability of Wikipedia to be a current and accurate source of information on rapidly-evolving events) and diversity (how Wikipedia might incorporate a wider range of viewpoints that may be situated outside of traditional academic publications)?
3. How might alternative policies and tools affect those principles of quality and diversity?

These are great questions! And great questions are always a good place to start. But deciding on how to answer these questions with limited time and resources is where I need to get creative and hopefully ask for help from wise academics, Wikipedians and friends. These are some of my initial ideas and as a newbie ethnographer I’m hoping for some kind but realistic responses.

I was originally going to start off with a bunch of interviews of Wikipedians working on topics related to big international events. But after chatting to a number of Wikipedians at Wikimania and doing a few more open-ended interviews, I’ve actually realised that starting with particular articles and telling the story of those articles could actually get me to a better understanding of what’s going on in “source talk” a lot quicker. I think that I was initially caught up in the regular social science and usability methods of conducting research where you decide on your sample, go out and collect responses to a specific set of questions and then analyse the data. Spending the past two days reading and analysing talk pages for the ‘hummus’ articles in English, Hebrew and Arabic has made me realise that there is a wealth of incredible data about how Wikipedians actually talk about sources and that a better approach could be more like a detective – starting with the little pieces of evidence and then following the story as I interview the characters reflected in the articles.

Like a detective (or at least the ones in the movies) I’m working towards understanding motivations in order to piece together narratives about what happened and why. When I first thought of Wikipedia article debates, I envisioned a large boardroom table with people sitting around it rationally discussing what to add and what to leave out. Actually, the debates are much more like noisy town hall meetings. You have the crazy person who keeps shouting all sorts of completely irrelevant details and then complaining that they’re being systematically ignored, the exhausted public administrator who has seen the same arguments play out over and over again and who is snippy and terse when newcomers try to cover old territory. There’s the polite newbie who is surely too polite to be making genuine statements, and the loud Westerners who drop into the meeting to make sarcastic remarks about how stupid everyone is for fighting about such trivial matters. I think that these narratives — who is allied to who, what happens to the debate when related national or international events hit, how disruptive editors can deadlock entire articles — are actually at the heart of bigger questions about verifiability and that, especially when we’re thinking of designing new tools to fit into current working methods of users and not the other way round, then understanding exactly how the articles tick makes sense to me.

I started by printing out articles and talk pages for ‘hummus’ in WP English, Hebrew and Arabic (Google translated version at least) and did some rough analysis and coding (using present participles to denote what I thought was happening) plus notes relating to what was interesting in comparison to the other language versions. I will go over this again and then type up themes with related quotations and summarised stories, then follow up leads with some of the editors who were involved, code all of their interviews, add to the thematic groupings and then do the final analysis.

After I’ve done the same thing for a page relating to an international news event (either the 2011 Egyptian Revolution or the 2011 Japanese Earthquake which I have started to look at), I’m hoping to be able to make some good conclusions about how Wikipedians understand verifiability and what the effect of current policies and tools are on issues of quality and diversity. You’ll notice that I’m choosing to go deep rather than wide but in order to really ‘people’ this analysis and understand who is behind the pages and what the dynamics are, I’m thinking that this might be the best way of going about it.

Would love any (kind) thoughts, suggestions and even, yes, encouragement in my lonely space over here

Christian Jensen, Povilas Pilkauskas and Thomas Lefevre are behind the WRS and have just published ‘Classification of Recommender Expertise in the Wikipedia Recommender System’ detailing the next version of their prototype. Jensen et al explain that the key problem of collaborative filtering systems is that a single user has varying levels of expertise in different areas and will therefore be good at rating some articles and not-so-good at rating others. You may have provided an accurate rating of an article about drag racing, for example, but that does not mean that you should automatically be believed when you provide feedback about painters from the Italian Renaissance. The first version of WRS worked with this kind of simplistic recommendation system and the team wanted to advance it to take account of a user’s expertise in different areas.

Now comes the hard part. In order for the system to take note of which topics users were proficient in, they had to decide what classification system they were going to use for all the information on Wikipedia (i.e. all “human knowledge”). They looked at the hundreds of classification schemes in order to find the “best” – “best” here being one that was intuitive, easy-to-use, complete, concise, useful and most importantly, unambiguous. If they were going to match up a user’s scores in a particular area, they needed to make sure that they were defining that area accurately enough, and on an encyclopedia like Wikipedia where there are no definitive categories for articles, this would prove pretty difficult.

They ran a bunch of tests to see how users would classify articles on Albert Einstein, for example, which could be classified as natural science by a physics student but as a biography by a history student. The online survey, completed by 130 people from “different countries and continents” (not sure which) asked informants to read four Wikipedia articles and categorize them according to one of four classification schemes. The classification schemes included categories from Wikiportals, Citizendium, the Dewey Decimal Classification scheme and the Open Directory Project (Dmoz), with Dmoz proving least ambiguous to users.

They then outline the new version of the WRS prototype that now includes an assessment of the expertise of recommenders according to the Dmoz scheme. They consider how reputation is affected by the two parameters: ratings and categories. If the user and the recommender agree on the quality of the article (quality is the primary parameter) but disagree on the category (category is the secondary parameter) then they apply a +1/2 value to the interaction. If the user and recommender disagree on both the rating and the category, then they consider the majority of the other recommenders and apply a -1/2 penalty to the interaction. If the majority agrees with the user, then the trust dynamics will be scored as -3/2; if the majority agrees with the recommender, then a score of -1/2 will be applied.

It’s an interesting paper and the algorithms are always fascinating but its important to realise that these systems are limited and should not hold too much power over us. The Dmoz categories a very specific and limited framework (arts, business, health, recreation, games etc) whereas our individual and communal expertise and interest often spans very different categories. I may be an expert in biographies but be weak in scientific theories; I may be expert at a broad range of topics from my country but understand little about topics in the same broad fields from other countries. When looking at designing similar systems for Ushahidi we need to make sure that there is an opportunity for feedback beyond the limited categories as well. That means encouraging people to make notes about why they rate one source more highly than another in a particular context and situation. It’s good to start somewhere and refine as we go along, and so projects like the WRS are incredibly helpful as we go down this road.

Sir John Daniel, President and CEO of the Commonwealth of Learning

Sir John Daniel from the Commonwealth of Learning was interviewed by Creative Commons’ Timothy Vollmer recently about his ideas on open education. He is one of the wisest, most gracious members of this community, and I just loved some of the answers.

Many of the COL member states are located in the global south. How does an OER policy affect global south states differently than the global north?

I’m exaggerating quite a bit here, but we’ve observed that in the north people are more focused on producing OER and that in the south people are more focused on how they can use OER. Just a few months ago I was at the Open Courseware Conference in Boston. Perhaps three-fourths of the presentations there focused on producing OER, while only a small number were about re-purposing and reusing OER content. This has to change for the OER movement to take off.

In the south, there’s a cautious attitude of “there’s lots of stuff available, why not use it?” We’ve been encouraging the north to take a more universal approach and think multidirectionally. This is why we’re delighted that a school like the University of Michigan is using OER from Malawi and Ghana in its medical programs. Why should the University of Michigan create OERs about tropical diseases when there are folks that live in the tropics that can do it better? So, we encourage people to see OER production and use as a multi-directional flow.

Yes! Open education should certainly be about mutual sharing and cooperation! It should be about breaking down barriers that copyright industries have reinforced over the years, rather than amplifying one party’s voice at others’ expense. I think that Sir Daniel’s observations are accurate and an unfortunate trend in the OER movement which is often masked by claims about quality and access.

Open source and open access are often explained in terms of the benefits of sharing to the individual (after all, we talk about Open Educational Resources as products, things, artifacts, consumables, rather than Open Educational Systems, for example). Copyright licenses reinforce this because they’re applied to a specific work and there is little focus on an ecosystem of sharing. I guess this also has to do with the way in which we measure “how much information” we possess and how we understand quality. In the West, there is “so much information” and so we need to share it with those “in need”. But if information is truly unlike physical objects, we should see the Internet as a way not just to transfer information goods from one place to another, but to share resources the other way too. If we cared as much about cooperation and understanding, then OERs would look a lot different. Right now, one-way sharing is valued more than two-way sharing. But is it necessarily more valuable to give someone a gift continuously than to humbly receive as well? How about a structure for mutual sharing rather than a model for helping us to more easily consume?

I also really loved his answer to the question regarding international cooperation.

The work of intergovernmental organizations (IGOs) is very important, but to the outside observer it is sometimes not apparent what IGOs do. What does COL do to “encourage and support governments and institutions to establish supportive policy frameworks to introduce practices relating to OER”?

If I may be so bold, I think your question reflects an American bias. The United States and other large, powerful countries tend to operate bilaterally. Smaller countries prefer the facilitative, collaborative approach of working via intergovernmental organizations. UNESCO is the extreme example, where 193 countries operate democratically, and everyone’s voice is at least in principle equal. When I worked at UNESCO, I was surprised how seriously the member states took the recommendations that were developed. They trust that sort of process more than directives that come at them bilaterally.

You can read the whole thing here.