Skip to Main Content

Tutorial: Food Stories

A class guide for the tutorial class "Food for Thought"

Evaluation is Essential

In our modern world, the challenge most researchers face is not finding resources, but evaluating resources. Creating convincing but inaccurate informational resources is a breeze and we are so inundated with information that its often difficult to take the time to spot sophisticated or unintentional misinformation. Everyone can be fooled by misinformation, so even experts need to closely evaluate the information they incorporate into their research. 

There are lots of ways to evaluate literature and the core of all the methods is a high level of critical thinking. The AACODS checklist, created by Jess Tyndall, is an example of an evaluation model that relies on the reader's critical thinking skills. Below are the areas the model uses to evaluate information. The questions are intended to serve as a place to start when considering the multitude of factors that can impact information accuracy. Ask the questions about the source you are using to determine whether a source is worthwhile. 

Authority

Authority

Why should we listen to what they are telling us? 

Overview

When deciding which information to believe, use, or trust, we must first look at who created the information and decide if they are in a position to give us this information. 

Authority is Constructed.

How we evaluate authority depends greatly on societal constructs.  Authority is constructed in that various communities may recognize different types of authority. 

Authority is Contextual

The context in which the information is gathered, created and used affects the information giver's authority. For example, a kindergarten teacher has authority on a subject within her classroom to her students, but not when speaking to a group of adults in bar about the same subject. 

Questions to ask

  • Are they associated with a reputable organization?
  • Do they have professional qualifications or considerable experience?
  • Have they produced/published other work in the field?
  • Are they recognized expert, identified in other sources?
  • Are they cited by others?
  • Are they a higher degree student under “expert” supervision?
  • Is the information about the author's lived experience? If so, how can we check its accuracy? Or, can we responsibly use it in our research? 

 

Accuracy

Accuracy

What  has the information creator done to ensure the information is correct?


Overview

 When evaluating information, we want to see what the creator of the information has done to ensure the accuracy of the information being presented. Things like peer-reviewing, correct and findable citations, clear explanations of how the information was gathered, and quick publishing of corrections can bluster a creator's reputation of accuracy. 

Questions to Ask

  • Does the item have a clearly stated aim or brief? If so, is this met?
  • Does it have a stated methodology? If so, is it adhered to?
  • Has it been peer-reviewed?
  • Has it been edited by a reputable authority?
  •  Is it supported by authoritative, documented references or credible sources?
  • Is it representative of work in the field? If not, is it a valid counterbalance?
  • Is any data collection explicit and appropriate for the research?
  •  If the item is secondary material (e.g. a policy brief of a technical report) refer to the original document. Is it an accurate, unbiased interpretation or analysis?
  • Can you find this information in other places? 
  • Are there recent reports or news claiming that the authors or the organization are inaccurate in their reporting? 
  • Does this document have citations to other works? 
  • Are the sources for the information named? 
  • If the document uses statistical statements or data visualizations, can the reader find the data set or the methods of how the data was collected?
  • Is the information presented in this document radically different from what other sources report? Do they have solid, verifiable information to back up these contrary claims? 
  • Look at older publications from the author or organization about different topics or events. Are these publications accurate? Does this creator have a track record of accuracy? 
  • Does the organization publish retractions or corrections to their documents? This could mean that they are dedicated to making sure their information is accurate, even if it means admitting they were previously inaccurate. 

 

Coverage

Coverage

What does this information apply to?

Overview

Anytime we are evaluating information, we need to be aware of the scope and berth of the information. It is difficult to make broad, sweeping, and general statements while still maintaining accuracy. Facts can change considerably based on the time, place, circumstances, involved population groups etc of the event. For example, statistics for asthma rates are going to be different if we are looking at a specific neighborhood versus an entire country or for the year 1953 versus the year 2023. Even scholarly works, made by the foremost experts on very niche areas, rarely make definitive statements and often conclude that more research needs to be done to fully understand the topic. How can we verify or compare information if we don’t know what exactly the information applies to? 

Questions to Ask

  • Are any limits clearly stated?
  • Is this document making sweeping, absolute statements?
  • What specific geographic area, situation, time period, legislation, etc. is this document reporting on? 
  • Is the focus of this document the same as the topic of our research? Or is it about something else and only making a tangential mention of our topic? If so, the statements made by the document on our target topic are not likely suitable for use in our research. 

Objectivity

Objectivity

How can biases affect this information?

Overview

Deciding the objectivity and biases of an information provider is perhaps the most important aspect of evaluating information in any context. Everyone has bias and that bias affects what we believe and what we tell others. Researchers will need to ask themselves what the author’s or organization’s agenda or mission is outside of the information in the document we are evaluating. Keep in mind that people's potential biases are rarely stated or acknowledged, so those looking to evaluate information will have to do some digging. 

Questions to ask

  • Opinion, expert or otherwise, is still opinion: is the author’s standpoint clear?
  • Does the work seem to be balanced in the presentation?
  • How does the author or the organization get their funding? Could they be altering the information for more sales, ad revenue, online views, increased donor activity etc.? 
  • What do they gain from getting me to believe what they are saying? (Political support, funding, paid endorsement, more views, etc) 
  • Why would someone publish false or misleading information about this topic? 
  • Does this author or organization have a track record of being objective or biased?  
  • Is the document written with passionate language and strong statements? 
  • What is the personal, educational, and professional history of the author? Are their past experiences coloring how they see this issue? 

 

Date

Date

How does time or timing affect this information?

 

Overview

Almost all information changes over time. More facts come to light, viewpoints change, potential biases are uncovered as time passes. Knowing the timeframe our information covers as well as when the literature was written and published is important to evaluating the resource. We need to know the context for the information which can be gauged from the time it was collected and published. For example, a pro-industrial organization may use a statistic about drops in pollution to bolster a claim about how factories are not harming the environment. However if the data was collected during the height of the 2020 pandemic, when quarantine stopped travel and limited factory operations, the claim would be misleading. On the other hand, we have to acknowledge that some information degrades and becomes less accurate over time. Researchers need to critically evaluate how time of publication will change the information presented. 

Questions to ask

  • Does the item have a clearly stated date related to content? No easily discernible date is a strong concern.
  • If no date is given, but can be closely ascertained, is there a valid reason for its absence?
  • Check the bibliography: have key contemporary material been included?
  • How fast-paced is the subject area? For example, information from 5 years ago in the field of Machine Learning will be obsolete today, while information about Anne Frank published 5 years ago would likely still be relevant. 
  • What events or changes in technology have occurred since this document was published? Would they change how this information is presented? 
  • What is the most recent news about this topic? Is it drastically different from what was published in the document? 
  • Has the organization/author published something more recently on the topic? 
  • If data and statistics are used, does the document state when was this information collected? 

Significance

Significance

What is the impact of this information to the field and to us as researchers?

Overview

Researchers need to critically consider the significance of the information they are evaluating in relation to the topic they are researching. On one hand, why bother with an item of literature that doesn't add anything new to the topic? (Keep in mind that adding a diverse viewpoint that has been historically ignored does add value to a topic even if it reiterates known information.) However, some of the most impactful and salacious information ends up being misinformation. Popular media can make insignificant events or data seem life-altering, and we only realize years later that it was not impactful. Most importantly, we need to decide if our source has any significance on the topic we are researching specifically. An article that addresses a different topic can be taken out of context and force to fit our topic, but that doesn't mean the information is valid in the context of our research. 

Questions to Ask

  • Is the item meaningful? (this incorporates feasibility, utility and relevance)
  • Does it add context?
  • Does it enrich or add something unique to the research?
  • Does it strengthen or refute a current position?
  • Would the research area be lesser without it?
  • Is it integral, representative, typical?
  • Does it have impact? (in the sense of influencing the work or behavior of others)"
  • What "new" thing is this information telling us? 
  • Does this document make significant claims that go against what other documents tell us? 
  • Can the claims in this document reasonably be applied to our research topic? For example, statistics about elementary school attendance cannot be reasonably applied to research about high school attendance. 
  • Are there scholarly sources that provide this information? 
  • Is this document referenced often by scholarly works on this topic? 
  • What effect did this piece of grey literature have on the world? For example, a US Supreme Court ruling will have significant outcomes for the US. 
  • Does this document provide a diverse viewpoint or a place for diverse voices that may not be covered in scholarly works?