9.2 Data analysis and interpretation

Data analysis is crucial for investigative reporting. It involves sourcing datasets, evaluating quality, and cleaning information to uncover hidden truths. Reporters must master these skills to transform raw numbers into compelling narratives.

Effective data visualization is key to storytelling. Choosing the right charts, applying design principles, and interpreting relationships help journalists present complex findings clearly. These techniques empower reporters to draw meaningful conclusions from data-driven investigations.

Data for Investigative Stories

Sourcing Datasets

Top images from around the web for Sourcing Datasets

• 

  Top 10 datasets - data.govt.nz View original

  Is this image relevant?

• 

  How to use screen scraping tools to extract data from the web | Opensource.com View original

  Is this image relevant?

• 

  Connecting freedom of information to open data: How to build a better FOIA.gov : Sunlight Foundation View original

  Is this image relevant?

• 

  Top 10 datasets - data.govt.nz View original

  Is this image relevant?

• 

  How to use screen scraping tools to extract data from the web | Opensource.com View original

  Is this image relevant?

1 of 3

Top images from around the web for Sourcing Datasets

• 

  Top 10 datasets - data.govt.nz View original

  Is this image relevant?

• 

  How to use screen scraping tools to extract data from the web | Opensource.com View original

  Is this image relevant?

• 

  Connecting freedom of information to open data: How to build a better FOIA.gov : Sunlight Foundation View original

  Is this image relevant?

• 

  Top 10 datasets - data.govt.nz View original

  Is this image relevant?

• 

  How to use screen scraping tools to extract data from the web | Opensource.com View original

  Is this image relevant?

1 of 3

• Obtain relevant datasets from government databases, academic research repositories, nonprofit organizations, and private sector companies
• Submit Freedom of Information Act (FOIA) requests to access government agency data not publicly available
• Employ web scraping techniques to extract data from websites lacking APIs or downloadable datasets
• Evaluate data quality by assessing accuracy, completeness, timeliness, and consistency
• Understand data collection methodologies to interpret limitations and potential biases
• Consider ethical implications in data acquisition including privacy laws, copyright restrictions, and terms of service agreements
• Network with data journalists and join professional organizations to access exclusive datasets and data-sharing opportunities

Evaluating Data Quality

• Assess dataset accuracy by cross-referencing with other reliable sources
• Check for completeness by identifying missing values or underrepresented categories
• Evaluate timeliness to ensure data reflects current conditions (Census data, economic indicators)
• Examine consistency across variables and time periods within the dataset
• Investigate potential biases in data collection methods (survey design, sampling techniques)
• Review metadata and documentation to understand data limitations and appropriate uses
• Consult subject matter experts to validate data quality and relevance to the investigative story

Cleaning and Organizing Data

Data Cleaning Techniques

• Identify and correct errors, inconsistencies, and inaccuracies in raw datasets
• Handle missing data through imputation methods, deletion of incomplete records, or statistical estimation
• Standardize data formats for consistency across variables (date formats, units of measurement)
• Address outliers by determining if they represent genuine anomalies or data errors
• Apply data transformation techniques like normalization and scaling for specific analytical methods
• Create a data dictionary defining variables, formats, and coding schemes to maintain data integrity
• Implement version control and document data cleaning steps for reproducibility and transparency

Data Organization Strategies

• Structure data in a tabular format with consistent column headers and row identifiers
• Remove duplicate entries to prevent skewed analysis results
• Merge multiple datasets using common identifiers (join operations)
• Split complex variables into separate columns for easier analysis (full names into first and last name)
• Create calculated fields to derive new insights from existing data (BMI from height and weight)
• Establish a clear file naming convention and folder structure for organized data storage
• Implement data validation rules to maintain data integrity during future updates or additions

Data Analysis for Trends

Descriptive Statistics

• Calculate measures of central tendency including mean, median, and mode
• Determine measures of dispersion such as range, variance, and standard deviation
• Compute percentiles and quartiles to understand data distribution
• Use frequency distributions to summarize categorical data
• Apply cross-tabulation to examine relationships between multiple variables
• Calculate ratios and rates to standardize comparisons (crime rates per capita)
• Identify skewness and kurtosis to characterize the shape of data distributions

Advanced Statistical Methods

• Conduct inferential statistics to draw conclusions about populations based on sample data
• Perform hypothesis testing to evaluate claims about population parameters
• Calculate confidence intervals to estimate population values within a range
• Apply correlation analysis to measure strength and direction of relationships between variables
• Utilize regression analysis to examine impact of independent variables on a dependent variable
• Employ time series analysis to identify patterns and trends in data collected over time
• Implement cluster analysis to group similar data points and reveal patterns within datasets

Data Visualization for Storytelling

Chart Selection and Design

• Choose appropriate chart types based on data characteristics and story objectives
• Use bar charts for comparing categorical data (comparing sales across product categories)
• Employ line graphs to show trends over time (stock price fluctuations)
• Utilize scatter plots to visualize relationships between two continuous variables
• Create pie charts to display proportions of a whole (market share analysis)
• Design heat maps to show patterns in large datasets (geographic distribution of crime rates)
• Implement small multiples to compare multiple related charts side by side

Visualization Principles and Tools

• Apply principles of data visualization including clarity, simplicity, and accuracy
• Utilize color theory to enhance readability and convey information effectively
• Develop interactive visualizations allowing readers to explore data dynamically
• Create infographics combining data visualizations with contextual information
• Consider accessibility in data visualization for visually impaired audiences
• Use tools ranging from spreadsheet programs (Excel, Google Sheets) to specialized platforms (Tableau, D3.js)
• Incorporate responsive design principles for visualizations across different devices and screen sizes

Data-Driven Conclusions

Interpreting Data Relationships

• Distinguish between correlation and causation when analyzing relationships in data
• Identify potential confounding variables influencing observed relationships
• Contextualize findings within broader scope of existing research and knowledge
• Acknowledge limitations of data and analysis methods used in the investigation
• Develop narratives effectively communicating significance of data findings to non-technical audiences
• Anticipate and address potential counterarguments or alternative interpretations of the data
• Consider ethical implications in presenting conclusions, avoiding sensationalism and maintaining objectivity

Validating and Communicating Findings

• Cross-validate results using different analytical methods or datasets
• Conduct sensitivity analysis to test the robustness of conclusions under different assumptions
• Seek peer review or expert consultation to validate findings and interpretations
• Develop clear and concise summaries of key findings for different stakeholder groups
• Use analogies and real-world examples to explain complex data concepts to general audiences
• Provide transparent documentation of data sources, methodologies, and analytical processes
• Prepare responses to potential criticisms or challenges to the data-driven conclusions