Material Phase 5: Data Analysis, Statistics & Qualitative Synthesis

Complete study guide for Course Unit 5. Master quantitative inferential statistics, parametric vs. non-parametric tests, Python SciPy analysis, and Braun & Clarke's 6-stage qualitative thematic analysis.

1. Quantitative Analysis & Hypothesis Testing

Quantitative analysis uses mathematical and statistical methods to summarize benchmark logs, user response metrics, and system execution timings.

Null Hypothesis ($H_0$)

States that there is NO statistically significant difference or effect between experimental groups. Example: $H_0: \mu_{\text{New Algorithm}} = \mu_{\text{Baseline}}$.

Alternative Hypothesis ($H_1$)

States that a statistically significant difference exists. Example: $H_1: \mu_{\text{New Algorithm}} < \mu_{\text{Baseline}}$.

p-value & Significance Level ($\alpha$)

Standard threshold is $\alpha = 0.05$ (5%). If $p < 0.05$, reject $H_0$ and conclude that observed improvements are statistically significant, not due to chance.

Choosing the Correct Statistical Test

Research Context	Parametric Test (Normal Dist.)	Non-Parametric Test (Skewed / Ordinal)
Comparing 2 Independent Groups (e.g., Redis vs. Memcached latency)	Independent Samples t-test	Mann-Whitney U Test
Comparing 2 Paired Groups (e.g., Memory before vs after compression on same server)	Paired Samples t-test	Wilcoxon Signed-Rank Test
Comparing 3+ Groups (e.g., Latency across 4 database engines)	One-Way ANOVA (Analysis of Variance)	Kruskal-Wallis Test
Testing Relationship Between Categorical Variables	N/A	Chi-Square Test ($\chi^2$)

⚠️ Exam Solver Rule:
If sample size $N < 30$ or data fails normality testing (Shapiro-Wilk test $p < 0.05$), you MUST use non-parametric tests (e.g. Mann-Whitney U instead of t-test) to maintain statistical validity.

2. Qualitative Analysis & Braun & Clarke's 6-Stage Thematic Analysis

Qualitative data (interview transcripts, developer survey open-ended comments) is analyzed using Thematic Analysis to identify patterns of meaning (themes).

Stage 1: Familiarization: Transcribing interviews, reading and re-reading text, noting initial ideas.
Stage 2: Generating Initial Codes: Systematically coding interesting features across the entire dataset in NVivo or spreadsheet software.
Stage 3: Searching for Themes: Collating codes into broader candidate themes (e.g., "Developer Friction", "Authentication Latency").
Stage 4: Reviewing Themes: Checking candidate themes against coded extracts and the entire dataset for consistency.
Stage 5: Defining & Naming Themes: Defining the exact scope and narrative of each theme.
Stage 6: Writing the Report: Selecting vivid quote extracts and relating analysis back to the research questions.

💡 Distinction Secret: NVivo Software vs Python Analysis

Quantitative telemetry logs $\rightarrow$ Processed using Python (Pandas, SciPy, Matplotlib) or R.
Qualitative transcript audio & texts $\rightarrow$ Coded using Qualitative Data Analysis (QDA) software like NVivo or Atlas.ti.