Master probability vs. non-probability sampling techniques, sample size justification, benchmarking data collection, and sampling bias mitigation.
Part 1: Sampling Techniques Breakdown (Practice Question 7)
M5-Q01[PRACTICE Q7] [DEFEND & COMPARE]
Discuss the main differences between Probability Sampling and Non-Probability Sampling. Why do researchers use sampling instead of testing the entire population? [6 Marks]
NUST Model Answer [6 Marks]:
Why Researchers Sample:
Saves time and financial costs during data collection.
Reduces hardware, network, and human resource strain.
Makes large-scale system evaluations feasible when analyzing whole population datasets (e.g. millions of network packets) is impractical.
Core Distinction:
Probability Sampling: Every element in the population has a known, non-zero chance of selection. Enables statistical generalizability and eliminates selection bias.
Non-Probability Sampling: Selection relies on researcher judgment, convenience, or explicit criteria. Used when generalizability is secondary to depth or accessibility.
M5-Q02[CLASSIFY & GROUND]
Detail TWO types of Probability Sampling and THREE types of Non-Probability Sampling with concrete CS research scenarios. [10 Marks]
NUST Model Answer [10 Marks]:
A. Probability Sampling Techniques:
1. Simple Random Sampling: Every item has equal probability. CS Example: Randomly sampling 1,000 incoming network packets out of 100,000 for intrusion detection analysis.
2. Stratified Sampling: Population is divided into non-overlapping sub-groups (strata) and random samples are drawn proportionally. CS Example: Stratifying students by academic year (1st, 2nd, 3rd, 4th year) before surveying adoption rates of an AI study tool.
B. Non-Probability Sampling Techniques:
1. Convenience Sampling: Selecting participants/data readily accessible to the researcher. CS Example: Testing an app interface on 10 classmates sitting nearby. (High risk of sampling bias!).
2. Purposive / Judgmental Sampling: Selecting subjects based on specific expertise or characteristics. CS Example: Interviewing senior Cybersecurity Engineers with > 10 years experience to evaluate zero-trust architecture.
3. Snowball Sampling: Initial participants refer additional participants who meet the criteria. CS Example: Researching dark-web ethical hackers by asking initial contacts to introduce peers.
M5-Q03[APPLY & SOLVE]
BENCHMARK DATA COLLECTION SCENARIO: You are collecting execution latency data to evaluate an openEuler char driver experiment. What controls must be placed during data collection to ensure high data integrity?
NUST Model Answer [5 Marks]:
Environment Isolation: Disable dynamic CPU frequency scaling (governor set to performance mode) and stop background cron jobs.
Warm-up Runs: Run 100 untracked warm-up iterations to pre-fill CPU caches before recording latency logs.
Sample Size & Iterations: Record at least 1,000 execution runs to compute mean, standard deviation, and 99th percentile latency ($p_{99}$).
Automated Raw Data Logging: Log raw timestamps directly to CSV/JSON format without manual transcriptions to prevent human error.