Research Methods

There's a statement about A/B testing that gets repeated so confidently that nobody pushes back on it. It's not wrong, exactly. It's just not as true as people think. "Random assignment ensures the groups are equivalent." This gets said in every A/B testing primer, every experimentation course, every stakeholder meeting where someone asks "but how do we know the groups are comparable?", and some of the comments on my LinkedIn posts. And it's... almost true. Random assignment ensures the groups are equivalent in expectation, meaning if you repeated the randomization thousands of times, the average difference between groups on any variable would be zero. In stats language, we would say, "There's no systematic bias." Practically, we can be confident that nobody is cherry-picking who gets treatment. But you don't run thousands of randomizations. You only run one. And one randomization is one draw from that distribution. Big sample? The draw is almost certainly fine. Small sample? You can get groups that look nothing alike, and the randomization didn't fail. That's just how probability works at small N. I ran a simulation. Take 40 people with a known covariate — say, prior engagement score — and randomly split them 20/20. Do it 500 times. Some splits are nearly perfect. Others are off by more than half a standard deviation. Every single one of those is "correctly randomized." Some of them will absolutely give you misleading results if you don't deal with it. Do the same thing at 1,000 per group and the distribution of imbalances basically disappears. That's the Law of Large Numbers doing its thing. But n=20 isn't large, so you can't count on LLN to save you. So what do you actually do when your experiment isn't huge? 𝗖𝗵𝗲𝗰𝗸 𝗯𝗮𝗹𝗮𝗻𝗰𝗲 𝗯𝗲𝗳𝗼𝗿𝗲 𝘆𝗼𝘂 𝗹𝗼𝗼𝗸 𝗮𝘁 𝗼𝘂𝘁𝗰𝗼𝗺𝗲𝘀. Compare the groups on everything you measured pre-treatment. Age, tenure, prior usage, whatever. If something is meaningfully off, you need to know before you interpret results. 𝗦𝘁𝗿𝗮𝘁𝗶𝗳𝘆 𝗼𝗿 𝗯𝗹𝗼𝗰𝗸 𝘁𝗵𝗲 𝗿𝗮𝗻𝗱𝗼𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻. If you know certain variables matter, force balance on them upfront. Randomize within strata. 𝗔𝗱𝗷𝘂𝘀𝘁 𝗳𝗼𝗿 𝗯𝗮𝘀𝗲𝗹𝗶𝗻𝗲 𝗰𝗼𝘃𝗮𝗿𝗶𝗮𝘁𝗲𝘀 𝗶𝗻 𝘁𝗵𝗲 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀. Be smart about using information you already have, and it almost always gives you tighter estimates. (But make sure you're not adding in colliders or post-treatment effects!) And if you ran a small experiment without doing any of this — just be honest about it. The results might be fine. But you're trusting luck more than you think. Randomization solves the systematic bias problem. It doesn't solve the bad luck problem. Those are different things, and small experiments are exactly where the difference shows up.

How do we reveal the true direction of an effect? Meta-analysis gives us the details. In our third session of the Systematic Review & Meta-Analysis series, in partnership with Schobot AI. We walked through a sequence of operational steps that form the internationally accepted framework for any high-quality meta-analysis: 1️⃣ Define the research question using PICO/PECO Transform the problem into measurable elements: Population, Intervention/Exposure, Comparison, and Outcome. 2️⃣ Include quantitative studies only We accept studies that provide: t-statistics (from t-tests or regression) • F-values • β coefficients • Odds ratios or risk ratios • Correlation coefficients (r) • Means, standard deviations, and sample sizes These values are then converted into a unified effect size, most commonly: ✔️r (correlation coefficient) ✔️Fisher’s Z ✔️SMD (Hedges g / Cohen’s d) ✔️log OR Such as experimental, quasi-experimental, longitudinal, and cross-sectional designs. 3️⃣ Pool results using Fixed or Random Effects models -Fixed-effect when studies share a highly similar context. -Random-effects when contexts differ Typically more appropriate in economic, managerial, and social research. → The output is a pooled estimate that reflects the true direction and size of the effect. 4️⃣ Assess heterogeneity Using: • Cochran’s Q to test for the presence of heterogeneity. • I² to quantify its magnitude (low – moderate – high). 5️⃣ Conduct a Risk of Bias assessment We applied tools to ensure evidence integrity: → RoB 2 for randomized trials → ROBINS-I for non-randomized studies → JBI Checklists for observational designs These tools evaluate study design, sampling, measurement quality, missing data, and control of confounding variables. 💡 Risk of bias assessment is critical because a single flawed study can distort the entire pooled outcome. 6️⃣ Evaluate the certainty of evidence using GRADE We explained how the GRADE framework strengthens transparency by rating evidence according to: ↳ Study quality ↳ Consistency of findings ↳ Precision ↳ Applicability ↳ Risk of bias The final rating classifies evidence as: High – Moderate – Low – Very Low Certainty Meta-analysis does not only tell you whether an effect exists. It reveals its direction, strength, consistency, and level of certainty after cutting through the noise of individual studies. Stay tuned! Next session, Hands-on implementation of all steps in R-Studio. 💾 Save this post to revisit later! ➕ Follow Dr. Saleh ASHRM for deeper insights

As academics, we all want our research to be trusted, reproducible, and strong enough to withstand review. Yet most of the problems we face during publication come from one place: weak statistical foundations and unclear experimental design. This is why I want to give you a quick, practical guide you can use to strengthen any study you are planning or refining. These principles are simple, but they prevent the most common errors I see across manuscripts, reviews, and collaborations. 1. Statistics is not about numbers. It is about reasoning. Each test, each calculation, tells a story about your data and what it truly means. 2. Experimental design begins with purpose. Define your objective clearly before you begin collecting data. The design should flow naturally from the research question. 3. Randomization protects integrity. Assign treatments randomly to eliminate bias and ensure valid comparisons. 4. Replication increases confidence. Repeating experiments strengthens conclusions and helps distinguish real effects from noise. 5. Control groups matter. They provide the baseline that gives your results meaning. Without controls, interpretation becomes speculation. 6. Choose tests based on data, not habit. Understand whether your variables are categorical, continuous, or ordinal. Then select the statistical method that fits the data, not the one that feels familiar. 7. Interpret, do not just report. Numbers are not the end of the story. Explain what they mean, why they matter, and how they support or challenge your hypothesis. 8. Visuals clarify understanding. Use tables and graphs to reveal patterns and relationships, but keep them clean, accurate, and purposeful. 9. Ethical analysis is non-negotiable. Never manipulate data to fit a narrative. Transparency and honesty sustain the credibility of your research. 10. Statistics and design are partners. Good design minimizes errors. Good statistics reveal the truth within them. One without the other cannot stand. These principles are not theoretical. They are the difference between a study that moves quickly through review and a study that struggles with rejection, uncertainty, or inconsistent conclusions. Download the full PDF below. Do you think your current research would benefit from this guide? Reply and tell me. I would love to know. ______________________________ 📌 This is Prof. Samira Hosseini. I’ve helped 12,000+ ambitious academics go from struggling with publishing papers in Q1 journals, limited visibility, and poor citation records to building a solid research trajectory and high 𝘩-index. Book a free Strategy Call, and we can dive into your challenges in top-tier journal publication and citation and see how I can best assist you: https://lnkd.in/ezqV64dX

What is a Pilot BE Study? A Pilot BE Study is a preliminary clinical investigation conducted before the pivotal BE study. Its primary role is to assess whether the test formulation performs comparably to the reference product and to guide adjustments in formulation, dosing, or study design before committing to a larger, regulatory-submissible pivotal study. Objectives of Study The pilot BE study serves multiple purposes. It evaluates the feasibility of achieving bioequivalence between the test and reference formulations. It is also used to optimize the formulation or manufacturing process if initial results suggest improvements are necessary. Moreover, the pilot study helps assess intra-subject variability and residual error, which is critical for determining the required sample size for the pivotal study. It also assists in selecting the most appropriate dosage strength, especially when multiple strengths are available. Additionally, it helps estimate the test/reference (T/R) ratio to ensure it is likely to fall within acceptable regulatory limits, and to refine the blood sampling schedule to capture key pharmacokinetic parameters like Cmax, Tmax, and AUC. Key Features of Study Typically, a pilot BE study involves a small number of subjects—usually around 6 to 12. The results are not intended for regulatory submission but instead inform the design of the pivotal study. The study design is most often a 2-way or 3-way crossover, and it is usually conducted as an open-label study. While the pharmacokinetic parameters analyzed are the same as in a pivotal study, the primary purpose is to make a go/no-go decision for progressing to a full-scale BE study. Regulatory Notes Regulatory agencies such as the USFDA do not mandate pilot BE studies, but they are recommended—especially for high-risk, complex, or modified-release formulations.EMA takes a similar position and views pilot studies as useful tools for formulation optimization. In India, the CDSCO also permits and encourages pilot BE studies to support the planning of pivotal BE studies. When Are Pilot Studies Crucial? Pilot studies become particularly important in several scenarios. For new or complex formulations, they help confirm that the in vivo performance aligns with expectations. When high variability is expected in drug absorption, pilot studies help estimate this variability for better planning of the pivotal study. They are also essential for modified-release or narrow therapeutic index drugs, where precise absorption control is critical. Lastly, when it is the first time a generic formulation is being tested in humans, a pilot study provides foundational data to proceed confidently. Limitations Despite their value, pilot BE studies have limitations. They are not acceptable for regulatory filing due to the small sample size and exploratory nature. Furthermore, results from pilot studies may not always predict the outcomes of pivotal studies due to limited statistical power.

Phd scholars: Anatomy of a publishable quantitative paper. Every strong result rests on strong design. Reviewers see logic before they see data. I recall how one of my research papers overflowed with analysis. But it lacked rhythm + coherence + story. Once I structured it right, reviewers called it “clear and rigorous.” ▶ Title: What: Concise title naming variables and population. Why: Signals it’s quantitative and measurable. How (Action): ➟ Include main variables and context ➟ Keep under 15 words ▶ Abstract: What: Snapshot of the study (problem, method, key results, conclusion). Why: Lets readers grasp relevance and validity fast. How: ➟ 1–2 lines background/problem ➟ 1 line objective/hypothesis ➟ 1 line method (design, sample, analysis) ➟ 2–3 lines key statistical results ➟ 1 line conclusion/significance Word count: 150–250 words ▶ Introduction: What: Problem, rationale, objectives, and hypotheses. Why: Builds logic from background to research question. How: ➟ 1 paragraph describing the issue/phenomenon ➟ 2–3 paragraphs reviewing literature and theory ➟ 1 paragraph stating research gap ➟ 1 paragraph outlining objectives/hypotheses Word count: 800–1,000 words ▶ Methods: What: Design, population, sampling, instruments, data collection, and analysis. Why: Ensures replicability and validity. How: ➟ 1 paragraph: research design (survey, experiment, correlational, etc.) + justification ➟ 1–2 paragraphs: population, sampling, and sample size determination ➟ 1 paragraph: instrument description (reliability and validity) ➟ 1 paragraph: data collection procedures ➟ 1–2 paragraphs: variables and measurement scales ➟ 2–3 paragraphs: statistical analysis (tests used, software, significance level) ➟ 1 paragraph: ethical approval and consent Word count: 1,200–1,500 words ▶ Results: What: Statistical findings organized around hypotheses or research questions. Why: Shows patterns, relationships, and significance. How: ➟ Present descriptive statistics first ➟ Follow with inferential results (t-test, ANOVA, regression, etc.) ➟ Use tables and figures clearly labeled ➟ Report p-values, effect sizes, and confidence intervals ➟ Avoid interpretation here, stick to what was found Word count: 1,800–2,500 words ▶ Discussion: What: Interpretation, implications, and limitations. Why: Explains meaning and connects findings to theory. How: ➟ Summarize main results briefly ➟ Compare with prior studies ➟ Explain why findings occurred ➟ Highlight contributions to theory, practice, or policy ➟ Note study limitations and future directions Word count: 1,500–1,800 words ▶ Others: What: Conflicts of interest and funding. Why: Ensures transparency. How : ➟ Declare conflicts (or none) ➟ Mention funding source and its role Word count: 50–100 words ➟ References: follow journal format ♻️ Find this useful? – Like + comment – Repost to help a struggling researcher 🔔 Follow Edidiong Ukpong(PhD Architecture) for more

𝗠𝗮𝗿𝗸𝗲𝘁𝗶𝗻𝗴 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝘀 – 𝗣𝗮𝗿𝘁 𝟱 “Running Experiments” doesn’t mean that you go and do a bunch of random crap and “see if it works”. There are people claiming to be "experts" at marketing experiments (one was just recently on a widely listened-to podcast) who simply have no clue what they're doing or talking about. A few things to watch for from fake "experts in running experiments" are: 👉 They talk about "experiments failing"...if so, they're likely NOT doing experiments. The only way an experiment "fails" is if it lacks sufficient data to clearly accept or falsify the hypothesis being tested. 👉 They never talk about the Model that the Experiment is based on or the specific Hypothesis the experiment is directly testing. 👉 They never talk about counterfactuals...the difference in outcomes between Scenario A and Scenario B in the experiment. 👉 They NEVER frame the experiment in terms of trying to answer any sort of "Compared to What?" question. 👉 They never mention key aspects that EVERY experiment has to have, such as a "Treatment Group" and a "Control Group" or critical practices like the use of "Hold Out Groups" or randomized selection. 👉 They never talk about testing the outcome against a "Null Hypothesis" (which simply measures the likelihood that the results could have happened entirely by chance) or testing for any sort of "significance" or "confidence" levels in the final data. The problem with just randomly "doing stuff" is that it lacks the structure to make any sort of "causal analysis." Did the thing you're testing actually cause the outcome you observed, or would that outcome have happened anyway? This is the ever-critical "Compared to What?" question that experiments ABSOLUTELY MUST answer to qualify as an actual valid experiment (vs. just doing random crap) --- Here's the basic MINIMAL structure needed to do a valid marketing experiment. This diagrammed below in the attached graphic. You have to start with some sort of "Model". As marketers, we NEVER directly interact with the world...we interact with beliefs and theories about how we think the world works. How marketing and pricing work, how buyers make buying decisions, how advertising influences those buyers, and how various marketing channels work. For many marketers, these are IMPLICIT (and often flawed) beliefs and assumptions that are typically never consciously examined. A “Model” is just a simplified description of how we think the world works. We then exact Hypotheses from the Models. Simply statements about what should happen in the real world based on our Model being correct. An Experiment has ONE purpose...to test the validity of our Models by testing if what we should expect to see happen (Hypotheses) does, in fact, match what actually happens in the real world. An “Experiment” is simply a controlled way to gather data that allows us to develop a causal understanding of whether our Hypothesis (and Model) is correct or not.

As statistical reasoning becomes central to evidence-based social science, this document offers a structured and accessible pathway into the world of quantitative research. It does not merely explain data analysis—it provides a complete learning journey from conceptual grounding to practical application with SPSS and Stata. M&E professionals, social researchers, and policy analysts are invited to move beyond theoretical assumptions and toward empirical testing, hypothesis validation, and data-driven interpretation. Here, statistics is not a secondary skill—it is a core language for understanding patterns, measuring change, and informing action. – It defines the foundations of empirical social research, including concepts, theories, hypotheses, and variables – It explains the full cycle of quantitative research, from question formulation to regression analysis – It introduces survey design principles, pre-testing, and implementation across various contexts – It presents univariate, bivariate, and multivariate statistical techniques with guided SPSS and Stata exercises – It includes step-by-step instructions for data visualization, hypothesis testing, and result interpretation – It illustrates real-world applications through student-led projects and group survey initiatives – It emphasizes the cumulative, falsifiable, and replicable nature of scientific social inquiry – It supports both classroom and self-directed learning with exercises, assignments, and accessible examples Blending theoretical clarity with applied structure, this guide equips readers to build, test, and interpret social science models with precision. Each chapter strengthens the analytical toolkit needed to transform abstract questions into measurable insights. More than a textbook, it is a foundational companion for statistical thinking and practical research across disciplines.

My supervisor rejected my literature review 3 times. The 4th time, it got approved without a single comment. Here's the exact 7-step framework that changed everything 👇 ――――――――――――――――― The mistake most researchers make? They summarize papers. A literature review is not a summary. It's a synthesis — and there's a massive difference. ――――――――――――――――― 𝟭. DEFINE RESEARCH SCOPE Write a 2-3 page outline first. Cover: research question, knowledge gaps, target audience, significance. No scope = no direction. No direction = rejection. 𝟮. LITERATURE SEARCH Build a systematic search strategy. Database selection + keyword mapping + inclusion criteria. If your search isn't structured, your review won't be either. 𝟯. CRITICAL ANALYSIS For every key paper: assess methodology, evaluate findings, score quality. Don't just read. Judge. 𝟰. FINDINGS SYNTHESIS Create a synthesis matrix. Identify themes → map relationships → note contradictions → spot gaps. This is exactly where most researchers fall apart. 𝟱. RESEARCH BLUEPRINT Build a 5-7 page outline showing your theoretical framework, argument structure, evidence mapping, and research positioning. Write the skeleton before the body. 𝟲. WRITING AND REVISION First draft → peer review → supervisor review → final manuscript. Multiple drafts aren't weakness. They're seriousness. 𝟳. ACADEMIC ENGAGEMENT Share via conferences, journal submissions, seminars, and networks. Research that stays in your drawer helps no one. ――――――――――――――――― The researchers who get published don't write more. They write smarter. Save this framework. Your next literature review will thank you. 🔖 Which step do you struggle with most? Drop it below 👇 #LiteratureReview #AcademicWriting #ResearchWriting #PhDLife #Dissertation #Thesis #ResearchMethodology #ScientificWriting #AcademicSuccess #PublishOrPerish #ResearchTips #HigherEducation #ResearchCrave #PeerReview #JournalPublication #TheoreticalFramework #KnowledgeGap #GrowthMindset #ResearchSkills #LifelongLearning

Sample Size Guidance - How do you calculate a sample size for a pilot or a feasibility study? First, some definitions: - Pilot Study: A version of a main (Pivotal) study run in miniature to test whether the components of the study can all work together. - Feasibility Study: Research done before a main study to answer the question "Can this study be done". The difficulty of calculating a sample size for a pilot or feasibility study is that there is (typically) no hypothesis test. Without a hypothesis to test, there is no type 1 or type 2 error, and without the type 2 error, there is no sample size calculation. Recall that we calculate sample sizes to estimate the statistical power we can achieve and recall that statistical power is our ability to avoid a type 2 error - be able to reject the null hypothesis when it is warranted to do so. So what to do? What should we use to inform our sample size in our pilot or feasibility study? Let's start by answering the question - who is the study for? These early studies are 100% FOR THE SPONSOR. They are meant to provide the sponsor with the information they need to make a go/no-go decision on later studies or to help inform them with evaluating the current performance/development status of their product. With this in mind, the sample size should be driven by what the sponsor needs to achieve their goal (product development/improvement, go/no-go decisions, etc). So should there be no math involved? Not necessarily - you, as the statistician can still provide the sponsor with direction and insight by doing the following: - You can calculate a sample size as if there was a hypothesis to test so the sponsor gets an idea of what will be needed in the pivotal study - You can produce example confidence intervals based on the sample size the sponsor is thinking and a standard alpha value of 0.05 (so create a 95% CI) - You can help identify reference studies that have been completed previously to give a good semblance of what is used to achieve similar objectives In general, I typically see pilot and feasibility studies at or around 50 patients per arm but this is not a general rule - this is just my observations over the years. I wish you all the best with your pilot and feasibility studies and if you need any assistance with them, please don't hesitate to reach out. Happy Monday

Your AI may give different answers - and why that matters: Most of us have noticed it: ask a chatbot the same question twice, and you get slightly different answers. That’s not just “randomness”. Even when we set the system to be deterministic (temperature 0), the results can still change. Horace He and the Thinking Machines Lab recently just published an important piece explaining why this happens and how to fix it - and I finally had time to read it. The article is called: "Defeating Nondeterminism in LLM Inference." The research matters because reproducibility is a cornerstone of science, and it should be for AI. If an LLM gives a different answer each time, we can’t fairly assess student work, audit decisions, or run reliable experiments. The team set out to find the true cause of this “nondeterminism” and discovered it’s not just GPU math quirks or concurrency issues, but how the system groups and processes user requests in batches. The output you get can depend on who else is using the system at the same time. If the batch size changes based on server load, the math that underlies your model’s answer can change, even if your question and settings are identical. What this means for education and policy: Imagine School ABC is using an AI tutor that automatically grades essays. At 8:00 AM, a student submits their essay and receives a B+. At 8:05 AM, another student submits the exact same essay but gets an A-. Both results are “technically correct” given the math, but the outcome is inconsistent and unfair. Or consider a district running an AI-powered reading assessment: If the model’s recommendations shift subtly depending on server traffic, a child could be flagged as “below grade level” one day and “at grade level” the next. These differences aren’t just academic. They can have real-world consequences for student placement, hiring decisions, or compliance audits. What is the solution? The research team demonstrates how to make models “batch-invariant,” meaning that the results no longer depend on how many other requests are being processed. In their experiments, once batch invariance was achieved, 1000 identical prompts produced 1000 identical completions. Exactly what deterministic mode should deliver. What to do now? Demand reproducibility: When procuring AI systems, ask vendors if their models are batch-invariant and whether they guarantee deterministic inference. Audit fairness impacts: For schools and HR, test your system by running identical inputs at different times of day and checking for result drift. Push for transparency: Document model, version, and settings in any decision-making workflow so that outcomes can be explained and defended. Determinism isn’t a luxury. It’s a prerequisite for trust. Without it, we risk building AI systems that are not just probabilistic but arbitrary. This research shows that we can get consistent results if we are willing to prioritize them, even at a small performance cost.