Subjects: Medicine, Pharmacy >> Clinical Medicine Subjects: Statistics >> Biomedical Statistics submitted time 2022-11-21
Abstract： Sample size determination for clinical trials is one of the key components of study design. Based on medical device registration review recently published by National Medical Products Administration, Center for Medical Device Evaluation, and other public information, we conducted an analysis of the sample size for medical device registration clinical trials, including study design, part of which being compared with that in the US. Our results showed that the median sample size for Class III medical device registration trials is 120 (IQR 90~167.5). Sample size was influenced significantly by regulation policies, and some differed significantly from that in the US. Disclose of registration review is a giant leap for medical device regulation in China; however, the disclosed information needs to be further improved.
Peer Review Status:
Awaiting Review
Subjects: Statistics >> Biomedical Statistics submitted time 2022-05-02
Abstract：
"
Peer Review Status:Awaiting Review
Subjects: Psychology >> Experimental Psychology Subjects: Psychology >> Psychological Measurement submitted time 2024-03-04
Abstract： Statistical power is one of the key indicators for assessing the robustness and replicability of research results. However, the standardization and completeness of calculating and reporting statistical power in event-related potential studies still need improvement. This paper aims to provide researchers with references for calculating and reporting statistical power during the design or preregistration of research protocols at various stages of event-related potential studies by summarizing the influencing factors, methods, and application examples of statistical power in such studies.
Peer Review Status:Awaiting Review
Subjects: Psychology >> Social Psychology submitted time 2023-03-27 Cooperative journals: 《心理学报》
Abstract： Generalizability Theory (GT) is widely applied in psychological measurement and evaluation. A larger generalizability coefficient often indicates a higher reliability the test may have. Generalizability coefficients can be improved by increasing sample sizes. However, the size of a sample would be subject to budget constraints. Therefore, it is important to examine how to effectively determine the size of a sample considering the budget constraints. The existing literature has been largely limited to traditional methods, such as the differential optimization method, the Lagrange method and the Cauchy Schwartz inequality method. These traditional methods have limited scope of application and their typical conditions are hard to satisfy. In addition, there is no unified comparison available. Fortunately, with the increased use of high performance computing, the Constrained Optimization Evolutionary Algorithms (COEAs) becomes highly feasible. This paper expands and compares the four methods—the differential optimization method, Lagrange method, Cauchy Schwartz inequality method, and COEAs—determine the best solution to the optimal sample size problem under the budget constraints in GT. Specifically, this paper compares the applicability of the four methods using three generalizability designs, including p × i × r, (r: p) × i and p × i × r × o designs. The results are presented as follows: (1) In the optimization performance of two-facet generalizability design of p × i × r and (r: p) × i, the performance of COEAs is slightly better than that of the traditional methods, whereas the performance of three traditional methods is equivalent. Although COEAs and the traditional methods have showed similar accuracy, the former has better compliance concerning budget constraints. (2) In the optimization performance of three-facet generalizability design of p × i × r × o, the performance of COEAs is obviously better than that of the traditional methods. The least ideal generalizability coefficient is obtained using the differential optimization method, whereas its budget compliance is the best; the generalizability coefficient obtained by Lagrange method is the best, but higher than the budget. The Cauchy inequality method obtains a better generalizability coefficient under special budget constraints. But, the performance of COEAs is slightly better than that of Cauchy Schwartz inequality method, especially closer to the budget constraints. (3) In terms of the algorithm complexity, COEAs obtains an obviously smaller algorithm complexity than do the traditional methods. The complexity of the three traditional methods is relatively high. However, COEAs does not rely on the derivation of mathematical formulas, and the algorithm is relatively less complex. (4) In terms of the algorithm applicability, COEAs is significantly better than the traditional methods. The applicability of the three traditional methods is relatively narrow. However, COEAs does not rely on a specific generalizability design or a budget expression, and, therefore, the applicability of COEAs is stronger. (5) In terms of the algorithm generalizability, COEAs is obviously better than the traditional methods. The limited mathematical principles make it difficult to extend the three traditional methods to more complex generalizability designs, and thus, the feasibility of calculation is poor. Howerve, COEAs has revealed stronger generalizability. (6) In terms of the possibility of getting the best solution, COEAs is also better than the traditional methods. Because evolutionary algorithm is a probabilistic algorithm, multiple tests can be conducted to obtain better results for optimal sample sizes. Under some conditions, COEAs can determine better solutions, which, however, is impossible for three traditional methods. (7) These results suggest that COEAs is superior to three traditional methods in estimating the optimal sample size problem under the budget constraints in GT. It is recommended that researchers use COEAs in future research to determine an optimal sample size in their psychological measurement and evaluation.
Subjects: Psychology >> Psychological Measurement submitted time 2022-05-03
Abstract：
Generalizability Theory (GT) is widely applied in psychological measurement and evaluation. A larger generalizability coefficient often indicates a higher reliability the test may have. Generalizability coefficients can be improved by increasing sample sizes. However, the size of a sample would be subject to budget constraints. Therefore, it is important to examine how to effectively determine the size of a sample considering the budget constraints. The existing literature has been largely limited to traditional methods, such as the differential optimization method (Woodward & Joe, 1973), the Lagrange method (Macrolides & Goldstein, 1990), and the Cauchy Schwartz inequality method (Sanders, 1992).
These traditional methods have limited scope of application and their typical conditions are hard to satisfy. In addition, there is no unified comparison available. Fortunately, with the increased use of high performance computing, the Constrained Optimization Evolutionary Algorithms (COEAs) becomes highly feasible.
This paper expands and compares the four methods—the differential optimization method, Lagrange method, Cauchy Schwartz inequality method, and COEAs—determine the best solution to the optimal sample size problem under the budget constraints in GT. Specifically, this paper compares the applicability of the four methods using three generalizability designs, including p× i × r, (r: p) × i and p × i × r × o designs. The results are presented as follows:
(1) In the optimization performance of two-facet generalizability design of p× i × r and (r: p) × i, the performance of COEAs is slightly better than that of the traditional methods, whereas the performance of three traditional methods is equivalent. Although COEAs and the traditional methods have showed similar accuracy, the former has better compliance concerning budget constraints.
(2) In the optimization performance of three-facet generalizability design of p × i × r × o, the performance of COEAs is obviously better than that of the traditional methods. The least ideal generalizability coefficient is obtained using the differential optimization method, whereas its budget compliance is the best; the generalizability coefficient obtained by Lagrange method is the best, but higher than the budget. The Cauchy inequality method obtains a better generalizability coefficient under special budget constraints. But, the performance of COEAs is slightly better than that of Cauchy Schwartz inequality method, especially closer to the budget constraints.
(3) In terms of the algorithm complexity, COEAs obtains an obviously smaller algorithm complexity than do the traditional methods. The complexity of the three traditional methods is relatively high. However, COEAs does not rely on the derivation of mathematical formulas, and the algorithm is relatively less complex.
(4) In terms of the algorithm applicability, COEAs is significantly better than the traditional methods. The applicability of the three traditional methods is relatively narrow. However, COEAs does not rely on a specific generalizability design or a budget expression, and, therefore, the applicability of COEAs is stronger.
(5) In terms of the algorithm generalizability, COEAs is obviously better than the traditional methods. The limited mathematical principles make it difficult to extend the three traditional methods to more complex generalizability designs, and thus, the feasibility of calculation is poor. Howerve, COEAs has revealed stronger generalizability.
(6) In terms of the possibility of getting the best solution, COEAs is also better than the traditional methods. Because evolutionary algorithm is a probabilistic algorithm, multiple tests can be conducted to obtain better results for optimal sample sizes. Under some conditions, COEAs can determine better solutions, which, however, is impossible for three traditional methods.
(7) These results suggest that COEAs is superior to three traditional methods in estimating the optimal sample size problem under the budget constraints in GT. It is recommended that researchers use COEAs in future research to determine an optimal sample size in their psychological measurement and evaluation.
Peer Review Status:Awaiting Review
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： With the development of data-collection technics and increasing complexity of study designs, nested data widely exists in psychological research. Linear mixed-effects models, unfortunately with an unreasonable hypothesis that the residual variances are homogenous, are generally used in nested data analysis. Meanwhile, Mixed-Effects Location-Scale Models (MELSM) has become more and more popular, because they can handle heterogenous residual variances and are able to add predictors for the two substructures (i.e., mean structure denoted as location model and variance structure denoted as scale model) in different levels. MELSM can avoid estimation bias due to inappropriate assumptions of homogenous variance and explore the relationship among traits and simultaneously investigate the inter- and intra-individual variability, as well as their explanatory variables. This study, aims at developing the methods of model construction and sample size planning for MELSM, using simulated studies and empirical studies. In detail, the main contents of this project are as follows. Study 1 focuses on comparing and selecting candidate models based on Bayesian fit indices to construct MELSM, taking into consideration the estimated method for complicated models. We propose that model selection for location model and scale model can be completed sequentially. Study 2 explores the method of sample size planning for MELSM, according to both power analysis (based on Monte Carlo simulation) and the accuracy in parameter estimation analysis (based on the credible interval of the posterior distribution). Adequate sample size is required for both the power and the accuracy in parameter estimation. Study 3 extends the sample size planning method for MELSM to better frame the considerations of uncertainty. By specifying the prior distribution of effect sizes, repeating sampling and selecting model based on the robust Bayesian fit index suggested by Study 1, three main sources of uncertainty can be well controlled: the uncertainty due to unknown population effect size, sampling variability and model approximation. With the simulated study results, we are able to provide reliable Bayesian fit indices for MELSM construction, and summary the process of sample size planning for MELSM in both determinate and uncertain situations. Moreover, Study 4 illustrates the application of MELSM in two empirical psychological studies and verifies the operability of the conclusions of the simulated studies in practice. The unique contribution of this paper is to further promote the methods of model construction and sample size planning for MELSM, as well as provide methodological foundation for researchers. In addition, we plan to integrate the functions above to develop a user-friendly R package for MELSM and provide a basis for promotion and application of MELSM, which help researchers make sample size planning, model construction and parameter estimation for MELSM easily, according to their specification. If these statistical models are widely implemented, the reproducibility and replicability of psychological studies will be enhanced finally.
submitted time 2023-03-25 Cooperative journals: 《心理科学进展》
Abstract： With the development of data-collection technics and increasing complexity of study designs, nested data widely exists in psychological research. Linear mixed-effects models, unfortunately with an unreasonable hypothesis that the residual variances are homogenous, are generally used in nested data analysis. Meanwhile, Mixed-Effects Location-Scale Models (MELSM) has become more and more popular, because they can handle heterogenous residual variances and are able to add predictors for the two substructures (i.e., mean structure denoted as location model and variance structure denoted as scale model) in different levels. MELSM can avoid estimation bias due to inappropriate assumptions of homogenous variance and explore the relationship among traits and simultaneously investigate the inter- and intra-individual variability, as well as their explanatory variables. This study, aims at developing the methods of model construction and sample size planning for MELSM, using simulated studies and empirical studies. In detail, the main contents of this project are as follows. Study 1 focuses on comparing and selecting candidate models based on Bayesian fit indices to construct MELSM, taking into consideration the estimated method for complicated models. We propose that model selection for location model and scale model can be completed sequentially. Study 2 explores the method of sample size planning for MELSM, according to both power analysis (based on Monte Carlo simulation) and the accuracy in parameter estimation analysis (based on the credible interval of the posterior distribution). Adequate sample size is required for both the power and the accuracy in parameter estimation. Study 3 extends the sample size planning method for MELSM to better frame the considerations of uncertainty. By specifying the prior distribution of effect sizes, repeating sampling and selecting model based on the robust Bayesian fit index suggested by Study 1, three main sources of uncertainty can be well controlled: the uncertainty due to unknown population effect size, sampling variability and model approximation. With the simulated study results, we are able to provide reliable Bayesian fit indices for MELSM construction, and summary the process of sample size planning for MELSM in both determinate and uncertain situations. Moreover, Study 4 illustrates the application of MELSM in two empirical psychological studies and verifies the operability of the conclusions of the simulated studies in practice. The unique contribution of this paper is to further promote the methods of model construction and sample size planning for MELSM, as well as provide methodological foundation for researchers. In addition, we plan to integrate the functions above to develop a user-friendly R package for MELSM and provide a basis for promotion and application of MELSM, which help researchers make sample size planning, model construction and parameter estimation for MELSM easily, according to their specification. If these statistical models are widely implemented, the reproducibility and replicability of psychological studies will be enhanced finally.
Subjects: Psychology >> Statistics in Psychology submitted time 2023-01-31
Abstract： With the advancement of research depth in psychology and the development of data collection technics, interest in Mixed-Effects Location-Scale Models (MELSM) has increased drastically. When residual variances are heterogeneous, these models are able to add predictors in different levels, then help explore the relationship among traits and simultaneously investigate the inter- and intra-individual variability, as well as their explanatory variables. This study includes both simulated studies and empirical studies. In detail, the main contents of this project are: 1) Comparing and selecting candidate models based on Bayesian fit indices to construct MELSM; 2) Planning sample size according to both power analysis and accuracy in parameter estimation analysis for MELSM; 3) Extending the sample size planning method for MELSM to better frame the considerations of uncertainty; 4) Developing an R package for MELSM and illustrating the application of MELSM in empirical psychological studies. Based on the study, we hope these statistical models can be widely implemented. Moreover, the reproducibility and replicability of psychological studies will be enhanced finally.
Peer Review Status:Awaiting Review
Subjects: Medicine, Pharmacy >> Preclinical Medicine Subjects: Mathematics >> Statistics and Probability submitted time 2023-04-19
Abstract： The common method of sample size calculation for single proportion comparison (performance goal) is normal asymptotic approach, sometimes with corresponding data transformation such as squared arcsine, while exact probability usually needs commercial statistics software or programming. We use the free software R to calculate the sample size for single proportion via exact probability, and considering of the non-monotone increasing relationship between power and sample size with exact probability, we provide intuitive figure demonstration besides giving direct calculation results. We hope this will facilitate study design with performance goal.
Peer Review Status:Awaiting Review
Hits
Hits
Downloads
Comment