Toward Cost-Effective Adaptive Random Testing: An Approximate Nearest Neighbor Approach
Authors: 
Rubing Huang, 
Chenhui Cui, Junlong Lian, 
Dave Towey, Weifeng Sun, Haibo ChenAuthors Info & Claims
Pages 1182 - 1214
Abstract
<italic>Adaptive Random Testing</italic> (ART) enhances the testing effectiveness (including fault-detection capability) of <italic>Random Testing</italic> (RT) by increasing the diversity of the random test cases throughout the input domain. Many ART algorithms have been investigated such as <italic>Fixed-Size-Candidate-Set ART</italic> (FSCS) and <italic>Restricted Random Testing</italic> (RRT), and have been widely used in many practical applications. Despite its popularity, ART suffers from the problem of high computational costs during test-case generation, especially as the number of test cases increases. Although several strategies have been proposed to enhance the ART testing efficiency, such as the <italic>forgetting strategy</italic> and the <inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="huang-ieq1-3379592.gif"/></alternatives></inline-formula> <italic>-dimensional tree strategy</italic>, these algorithms still face some challenges, including: (1) Although these algorithms can reduce the computation time, their execution costs are still very high, especially when the number of test cases is large; and (2) To achieve low computational costs, they may sacrifice some fault-detection capability. In this paper, we propose an approach based on <italic>Approximate Nearest Neighbors</italic> (ANNs), called <italic>Locality-Sensitive Hashing ART</italic> (LSH-ART). When calculating distances among different test inputs, LSH-ART identifies the approximate (not necessarily exact) nearest neighbors for candidates in an efficient way. LSH-ART attempts to balance ART testing effectiveness and efficiency.
References
[1]
A. Orso and G. Rothermel, “Software testing: A research travelogue (2000–2014),” in Proc. Future Softw. Eng. (FOSE’14), 2014, pp. 117–132.
[2]
S. Anand et al., “An orchestrated survey of methodologies for automated software test case generation,” J. Syst. Softw., vol. 86, no. 8, pp. 1978–2001, 2013.
[3]
R. Hamlet, Random Testing. Hoboken, NJ, USA: Wiley, 2002.
[4]
P. G. Frankl, R. G. Hamlet, B. Littlewood, and L. Strigini, “Evaluating testing methods by delivered reliability,” IEEE Trans. Softw. Eng., vol. 24, no. 8, pp. 586–601, Aug. 1998.
[5]
D. R. Slutz, “Massive stochastic testing of SQL,” in Proc. 24th Int. Conf. Very Large Data Bases (VLDB’98), 1998, pp. 618–622.
[6]
H. Bati, L. Giakoumakis, S. Herbert, and A. Surna, “A genetic approach for random testing of database systems,” in Proc. 33rd Int. Conf. Very Large Data Bases (VLDB’07), 2007, pp. 1243–1251.
[7]
J. Regehr, “Random testing of interrupt-driven software,” in Proc. 5th ACM Int. Conf. Embedded Softw. (EMSOFT’05), 2005, pp. 290–298.
[8]
T. Yoshikawa, K. Shimura, and T. Ozawa, “Random program generator for Java JIT compiler test system,” in Proc. 3rd Int. Conf. Qual. Softw. (QSIC’03), 2003, pp. 20–23.
[9]
P. Godefroid, A. Kiezun, and M. Y. Levin, “Grammar-based whitebox fuzzing,” in Proc. 29th ACM SIGPLAN Conf. Program. Lang. Des. Implementation (PLDI’08), 2008, pp. 206–215.
[10]
C. Pacheco, S. K. Lahiri, and T. Ball, “Finding errors in.NET with feedback-directed random testing,” in Proc. 17th Int. Symp. Softw. Testing Anal. (ISSTA’08), 2008, pp. 87–96.
[11]
A. Arcuri, M. Z. Iqbal, and L. Briand, “Random testing: Theoretical results and practical implications,” IEEE Trans. Softw. Eng., vol. 38, no. 2, pp. 258–277, Mar./Apr. 2012.
[12]
G. J. Myers, C. Sandler, and T. Badgett, The Art of Software Testing. New York, NY, USA: Wiley, 2011.
[13]
T. Y. Chen, F.-C. Kuo, R. G. Merkel, and T. H. Tse, “Adaptive random testing: The art of test case diversity,” J. Syst. Softw., vol. 83, no. 1, pp. 60–66, 2010.
[14]
R. Huang, W. Sun, Y. Xu, H. Chen, D. Towey, and X. Xia, “A survey on adaptive random testing,” IEEE Trans. Softw. Eng., vol. 47, no. 10, pp. 2052–2083, Oct. 2021.
[15]
L. White and E. Cohen, “A domain strategy for computer program testing,” IEEE Trans. Softw. Eng., vol. SE-6, no. 3, pp. 247–257, May 1980.
[16]
P. E. Ammann and J. C. Knight, “Data diversity: An approach to software fault tolerance,” IEEE Trans. Comput., vol. 37, no. 4, pp. 418–425, Apr. 1988.
[17]
G. B. Finelli, “NASA software failure characterization experiments,” Rel. Eng. Syst. Saf., vol. 32, nos. 1–2, pp. 155–169, 1991.
[18]
P. G. Bishop, “The variation of software survival times for different operational input profiles,” in Proc. 23rd Int. Symp. Fault-Tolerant Comput. (FTCS’93), 1993, pp. 98–107.
[19]
C. Schneckenburger and J. Mayer, “Towards the determination of typical failure patterns,” in Proc. 4th Int. Workshop Softw. Qual. Assurance (SOQUA’07)/ 6th Joint Meeting Eur. Softw. Eng. Conf./ACM SIGSOFT Int. Symp. Found. Softw. Eng. (ESEC/FSE’07), 2007, pp. 90–93.
[20]
T. Y. Chen, F.-C. Kuo, D. Towey, and Z. Q. Zhou, “A revisit of three studies related to random testing,” Sci. China Inf. Sci., vol. 58, no. 5, pp. 1–9, 2015.
[21]
T. Y. Chen, F.-C. Kuo, and Z. Q. Zhou, “On favourable conditions for adaptive random testing,” Int. J. Softw. Eng. Knowl. Eng., vol. 17, no. 6, pp. 805–825, 2007.
[22]
T. Y. Chen, F.-C. Kuo, H. Liu, and W. E. Wong, “Code coverage of adaptive random testing,” IEEE Trans. Rel., vol. 62, no. 1, pp. 226–237, Mar. 2013.
[23]
T. Y. Chen, F.-C. Kuo, and H. Liu, “On test case distributions of adaptive random testing,” in Proc. 9th Int. Conf. Softw. Eng. Knowl. Eng. (SEKE’07), 2007, pp. 141–144.
[24]
K. P. Chan, T. Y. Chen, and D. Towey, “Restricted random testing: Adaptive random testing by exclusion,” Int. J. Softw. Eng. Knowl. Eng., vol. 16, no. 4, pp. 553–584, 2006.
[25]
G. T. Wilfong, “Nearest neighbor problems,” in Proc. 7th Annu. Symp. Comput. Geometry (SCG’91), 1991, pp. 224–233.
[26]
K. P. Chan, T. Y. Chen, and D. Towey, “Forgetting test cases,” in Proc. IEEE 30th Annu. Int. Comput. Softw. Appl. Conf. (COMPSAC’06), 2006, pp. 485–494.
[27]
C. Mao, X. Zhan, T. H. Tse, and T. Y. Chen, “KDFC-ART: A KD-tree approach to enhancing fixed-size-candidate-set adaptive Random Testing,” IEEE Trans. Rel., vol. 68, no. 4, pp. 1444–1469, Dec. 2019.
[28]
J. L. Bentley, “Multidimensional binary search trees used for associative searching,” Commun. ACM, vol. 18, no. 9, pp. 509–517, 1975.
[29]
H. Jagadish, B. C. Ooi, and Q. H. Vu, “BATON: A balanced tree structure for peer-to-peer networks,” in Proc. 31st Int. Conf. Very Large Data Bases (VLDB’05), 2005, pp. 661–672.
[30]
M. Bawa, T. Condie, and P. Ganesan, “LSH forest: Self-tuning indexes for similarity search,” in Proc. 14th Int. Conf. World Wide Web (WWW’05), 2005, pp. 651–660.
[31]
H.-M. Hu and F. Jiang, “Scalable locality sensitive hashing scheme for dynamic high-dimensional data indexing,” Ruan Jian Xue Bao/J. Softw., vol. 26, no. S2, pp. 228–238, 2015.
[32]
J. Li et al., “LoSHa: A general framework for scalable locality sensitive hashing,” in Proc. 40th Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval (SIGIR’17), 2017, pp. 635–644.
[33]
P. Indyk and R. Motwani, “Approximate nearest neighbors: Towards removing the curse of dimensionality,” in Proc. 30th Annu. ACM Symp. Theory Comput. (STOC’98), 1998, pp. 604–613.
[34]
E. J. Weyuker, “On testing non-testable programs,” Comput. J., vol. 4, no. 25, pp. 465–470, 1982.
[35]
E. T. Barr, M. Harman, P. McMinn, M. Shahbaz, and S. Yoo, “The oracle problem in software testing: A survey,” IEEE Trans. Softw. Eng., vol. 41, no. 5, pp. 507–525, May 2014.
[36]
Z. Q. Zhou, D. Towey, P. Poon, and T. H. Tse, “Introduction to the special issue on test oracles,” J. Syst. Softw., vol. 136, 2018, Art. no.
[37]
F. T. Chan, T. Y. Chen, I. K. Mak, and Y. T. Yu, “Proportional sampling strategy: Guidelines for software testing practitioners,” Inf. Softw. Technol., vol. 38, no. 12, pp. 775–782, 1996.
[38]
T. Y. Chen, H. Leung, and I. K. Mak, “Adaptive random testing,” in Proc. 9th Asian Comput. Sci. Conf. (ASIAN’04), 2004, pp. 320–329.
[39]
K. P. Chan, T. Y. Chen, and D. Towey, “Controlling restricted random testing: An examination of the exclusion ratio parameter,” in Proc. 19th Int. Conf. Softw. Eng. Knowl. Eng. (SEKE’07), 2007, pp. 163–166.
[40]
S. Arya and D. Mount, “Approximate nearest neighbor queries in fixed dimensions,” in Proc. 4th ACM-SIAM Symp. Discrete Algorithms (SODA’93), 1993, pp. 271–280.
[41]
S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu, “An optimal algorithm for approximate nearest neighbor searching fixed dimensions,” J. ACM, vol. 45, no. 6, pp. 891–923, 1998.
[42]
A. Andoni and P. Indyk, “Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions,” in Proc. 47th Annu. IEEE Symp. Found. Comput. Sci. (FOCS’06), 2006, pp. 459–468.
[43]
M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, “Locality-sensitive hashing scheme based on p-stable distributions,” in Proc. 20th Annu. Symp. Comput. Geometry (SCG’04), 2004, pp. 253–262.
[44]
J. Mayer and C. Schneckenburger, “An empirical analysis and comparison of random testing techniques,” in Proc. 5th ACM/IEEE Int. Symp. Empirical Softw. Eng. (ISESE’06), 2006, pp. 105–114.
[45]
T. Y. Chen, F.-C. Kuo, R. G. Merkel, and S. P. Ng, “Mirror adaptive random testing,” Inf. Softw. Technol., vol. 46, no. 15, pp. 1001–1010, 2004.
[46]
C. Mao, T. Y. Chen, and F.-C. Kuo, “Out of sight, out of mind: A distance-aware forgetting strategy for adaptive random testing,” Sci. China Inf. Sci., vol. 60, no. 9, 2017.
[47]
H. Liu, X. Xie, J. Yang, Y. Lu, and T. Y. Chen, “Adaptive random testing through test profiles,” Softw.: Pract. Exp., vol. 41, no. 10, pp. 1131–1154, 2011.
[48]
H. Ackah-Arthur, J. Chen, D. Towey, M. Omari, J. Xi, and R. Huang, “One-domain-one-input: Adaptive random testing by orthogonal recursive bisection with restriction,” IEEE Trans. Rel., vol. 68, no. 4, pp. 1404–1428, Dec. 2019.
[49]
J. W. Duran and S. C. Ntafos, “An evaluation of random testing,” IEEE Trans. Softw. Eng., vol. SE-10, no. 4, pp. 438–444, Jul. 1984.
[50]
A. Shahbazi, A. F. Tappenden, and J. Miller, “Centroidal Voronoi Tessellations—A new approach to random testing,” IEEE Trans. Softw. Eng., vol. 39, no. 2, pp. 163–183, Feb. 2013.
[51]
J. Mayer, “Adaptive random testing by bisection and localization,” in Proc. 5th Int. Workshop Formal Approaches Softw. Testing (FATES’05), 2005, pp. 72–86.
[52]
J. Mayer, “Adaptive random testing by bisection with restriction,” in Proc. 7th Int. Conf. Formal Eng. Methods (ICFEM’05), 2005, pp. 251–263.
[53]
A. F. Tappenden and J. Miller, “A novel evolutionary approach for adaptive random testing,” IEEE Trans. Rel., vol. 58, no. 4, pp. 619–633, Dec. 2009.
[54]
M. Ashfaq, R. Huang, and M. Omari, “FSCS-SIMD: An efficient implementation of fixed-size-candidate-set adaptive random testing using SIMD instructions,” in Proc. IEEE 31st Int. Symp. Softw. Rel. Eng. (ISSRE’20), 2020, pp. 277–288.
[55]
T. Y. Chen, F.-C. Kuo, and R. Merkel, “On the statistical properties of testing effectiveness measures,” J. Syst. Softw., vol. 79, no. 5, pp. 591–601, 2006.
[56]
Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Trans. Softw. Eng., vol. 37, no. 5, pp. 649–678, Sep./Oct. 2011.
[57]
A. Arcuri and L. Briand, “Adaptive random testing: An illusion of effectiveness?” in Proc. 20th Int. Symp. Softw. Testing Anal. (ISSTA’11), 2011, pp. 265–275.
[58]
Y.-S. Ma, J. Offutt, and Y. R. Kwon, “MuJava: An automated class mutation system,” Softw. Testing, Verification Rel., vol. 15, no. 2, pp. 97–133, 2005.
[59]
A. Arcuri and L. Briand, “A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering,” Softw. Testing, Verification Rel., vol. 24, no. 3, pp. 219–250, 2014.
[60]
F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics, vol. 1, no. 6, pp. 80–83, 1945.
[61]
A. Vargha and H. D. Delaney, “A critique and improvement of the CL common language effect size statistics of McGraw and Wong,” J. Educ. Behav. Statist., vol. 25, no. 2, pp. 101–132, 2000.
[62]
J. Klein and M. Moeschberger, Survival Analysis: Techniques for Censored and Truncated Data. Berlin, Germany: Springer-Verlag, 2003.
[63]
R. Grissom and J. Kim, Effect Sizes for Research: A Broad Practical Approach. London, U.K.: Lawrence Erlbaum, 2005.
[64]
R. Huang, C. Cui, J. Lian, D. Towey, W. Sun, and H. Chen, “Appendix: Detailed results for LSH-ART,” GitHub. Accessed: Mar. 18, 2024. [Online]. Available: https://github.com/huangrubing/detailed-results-lsh-art
[65]
R. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton Univ. Press, 1957.
[66]
J. Mayer, “Efficient and effective random testing based on partitioning and neighborhood,” in Proc. 18th Int. Conf. Softw. Eng. Knowl. Eng. (SEKE’06), 2006, pp. 499–504.
[67]
A. J. Offutt, “Investigations of the software testing coupling effect,” ACM Trans. Softw. Eng. Methodol., vol. 1, no. 1, pp. 5–20, 1992.
[68]
L. Morell, “A theory of fault-based testing,” IEEE Trans. Softw. Eng., vol. 16, no. 8, pp. 844–857, Aug. 1990.
[69]
R. DeMillo, R. Lipton, and F. Sayward, “Hints on test data selection: Help for the practicing programmer,” Computer, vol. 11, no. 4, pp. 34–41, 1978.
[70]
M. Papadakis, M. Kintis, J. Zhang, Y. Jia, Y. L. Traon, and M. Harman, “Chapter six-Mutation testing advances: An analysis and survey,” in Proc. Adv. Comput., vol. 112, 2019, pp. 275–378.
[71]
R. Just, D. Jalali, L. Inozemtseva, M. D. Ernst, R. Holmes, and G. Fraser, “Are mutants a valid substitute for real faults in software testing?” in Proc. 22nd ACM SIGSOFT Int. Symp. Found. Softw. Eng. (FSE’14), 2014, pp. 654–665.
[72]
T. T. Chekam, M. Papadakis, Y. Le Traon, and M. Harman, “An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption,” in Proc. IEEE/ACM 39th Int. Conf. Softw. Eng. (ICSE’17), 2017, pp. 597–608.
[73]
M. Papadakis, D. Shin, S. Yoo, and D.-H. Bae, “Are mutation scores correlated with real fault detection? A large scale empirical study on the relationship between mutants and real faults,” in Proc. IEEE/ACM 40th Int. Conf. Softw. Eng. (ICSE’18), 2018, pp. 537–548.
[74]
F. Medeiros, C. Kästner, M. Ribeiro, R. Gheyi, and S. Apel, “A comparison of 10 sampling algorithms for configurable systems,” in Proc. 38th Int. Conf. Softw. Eng. (ICSE’16), 2016, pp. 643–654.
[75]
A. B. Sánchez, S. Segura, J. A. Parejo, and A. R. Cortés, “Variability testing in the wild: The Drupal case study,” Softw. Syst. Model., vol. 16, no. 1, pp. 173–194, 2017.
[76]
H. Wu, C. Nie, J. Petke, Y. Jia, and M. Harman, “An empirical comparison of combinatorial testing, random testing and adaptive random testing,” IEEE Trans. Softw. Eng., vol. 46, no. 3, pp. 302–320, Mar. 2020.
[77]
R. Tartler, D. Lohmann, J. Sincero, and W. Schröder-Preikschat, “Feature consistency in compile-time-configurable system software: Facing the linux 10,000 feature problem,” in Proc. 6th Eur. Conf. Comput. Syst. (EuroSys’11), 2011, pp. 47–60.
[78]
R. Huang, C. Cui, W. Sun, and D. Towey, “Poster: Is euclidean distance the best distance measurement for adaptive random testing?” in Proc. 13th IEEE Int. Conf. Softw. Testing, Validation Verification (ICST’20), 2020, pp. 406–409.
[79]
IEEE, “ISO/IEC/IEEE International Standard—Systems and Software Engineering—Vocabulary,” ISO/IEC/IEEE 24765:2010(E), 2010, pp. 1–418.
[80]
W. E. Wong, R. Gao, Y. Li, R. Abreu, and F. Wotawa, “A survey on software fault localization,” IEEE Trans. Softw. Eng., vol. 42, no. 8, pp. 707–740, Aug. 2016.
[81]
N. DiGiuseppe and J. A. Jones, “Fault density, fault types, and spectra-based fault localization,” Empirical Softw. Eng., vol. 20, no. 1, pp. 928–967, 2015.
[82]
M. D. Smith and D. J. Robson, “A framework for testing object-oriented programs,” J. Object-Oriented Program., vol. 5, no. 3, pp. 45–53, 1992.
[83]
J. H. Hayes, “Testing of object-oriented programming systems (OOPS): A fault-based approach,” in Proc. Int. Symp. Object-Oriented Methodol. Syst. (ISOOMS’94), 1994, pp. 205–220.
[84]
D. C. Kung, J. Gao, and C.-H. Kung, Testing Object-Oriented Software. Los Alamitos, CA, USA: IEEE Comput. Soc. Press, 1998.
[85]
J. H. Hayes, I. R. Chemannoor, and E. A. Holbrook, “Improved code defect detection with fault links,” Softw. Testing, Verification Rel., vol. 21, no. 4, pp. 299–325, 2011.
[86]
K. K. Sabor and S. Thiel, “Adaptive random testing by static partitioning,” in Proc. 10th Int. Workshop Automat. Softw. Test (AST’15), 2015, pp. 28–32.
[87]
T. Y. Chen, R. G. Merkel, P. K. Wong, and G. Eddy, “Adaptive random testing through dynamic partitioning,” in Proc. 4th Int. Conf. Qual. Softw. (QSIC’04), 2004, pp. 79–86.
[88]
T. Y. Chen, D. H. Huang, and Z. Q. Zhou, “Adaptive random testing through iterative partitioning,” in Proc. 11th Ada-Europe Int. Conf. Reliable Softw. Technol. (Ada-Europe’06), 2006, pp. 155–166.
[89]
C. Chow, T. Y. Chen, and T. H. Tse, “The ART of divide and conquer: An innovative approach to improving the efficiency of adaptive random testing,” in Proc. 13th Int. Conf. Qual. Softw. (QSIC’13), 2013, pp. 268–275.
[90]
T. Y. Chen and R. G. Merkel, “Quasi-random testing,” IEEE Trans. Rel., vol. 56, no. 3, pp. 562–568, 2007.
[91]
J. H. Halton, “Algorithm 247: Radical-inverse quasi-random point sequence,” Commun. ACM, vol. 7, no. 12, pp. 701–702, 1964.
[92]
I. M. Sobol, “Uniformly distributed sequences with an additional uniform property,” USSR Comput. Math. Math. Phys., vol. 16, no. 5, pp. 236–242, 1976.
[93]
H. Niederreiter, “Low-discrepancy and low-dispersion sequences,” J. Number Theory, vol. 30, no. 1, pp. 51–70, 1988.
[94]
R. Cranley and T. N. L. Patterson, “Randomization of number theoretic methods for multiple integration,” SIAM J. Numer. Anal., vol. 13, no. 6, pp. 904–914, 1976.
[95]
T. Kollig and A. Keller, “Efficient multidimensional sampling,” Comput. Graph. Forum, vol. 21, no. 3, pp. 557–563, 2002.
[96]
A. B. Owen, “Randomly permuted (t,m,s)-nets and (t, s)-sequences,” in Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, H. Niederreiter and P. J.-S. Shiue, Eds., New York, NY, USA: Springer-Verlag, 1995, pp. 299–317.
[97]
H. Liu and T. Y. Chen, “Randomized quasi-random testing,” IEEE Trans. Comput., vol. 65, no. 6, pp. 1896–1909, Jun. 2016.
[98]
P. McMinn, “Search-based software test data generation: A survey,” Software Testing, Verification Rel., vol. 14, no. 2, pp. 105–156, 2004.
[99]
M. Harman and P. McMinn, “A theoretical and empirical study of search-based testing: Local, global, and hybrid search,” IEEE Trans. Softw. Eng., vol. 36, no. 2, pp. 226–247, Mar./Apr. 2010.
[100]
C. Schneckenburger and F. Schweiggert, “Investigating the dimensionality problem of adaptive random testing incorporating a local search technique,” in Proc. 1st IEEE Int. Conf. Softw. Testing Verification Validation Workshop (ICSTW’08), 2008, pp. 241–250.
[101]
P. M. S. Bueno, M. Jino, and W. E. Wong, “Diversity oriented test data generation using metaheuristic search techniques,” Inf. Sci., vol. 259, pp. 490–509, Feb. 2014.
[102]
P. M. S. Bueno, W. E. Wong, and M. Jino, “Improving random test sets using the diversity oriented test data generation,” in Proc. 2nd Int. Workshop Random Testing (RT’07)/22nd IEEE/ACM Int. Conf. Automated Softw. Eng. (ASE’07), 2007, pp. 10–17.
[103]
X. Huang, L. Huang, S. Zhang, L. Zhou, M. Wu, and M. Chen, “Improving random test sets using a locally spreading approach,” in Proc. IEEE Int. Conf. Softw. Qual., Rel. Secur. (QRS’17), 2017, pp. 32–41.
[104]
R. Gray and D. Neuhoff, “Quantization,” IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2325–2383, Oct. 1998.
[105]
R. Salakhutdinov and G. Hinton, “Semantic hashing,” Int. J. Approx. Reasoning, vol. 50, no. 7, pp. 969–978, 2009.
[106]
H. Jégou, M. Douze, and C. Schmid, “Product quantization for nearest neighbor search,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 117–128, Jan. 2011.
Recommendations
A nearest-neighbor divide-and-conquer approach for adaptive random testing
Highlights- We propose a new ART approach to enhance testing efficacy and efficiency of FSCS- ART.
AbstractAdaptive Random Testing (ART) aims at enhancing the failure detection capability of Random Testing (RT) by evenly distributing test cases over the input domain. Many ART algorithms have been proposed to achieve an even ...
On test suite composition and cost-effective regression testing
Regression testing is an expensive testing process used to revalidate software as it evolves. Various methodologies for improving regression testing processes have been explored, but the cost-effectiveness of these methodologies has been shown to vary ...
Adaptive Random Testing
QSIC '08: Proceedings of the 2008 The Eighth International Conference on Quality SoftwareRandom testing is a basic testing technique. Motivated by the observation that neighboring inputs normally exhibit similar failure behavior, the approach of Adaptive Random Testing has recently been proposed to enhance the fault detection capability of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
0098-5589 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 21 March 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 20 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in