Reproducibility in Benchmarking Parallel Fast Fourier Transform based Applications

D. Alvarez, A. O'Cais, M. Geimer, and K. Hoste. 2016. Scientific Software Man-agement in Real Life: Deployment of Easy build on a Large Scale System. In Proceedings of the Third International Workshop on HPC User Support Tools (HUST'16). IEEE Press, Piscataway, NJ, USA, 31--40.

S. Aseeri, O. Batrasev, M. Icardi, B. Leu, A. Liu, N. Li, B.K. Muite, E. Müller, B. Palen,M. Quell, H. Servat, P. Sheth, R. Speck, M. Van Moer, and J. Vienne. 2015. Solving the Klein-Gordon Equation Using Fourier Spectral Methods: A Benchmark Testfor Computer Performance. In Proceedings of the Symposium on High Performance Computing (HPC '15). Society for Computer Simulation International, San Diego,CA, USA, 182--191. http://dl.acm.org/citation.cfm?id=2872599.2872622;https://arxiv.org/abs/1501.04552

S. Aseeri, B.K. Muite, and D. Takahashi. 2019. Data for figures in "Reproducibility in Benchmarking Parallel Fast Fourier Transform based Applications".

G. Becker, P. Scheibel, M. LeGendre, and T. Gamblin. 2016. Managing Combinatorial Software Installations with Spack. In Proceedings of the Third Internation Workshop on HPC User Support Tools (HUST '16). IEEE Press, Piscataway, NJ, USA,14--23.

A. Bhatele, K. Mohror, S.H. Langer, and K.E. Isaacs. 2013. There Goes the Neighborhood: Performance Degradation Due to Nearby Jobs. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC '13). ACM, New York, NY, USA, Article 41, 12 pages.

I. Chinavinijkul, J. Newcomb, L. Xi, and D.P. Bunde. 2018. Brief Announcement: Coloring-based Task Mapping for Dragonfly Systems. In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures (SPAA '18). ACM, New York, NY, USA, 91--93.

D. Chisnall. 2018. C Is Not a Low-level Language. Queue 16, 2, Article 10 (April2018), 13 pages.

B. Cloutier. 2014. MPI Fortran Carpenter-Kennedy Incompressible Navier Stokes Solver. (2014). https://github.com/bcloutier/PSNM/blob/master/NavierStokes/Programs/NavierStokes3dFortranMPI/navierstokes.f90.

B. Cloutier. 2014."MPI Fortran Implicit Midpoint Rule Incompressible Navier Stokes Solver. (2014). https://github.com/bcloutier/PSNM/blob/master/NavierStokes/Programs/NavierStokes3dFortranMPI/navierstokes_IMR.f90.

M. Eleftheriou, B.G. Fitch, A. Rayshubskiy, T.J.C. Ward, P. Heidelberger, and R.S. Germain. 2008. A Study of the Effects of Machine Geometry and Mappingon Distributed Transpose Performance. In Proceedings of the 5th Conference on Computing Frontiers (CF '08). ACM, New York, NY, USA, 79--86.

M. Frigo and S. G. Johnson. 2005. The Design and Implementation of FFTW3. Proc. IEEE93, 2 (Feb 2005), 216--231.

T. Gamblin, M. LeGendre, M.R. Collette, G.L. Lee, A. Moody, B.R. de Supinski, and S. Futral. 2015. The Spack Package Manager: Bringing Order to HPC Software Chaos. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '15). ACM, New York, NY, USA, Article 40, 12 pages.

M. Geimer, K. Hoste, and R. McLay. 2014. Modern Scientific Software Management Using Eas yBuild and Lmod. In Proceedings of the First International Workshopon HPC User Support Tools (HUST '14). IEEE Press, Piscataway, NJ, USA, 41--51.

Standard Performance Evaluation Corporation High Performance Group. 2011.SPEC MPI2007 Run and Reporting Rules. (August 2011).

T. Hoefler and R. Belli. 2015. Scientific Benchmarking of Parallel ComputingSystems: Twelve Ways to Tell the Masses when Reporting Performance Results. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '15). ACM, New York, NY, USA, Article 73,12 pages.

B. Hoffman and K. Martin. 2003. The CMake Build Manager. Dr. Dobbs Journal(Jan. 2003). http://www.drdobbs.com/cpp/the-cmake-build-manager/184405251#

N. Jain, A. Bhatele, X. Ni, N.J. Wright, and L.V. Kale. 2014. Maximizing Throughputon a Dragonfly Network. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '14). IEEE Press, Piscataway, NJ, USA, 336--347.

I. Jimenez, M. Sevilla, N. Watkins, C. Maltzahn, J. Lofstead, K. Mohror, A. Arpaci-Dusseau, and R. Arpaci-Dusseau. 2017. The popper convention: Making re-producible systems evaluation practical. In Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2017 IEEE International. IEEE, 1561--1570.

A.M. Jokisaari, P.W. Voorhees, J.E. Guyer, J. Warren, and O.G. Heinonen. 2017. Benchmark problems for numerical implementations of phase field models.Computational Materials Science126 (2017), 139--151.

N. Li. 2019. 2 DECOMP & FFT. http://www.2decomp.org/

N. Li and S. Laizet. 2010. 2 DECOMP & FFT -- A highly scalable 2D decomposition library and FFT interface. In Proc. Cray User Group 2010 conference.

K. Martin and B. Hoffman. 2007. An Open Source Approach to Developing Software in a Small Organization. IEEE Software 24, 1 (Jan 2007), 46--53.

R. Moore, R. Kearfott, and M. Cloud. 2009. Introduction to Interval Analysis. Societyfor Industrial and Applied Mathematics. arXiv:https://epubs.siam.org/doi/pdf/10.1137/1.9780898717716

T. Narumi, R. Susukita, T. Koishi, K. Yasuoka, H. Furusawa, A. Kawai, and T.Ebisuzaki. 2000. 1.34 Tflops Molecular Dynamics Simulation for NaCl with a Special-purpose Computer: MDM. In Proceedings of the 2000 ACM/IEEE Conference on Supercomputing (SC '00). IEEE Computer Society, Washington, DC, USA,Article 54. http://dl.acm.org/citation.cfm?id=370049.370465

D. San Roman Alerigi, S. Balakrishanan, A.H. Bargash, G. Chen, B. Cloutier, N. Li,D. Malicke, B.K. Muite, M. Quell, P. Rigge, M. Solimani, A. Souza, A.S. Thiban, J.West, and M. Van Moer. 2018. Parallel Spectral Numerical Methods. (November 2018). http://en.wikibooks.org/w/index.php?title=Parallel_Spectral_Numerical_Methods.

D. San Roman Alerigi, S. Balakrishanan, A.H. Bargash, G. Chen, B. Cloutier, N.Li, D. Malicke, B.K. Muite, M. Quell, P. Rigge, M. Solimani, A. Souza, A.S. Thiban, J. West, and M. Van Moer. 2019. Parallel Spectral Numerical Methods. (January 2019). https://github.com/openmichigan/PSNM

D.E. Shaw, J.P. Grossman, J.A. Bank, B. Batson, J.A. Butts, J.C. Chao, M.M. Deneroff, R.O. Dror, A. Even, C.H. Fenton, A. Forte, J. Gagliardo, G. Gill, B. Greskamp, C.R. Ho, D.J. Ierardi, L. Iserovich, J.S. Kuskin, R.H. Larson, T. Layman, L.-S. Lee, A.K. Lerer, C. Li, D. Killebrew, K.M. Mackenzie, S.Y.-H. Mok, M.A. Moraes, R.Mueller, L.J. Nociolo, J.L. Peticolas, T. Quan, D. Ramot, J.K. Salmon, D.P. Scarpazza, U. Ben Schafer, N. Siddique, C.W. Snyder, J. Spengler, P.T.P. Tang, M. Theobald, H. Toma, B. Towles, B. Vitale, S.C. Wang, and C. Young. 2014. Anton 2: Raisingthe Bar for Performance and Programmability in a Special-purpose Molecular Dynamics Supercomputer. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '14). IEEE Press, Piscataway, NJ, USA, 41--53.

M.R. Shirts, C. Klein, J.M. Swails, J. Yin, M.K. Gilson, D.L. Mobley, D.A. Case, andE.D. Zhong. 2017. Lessons learned from comparing molecular dynamics engineson the SAMPL 5 dataset. Journal of Computer-Aided Molecular Design 31, 1 (01 Jan 2017), 147--161.

S. Sreepathi, E. D'Azevedo, B. Philip, and P. Worley. 2016. Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers. In Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering (ICPE '16). ACM, New York,NY, USA, 225--236.

P. Steinbach and M. Werner. 2017. gearshifft -- The FFT Benchmark Suite for Heterogeneous Platforms. In High Performance Computing, J.M. Kunkel, R. Yokota, P. Balaji, and D. Keyes (Eds.). Springer International Publishing, Cham, 199--216.

V. Stodden, C. Hurlin, and C. PÃrignon. 2012. Run My Code.org: A novel dis-semination and collaboration platform for executing published computational results. In 2012 IEEE 8th International Conference on E-Science. 1--8.

D. Takahashi. 2001. A Blocking Algorithm for FFT on Cache-Based Processors. In High-Performance Computing and Networking, B. Hertzberger, A. Hoekstra, and R. Williams (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 551--554.

D. Takahashi. 2003. A parallel 1-D FFT algorithm for the Hitachi SR8000.ParallelComput.29, 6 (2003), 679 -- 690.

D. Takahashi. 2014. FFTE: A Fast Fourier Transform Package. http://ffte.jp/

M. Werner and P. Steinbach. 2019. mpicbg-scicomp/gearshifft: v0.4.0.

L. Wu, X. Xu, Y. Wei, and X. Liu. 2017. A Survey About Quantitative Measurementof Performance Variability in High Performance Computers. In Advanced Parallel Processing Technologies, Y. Dou, H. Lin, G. Sun, J. Wu, D. Heras, and L. Bougé(Eds.). Springer International Publishing, Cham, 76--86.

P. Yébenes, J. Escudero-Sahuquillo, P.J. García, and F.J. Quiles. 2016. Straightfor-ward solutions to reduce HoL blocking in different Dragonfly fully-connected interconnection patterns. The Journal of Supercomputing 72, 12 (01 Dec 2016), 4497--4519.

Y. Zhang, O. Tuncer, F. Kaplan, K. Olcoz, V. J. Leung, and A. K. Coskun. 2018.Level-Spread: A New Job Allocation Policy for Dragonfly Networks. In 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS). 1123--1132.

X. Wang, M. Mubarak, X. Yang, R. B. Ross, and Z. Lan. 2018. Trade-Off Study of Localizing Communication and Balancing Network Traffic on a Dragonfly System. In 2018 IEEE International Parallel and Distributed Processing Symposium(IPDPS). 1113--1122.