Introduction

Porous materials consist of a wide variety of chemistries, which include traditional inorganic zeolites, as well as metal-organic frameworks (MOFs) and composite frameworks1,2. Their microporous and mesoporous structures have diverse practical applications3,4, such as gas/vapor storage and separation5,6,7,8,9, purification, chemical sensing10, catalysis11,12, thermoelectric13, nerve degradation14, drug delivery15,16,17, fuel cell18, and energy storage and production19,20. These materials possess large surface areas and customized pore structures, which make them extremely adaptable for solving complex problems21,22. As scientific investigation advances, porous materials offer new opportunities for innovation and technological advances23. On the other hand, it is fundamentally implied that repeated experiments under the same conditions would produce the same result if an experimentally derived observation were accepted as factual24. In practical terms, developing a new material for a practical application cannot be accomplished unless its properties are routinely repeatable. By way of explanation, achieving reproducibility in the field of porous materials research means being able to duplicate the experimental results or computational outcomes consistently when working with these materials. However, there are a number of factors that contribute to the difficulty of this endeavor. In the first instance, reproducibility can be best explored by repeating experiments systematically, which, unfortunately, requires a substantial amount of resources. Second, due to defects, impurities, and experimental and experimenter inconsistencies, the study results may contradict those of an earlier study, posing additional challenges.

Albeit, several approaches may be able to help improve the situation. Among these are the standardization of measurement methods, the definition of measurement or reporting guidelines, and an increase in collaboration between experimentalists and theorists, as well as different laboratories, in order to help corroborate research results, in addition to the use of various tests and cross-checks to ensure that each set of data on adsorption/desorption is valid. Figure 1 is a schematic illustration describing the fundamentals of porous materials research.

Fig. 1
figure 1

Schematic illustration describing the fundamentals in porous materials’ research.

Full size image

In the following sections we provide a brief overview of our work on the rapid scaling up of new framework materials from laboratory to commercial scale production and application. As well, they present our perspective on how the community can benefit from the understanding of zeolites for rapid scale up and commercialization of relatively new framework materials such as MOFs, polymer/porous material composites (PPMCs), and porous liquids (PLs).

Rapid scale-up of new framework materials from laboratory to commercial scale

Zeolites

The 1950s heralded a breakthrough for zeolites with respect to their industrial-scale deployment in a wide range of applications such as catalysis, ion exchange, and adsorption-based separation processes25. When compared to relatively new porous frameworks/materials with a wide range of applications, zeolites are perhaps the most versatile in terms of synthetic and characterization protocols. Despite their potential, zeolite manufacturing processes are complex and limited by the cost associated with technology development, the energy-intensive production process, and a significant carbon footprint26. A success story in the commercial scale production of zeolites is that of the Zeolite Linde Type A (Zeolite LTA or Zeolite A) which is one of the largest zeolites employed by volume and value27. Zeolites such as Faujasite and Mordenite (MOR) have also been demonstrated for large scale applications such as O2 production and CO2 capture28, only possible because of the capacity to manufacture them at kg scale. Commercial-scale autoclaves required for manufacturing zeolites have been reported in the 10–20 m3 scale26,29.

To tailor their structures and properties for specific applications, the zeolite community used their understanding of the hydrothermal process30,31,32, and different crystal mechanisms33,34,35,36 to develop sustainable pathways for large-scale zeolite production by selecting low-cost readily available raw materials37; eliminating the use of organic precursors38,39,40,41; and tuning of operating conditions (mixing rate and intensity)29. These early successes provided the platform to explore a wider range of sustainable synthetic pathways that leverage the combination of predictive modeling42,43, and new chemistries30,44,45,46,47 to develop and produce zeolites at scale for industrial applications. For example, Chen et al.48 discussed the progress in research on the production of zeolites from coal fly ash, one of the most emitted solid wastes globally49, and concluded that only two methods (two-step hydrothermal and alkali melting methods) present the most feasible pathway to commercial production and application. This underscores the need for a holistic approach at the onset of the development phase of new zeolites that not only considers new synthetic pathways, materials, and processes but also incorporates techniques such as life cycle assessment (LCA) to ensure a sustainable and environmentally friendly production process27.

The last half century has seen the development of new characterization techniques (e.g., surface area measurements, X-ray diffraction, and electron microscopy)30,44,50,51, with advances in material chemistry heralding the growth and deployment of zeolites beyond laboratory-based experiments to active materials used for gas separation, catalysis, water processing, agriculture, and biotechnology to mention but a few. Thus, the porous materials community attempts to develop versatile protocols and pathways towards the scale-up and production of new framework materials, adapting the best practices and methods that have been demonstrated to work so well for zeolites. For example, understanding the impact of stirring rates on the crystallinity of ZSM-22 facilitated its industrial-scale production via a sustainable pathway29. Therefore, the community must explore and utilize the knowledge generated in process chemistry to produce zeolites at scale to fast-track translating MOF synthesis from the lab bench to industrial-scale processes for a wide range of MOFs.

Metal-organic frameworks

MOFs are a class of generally porous materials, that have an immense potential for gas storage and separation52,53,54,55, as well as drug delivery56,57,58,59, and catalysis60,61,62. Over the past decade, it has become apparent that there is a broad reproducibility crisis within the literature including both chemistry and chemical engineering63,64. We have found that the inability to reproduce MOFs reported in the literature hampers both further research efforts and potential benefits to the wider community.

Notably, there have been few systematic reproducibility studies on MOFs65,66,67. Boström and co-workers highlighted some of the reproducibility issues in 202365 when they had ten laboratories prepare two closely related MOFs (PCN-222 and PCN-224). Synthetic details were prescribed and included solvent, modulator, temperature, reaction time, and reagent concentration. Despite this, only one of the ten groups produced a phase pure sample of PCN-222 and three groups were able to prepare a phase pure sample of PCN-224. However, it is evident, based on the procedures that individual groups reported, that other factors beyond the ones described above are at play when attempting to reproduce a phase pure sample of PCN-222 or PCN-224. It is not unreasonable to suggest this is likely the case for the synthetic production of many other MOFs at both small and large scales. We take the same stance as Boström and co-workers in that it is important to include as much detail as possible in describing the synthesis of porous materials, including MOFs at all magnitudes of scale-up. An example of detailed procedures is that for articles published in Nature Protocols.

Over the years, MOFs have been synthesized using various methods, including ambient pressure, solvothermal, mechanochemical, microwave-assisted, electrochemical, and flow-based production68. Each method has advantages and challenges in reproducibility (see Table 1), primarily arising from an inability to precisely control reaction parameters like time, temperature, pressure, reactant concentrations, flow rates, and surface area-to-volume ratios. Adequate control of these parameters ensures reproducibility across the various synthetic techniques. Our group has succeeded at scaling from tens of milligrams to hundreds of kilograms, through careful control of synthetic variables. Generally, intermediate steps (e.g., 1 kg → 10 kg → 100 kg) and optimization at each stage are required for scale-up69.

Table 1 Advantages and disadvantages of MOF synthesis methods with respect to reproducibility
Full size table

Tens of thousands of MOFs are known, but very few are industrially produced in part due to complex syntheses. Currently there are no standardized procedures for the scale-up of commercially viable MOFs. However, this situation is likely changing. A number of MOF systems are being pursued for scale-up and commercialization including amine-grafted MOFs for CO2 capture70, MIL-160 for water harvesting71, CAU-10-H for water harvesting72, and CALF-20 also for flue gas CO2 capture73, with BASF producing tons using green chemistry74. Scaling up MOF synthesis can impact reproducibility, as changes in conditions affect crystal size, purity, and morphology. For more detail on large-scale MOF production, highlighting synthetic challenges, we refer the reader to the recent article by Chakraborthy et al.75.

Further areas worth specifically mentioning are defects and activation processes. Defects are common in MOFs and can have a significant impact on application-specific performance. Despite this, they are often not well characterized. When scaling up, varying levels of defects can have a tremendous impact. An illustrative example of this could be the impact resulting from the variability of defects in MOFs leading to the presence of open metal sites for catalysis76. If the material has a variable loading of open metal sites due to defects, it stands to reason that the material's performance will be variable. As part of standard characterization approaches required for MOFs, defects, both of linker and metal type, should be quantified and reported at both laboratory and scale-up stages, especially where such defects are likely to have a considerable impact on application outcomes. We note, however, that this is often not a simple and straightforward process. Fortunately, there have been some characterization successes with standard techniques including thermal gravimetric analysis (TGA) and nuclear magnetic resonance (NMR)77,78.

Activation is a crucial step in MOF preparation and can be challenging to conduct especially at scale. Over several years, within our group, we have identified a couple of ways to activate materials when transitioning from laboratory scale to pilot scale. These are the use of carrier gas to remove volatile species within the framework79 and the recognition that MOFs are quite thermally insulative. We have found that if He flows through the MOF-based material whilst heating, we end up with an activated product that has better performance than just heating under a vacuum. We have found this approach to be successful with a varied set of porous materials and consider this to be a valid approach to activate such materials including MOFs. We do, however, note the use of N2, which has both a lower thermal conductivity and specific heat capacity, does not have a similar enhancement. Further, if a large batch of porous material is heated in an oven as a clump, only the outer layer is suitably activated. A simple approach for overcoming this is the use of multiple sparsely layered trays with the MOF material.

For gas-based applications, shaping and pelletization of MOFs are crucial to facilitate efficient mass transport in scaled-up systems. We will focus on pelletization with a polymeric binder, one of the more common methods to produce MOF pellets. An almost universal problem is the partial pore blocking of the MOF with the polymeric binder—this reduces the MOF surface area and performance (e.g., total uptake). However, there is a delicate balance between the performance loss from pore blocking and the gains from the increased mass transport at scale. Efforts are ongoing to address these binder interactions and microstructure, which requires a multidisciplinary approach80,81.

Composite frameworks

A multitude of applications can be derived from composite materials due to their ability to combine distinct constituents with distinguishing characteristics, thus providing customized properties that exceed those of the constituents individually. It is, however, essential that reproducibility be achieved in order to utilize these benefits82,83. Cost management, synthesis conditions, maintaining consistency in material ratios, employing reliable characterization techniques, conducting quality control testing, and overseeing intermediate-scale assessments are among the factors to be considered68,84. Wherefore, as presented in Fig. 2, it has been Hill’s team’s objective to achieve synthesis replication, which has resulted in significant progress in the production of a variety of composite frameworks that have been produced efficiently, including porous material pellets (PMPs), magnetic framework composites (MFCs), PPMCs, and type II and III PLs, which are porous framework-solvent composites with exceptional reproducibility.

Fig. 2
figure 2

Key composite frameworks development history of Hill’s team (Reproduced with permission from ref. 98. Copyright 2014, Wiley-VCH85; Copyright 2018, American Chemical Society89; Copyright 2019, Wiley-VCH102; Copyright 2019, Elsevier88; Copyright 2019, American Chemical Society87; Copyright 2019, Royal Society of Chemistry103; Copyright 2020, American Chemical Society110; Copyright 2022, Tsinghua University Press112; Copyright 2023, Royal Society of Chemistry).

Full size image

Examining PMPs requires an in-depth assessment of their robustness, reliability, and operability. To accomplish this, various factors must be examined in depth, including the type of binders used and the quantity of them, as well as the shaping technique used, such as pelletization under pressure, foaming, extrusion, granulation, and cake crushing extrusion. PMP advancement is a significant step toward its commercial viability. In spite of this, the process of creating scalable PMPs is complex and requires tailoring for each specific porous material68,85.

In the area of MFCs, evaluating the long-term functioning/stability, effectiveness of the composite material, and dependability of magnetic characteristics86,87,88 involves cyclic performance tests (e.g., static and dynamic sorption and desorption experiments) as well as regeneration experiments using induction heating systems, also referred to as triggered release89,90,91. Even so, it is important to note that the development of scalable MFCs that have enhanced sorption and regeneration capabilities requires not only pellet formation and long-term stability but also high productivity and energy efficiency, as well as the ability to work at low regeneration temperatures while minimizing energy consumption85,92.

For instance, during the film casting process of PPMCs, the rate at which the solvent evaporates, the precise timing needed to form a porous framework dispersion within a polymer dope solution to ensure consistent agglomeration across batches, and colloidal stability of the resulting casting solution all play a key role93,94,95,96,97. In addition, to ensure a high level of consistency and reliability, several samples are generated and comparison studies are conducted. Also, in order to detect any changes in characteristics that have occurred over an extended period of time, long-term examinations are crucial98,99,100. Meanwhile, achieving scalable production of PPMCs requires a number of factors, including simple processing steps, a dual-layer structure, strong bonding between the polymer and porous framework, uniform particle distribution with minimal aggregation, and the absence of defects in active and substrate layers101,102,103,104. Yet, economically acceptable selectivity, enhanced performance, and reduced material expenditures can only be achieved by ensuring a high particle loading, uniformity in the thickness of both the active and substrate layers, and the use of low-cost polymers exhibiting a strong interlayer adhesion property105,106,107.

The transition from the laboratory to the practical application of type II and III PLs in a variety of industries requires an integrated strategy including synthesis, stability, repeatability, appropriate viscosity criteria for porous frameworks and solvent composites, recovery of porous frameworks following the synthesis of the PLs, applications, and economic viability. For instance, several factors contribute to repeatability, including buoyancy, gravity, and interactions between the porous framework and the solvent of the component materials. In order to ensure a functionally appropriate amount of open porosity for functional purposes, it is imperative to prevent the penetration of solvent into the pores of the porous framework108,109. In addition, the recovery of the material allows for its continuous use, thus reducing the need to synthesize new materials110,111. In order to achieve maximum performance, substantial consideration should be given to the energy consumption associated with the regeneration of the adsorbent and absorbent constituents of PLs112,113.

Characterization considerations in porous materials research

At first glance, characterizing porous materials may appear straightforward, however it is often fraught with difficulties114. However, by recognizing these traps, awkward mistakes can be avoided, and the data will be able to withstand critical scrutiny.

The BET method, developed in the 1930s for open surfaces and widely used for micro- and mesoporous materials, faces challenges in reproducibility. Researchers should be mindful of BET theory limitations, especially for microporous adsorbents. During a study in which Hill’s team participated, 18 raw adsorption isotherms were provided to sixty-one labs for calculation of the corresponding BET areas115. The results of this study showed clear reporting of the pressure range and data points is crucial for BET surface areas. Transparent presentation of isotherms, including a semi-log representation for low-pressure regions, is recommended. Emphasis should be placed on scrutinizing the adsorption isotherm rather than solely relying on the derived BET area. Additionally, using modern computational methods like the BET surface identification (BETSI) algorithm, based on the criteria suggested by Rouquerol et al., for selecting a suitable p/p˳ range, can be very useful for enhancing the transparency of data reported115,116,117,118.

While physisorption isotherms are generally reliable for rigid adsorbents, flexibility in materials can lead to variability. A study by Kaskel et al.119, analyzed 50 nitrogen physisorption isotherms at 77 K, correlating them with the synthetic and outgassing conditions of DUT-8(Ni), a “gate opening” MOF. The research highlights the importance of accurately documenting experimental details for the reproducibility of scientific results.

Despite the widespread interest in CO2 adsorption in porous materials, there are only a small number of MOFs for which firm conclusions can be drawn about the reproducibility of these measurements. The study by Sholl et al.120, reveals that approximately 20% of reported adsorption isotherms for alcohols in nanoporous materials are considered outliers, cautioning against the indiscriminate use of individual isotherms, an observation that is similar to earlier analyses of CO2 adsorption experiments66. The extended study by this group on the adsorption of alkanes in nanoporous materials shows that 15% of the replicate alkane isotherms are inconsistent with other replicate measurements.

The occurrence of outliers is attributed to variations in material properties stemming from synthesis and sample preparation, including but not limited to sample activation under vacuum or elevated temperature, which may have a marked impact on measured adsorption properties, especially for materials that strongly adsorb water when exposed to ambient conditions. Adsorption in structurally sensitive materials may also be affected by the history of the sample being used or degradation. Taking CuBTC as an example used in different projects in Hill’s team, it has aged over time if not stored under the proper conditions and the isotherms are not reproducible.

The current need to establish a universal format for archiving adsorption data is crucial. There are some studies like the one reported by Evans et al.121, who introduced a standard file, AIF, based on the self-defining text archive and retrieval (STAR) procedure, which is an easily extended free-format archive file that is both human and machine-readable. IUPAC has approved the AIF format, and we encourage authors to provide their isotherm data in the AIF development format, as part of their paper supporting information.

Additionally, studies like the one by Smit et al.122,123, in making connections between databases of gas adsorption experiments and databases of the atomic crystal structure of the corresponding materials can be helpful.

Despite certain research on this topic66,67,115,119,124,125,126,127,128,129, there is still a crucial demand for a more extensive investigation like a comprehensive study by Hirscher et al.130, on improving reproducibility in hydrogen storage material research to thoroughly assess the reproducibility of different characterization techniques.

As a general conclusion, to assess the properties of a given porous material comprehensively, it is recommended to use a combination of characterization methods such as X-ray diffraction, electron microscopy, and adsorption/desorption isotherms rather than focusing on a single method.

Forward-looking outlook

Overall, there is a lack of standardized protocols for both synthesis and characterization of porous materials. Without standardized methods, comparing results from different studies and reproducing experiments becomes difficult. It is imperative that the community uses the array of tools and techniques currently available to narrow the development period by rapidly translating ideas to products so that it is able to develop new framework materials from lab to industrial scale production with uniform protocols resulting in reproducible structures and materials.

However, having an extensive collection of materials131,132 opens up a number of exciting possibilities, but also presents some challenges; our data and structures are simply too numerous. Considering the large number of structures, there are many issues to be addressed, including how to manage so much data and how to use the data for the discovery of new science. To exploit the unreasonable effectiveness of data, materials scientists should collaborate with data scientists to apply the tools of big data science. To achieve this, it is evident that a clear protocol, open data sharing, and transparency in research practices can significantly increase the reproducibility of studies involving porous materials. As long as all experimental data is transparently shared, libraries/databases of combined experimental efforts involving predicted or hypothetical porous materials generated through density functional theory (DFT) calculations and molecular simulations can be developed133,134. This is a powerful technique for studying these materials and discovering complex correlations using big-data methods. A machine learning (ML) algorithm is then employed to screen these libraries/databases to identify the most promising materials for a particular application135,136. Nevertheless, for these challenges to be overcome, researchers, standardization organizations, and funding agencies must work together.

As seen in Fig. 3, key recommendations to research teams working at scale who are looking to improve reproducibility include:

  1. 1.

    Employing identical raw materials is of utmost importance. The influence of raw materials on reproducibility is often overlooked. It’s known that the purity and particle size of raw solids can vary between suppliers, potentially affecting crystallization outcomes.

  2. 2.

    Maintaining the same production size, reactor type, stirring system, recirculation, and other relevant aspects to achieve consistency in particle size and distribution. Particle size and distribution play a critical role, and they can vary significantly with any changes.

  3. 3.

    Using the same production equipment with documented clean-in-place (CIP) procedures. As MOFs are formed by a nucleation process, the surface of the reactor plays a huge role. It must be kept as consistent as possible. For this same reason, it is important to keep to a previously validated scale for all batch production processes.

  4. 4.

    Being mindful of drying and activation procedures. At scale, the highly thermally insulating nature of these materials means that desolvation is not facile. Care must be taken to conduct heat to all of the material.

  5. 5.

    Keeping detailed records of all parameters for each batch. Many of the variations in materials quality, e.g., defects, surface morphology, and surface charge, may not be immediately obvious in characterization processes but drastically change performance outcomes. MOFs and related materials have a ‘memory’ of their production and handling inbuilt; this should be carefully documented.

  6. 6.

    Carrying out comprehensive crystallization investigations on important porous materials, specifically within the domain of MOFs. This is because the formation mechanisms of most benchmark MOFs are not still well understood. These investigations could assist researchers in more accurately evaluating the robustness of MOF reproducibility at different scales.

Fig. 3
figure 3

A path towards improving reproducibility.

Full size image