Auditing in the

Age of Generative AI
April 2024
About the Center for Audit Quality
The Center for Audit Quality (CAQ) is a nonpartisan public policy organization serving
as the voice of U.S. public company auditors and matters related to the audits of public
companies. The CAQ promotes high-quality performance by U.S. public company
auditors; convenes capital market stakeholders to advance the discussion of critical
issues affecting audit quality, U.S. public company reporting, and investor trust in the
capital markets; and using independent research and analyses, champions policies and
standards that bolster and support the effectiveness and responsiveness of U.S. public
company auditors and audits to dynamic market conditions.

Auditing in the Age of Generative AI

Please note that this publication is intended as general information and should not be relied on as being definitive or all-inclusive. As with all
other CAQ resources, this publication is not authoritative, and readers are urged to refer to relevant rules and standards. If legal advice or other
expert assistance is required, the services of a competent professional should be sought. The CAQ makes no representations, warranties, or
guarantees about, and assumes no responsibility for, the content or application of the material contained herein. The CAQ expressly disclaims all
liability for any damages arising out of the use of, reference to, or reliance on this material. This publication does not represent an official position
of the CAQ, its board, or its members.

2
 4 Introduction
Contents
Overview of GenAI:
5
What Auditors Need to Know

9 The Regulatory Environment

Considerations When Auditing

11
Companies That Are Deploying GenAI

16 Example Use Cases

18 Additional Audit Considerations

18 Conclusion

Auditing in the Age of Generative AI

3
Introduction

Collective interest in and awareness of generative AI (genAI) has grown

exponentially since the public release of several genAI chatbots powered One in three audit
by large language models beginning in November 2022. While artificial
intelligence (AI) and machine learning are not new, the accessibility partners see
and ease of use provided by genAI chatbots and similar large language
models have led to increased use by individuals and companies. A recent companies in their
CAQ survey found that one in three audit partners see companies in
their primary industry sector deploying or planning to deploy AI in their primary industry
financial reporting process.1 This number will likely continue to grow as
companies explore the ways in which AI, including genAI, can streamline sector deploying
or enhance accounting and financial reporting operations and processes.
or planning to
This publication explores some fundamental principles of genAI, new
risks arising from its use in processes relevant to financial reporting deploy AI in their
(financial reporting processes) or internal control over financial
reporting (ICFR), and related audit implications. Although some of the financial reporting
considerations discussed may also be applicable for other types of AI,
the focus of this publication is specifically on genAI. process.
CAQ’s Audit Partner
Pulse Survey, Fall 2023

Auditing in the Age of Generative AI

1 TheCAQ.org | Audit Partner Pulse Survey | Fall 2023

4
Overview of GenAI:
What Auditors Need to Know

In order for auditors to identify where and how companies are using
genAI in financial reporting processes and ICFR and the risks that could
arise from its use that may be relevant to the audit, it will be helpful to
have a foundational understanding of some fundamental principles
of genAI, including key features of the technology and how it differs
from other technologies that companies may be using. As the genAI
technology, use cases, and regulatory environment are rapidly changing,
it is important for auditors to continue to monitor developments.

WHERE DOES GenAI FIT WITH OTHER AI TECHNOLOGIES?

AI includes a broad range of technologies, of which genAI is a subset.

While there are other types of AI beyond those shown to the right, this
graphic depicts where genAI fits with other categories of AI technologies.

Artificial Intelligence | AI broadly refers to machines that mimic human- Artificial

like cognitive abilities. AI includes capabilities such as natural language Intelligence
processing, problem-solving, pattern recognition, anomaly identification, and
decision-making. An example of AI is an online language translation service. Machine Learning

Machine Learning | Machine learning is a subset of AI that uses algorithms

to learn from and make predictions or decisions based on data. Machine Deep Learning
learning algorithms are designed to learn and improve from experience.
Machine learning is useful for identifying patterns, extracting insights, and
making informed predictions. Different methods of machine learning include
GenAI
supervised learning, unsupervised learning, and reinforcement learning.2 An
example of machine learning is a system used by a streaming service that
provides recommendations to customers based on their viewing habits.

Deep Learning | Deep learning is a subset of machine learning that uses

algorithms that roughly approximate the structure and capabilities of the
human brain. Deep learning algorithms can simulate an array of neurons in
an artificial neural network that learns from vast sources of data enabling
the technology to handle complex tasks similar to how humans can. An
example of deep learning is driverless cars which can recognize and
respond to different situations on the road.
Auditing in the Age of Generative AI

GenAI | GenAI refers to a subset of deep learning based on probabilistic

technology that can create content, including text, images, audio, or video,
when prompted by a user. GenAI creates responses using algorithms that
are often trained on open-source information, such as text and images from
the internet.3 Through its ease-of-use, genAI has democratized artificial
intelligence making the technology accessible to any user, whereas other
types of artificial intelligence have generally only been accessible to data
scientists. AI chatbots, like ChatGPT and Copilot, are examples of genAI.

2F or further discussion of these methods of machine learning, refer to the AICPA and CPA Canada’s A CPA’s Introduction to AI: From Algorithms to Deep Learning,
What You Need to Know publication.
3 Science & Tech Spotlight: Generative AI | U.S. GAO 5
HOW DOES GenAI WORK?
CONSIDERATIONS FOR
Learning and Generating New Content AUDITORS
The probabilistic nature of genAI
GenAI technologies are trained on large datasets where they learn is a key distinction from other
patterns, structures, and representations from the training data. For technologies that auditors may
example, based on the training dataset, genAI learns grammar and have historically encountered in
a company’s financial reporting
syntax and uses its advanced predictive capabilities to mimic knowledge
processes, which may inform
on a wide range of topics. Based on this training data, when prompted by auditors’ identification and
a user, genAI technologies make predictions of the next character, word, assessment of risks of material
phrase, pixel, etc. to formulate a probable response to the user prompt.4 misstatement, including the
identification of process level risks
GenAI technologies are predictive technologies, and therefore, the or risks arising from IT. Further,
outputs are based on what the genAI technology has determined is a when performing audit procedures
probable response. If a user asks the same question multiple times, they over information generated by
might get different answers each time. Different answers may result genAI, auditors need to be aware
because genAI technologies are designed to generate varied responses that the information produced
and are trained on diverse datasets, which leads to a wide range of by genAI is not necessarily
factual and may not be able to
probable responses to a single prompt.5 Accordingly, genAI technologies
be replicated by the same genAI
are especially helpful for tasks that need creativity or diversity of technology, even if the same input
responses, including generating new content or information, but genAI is provided again, which may
may not always provide reliable or repeatable information. GenAI influence how auditors design
technologies do not work like search engines finding facts within their and execute audit procedures.8
training data but are instead creating new coherent, human-like text. Auditors’ responsibility to obtain
sufficient and appropriate audit
Foundation Models and GenAI Technologies Supported by Those evidence under applicable auditing
Models standards remains unchanged.

When developing and deploying genAI technologies, companies may

build and train their own models,6 or they may begin with a foundation
model. Foundation models are large language models that can be
adapted to a wide range of downstream tasks, providing the basis CONSIDERATIONS FOR
for various genAI technologies.7 There are many foundation models AUDITORS
currently available. One example is GPT-4, which is the foundation model A company may develop its own
used by one version of ChatGPT. This same foundation model can also model, build customizations on
be the basis for other applications. For example, a company could also top of a foundation model, or
use GPT-4 as the basis for its own internal chatbot. use a pre-built solution based
on a foundation model (such
as a publicly available chatbot)
Companies can build their own customizations on top of foundation
depending on their specific
models. Customizations may include incremental training with the needs. The risks arising from the
company’s own data and fine-tuning the model for specific uses within company’s use of genAI will vary
the company. Using a foundation model can allow companies to develop depending on the nature of the
custom genAI technologies without the significant effort involved in genAI technology. Auditors may
developing their own model. However, companies using foundation consider the following questions:
models may not have visibility into the data and methods used to train + Did the company develop its
the foundation model. own genAI model or is the genAI
Auditing in the Age of Generative AI

technology built on a foundation

model?
+ If the company is using a
4P  rompts are the information (such as a question, command, etc.) entered into a genAI technology to
generate a response.
foundation model, which
5 It is possible to configure certain genAI technologies to provide more deterministic responses (i.e., foundation model supports the
provide consistent and predictable responses). However, the diversity of the datasets that genAI genAI technology?
technologies are trained on will still lead to a range (albeit narrower) of probable responses to a prompt.
6A  lthough it is possible, it may be rare for companies to build and train their own genAI large language
+ Did the foundation model
models. require incremental training or
7 Explainer: What is a foundation model? | Ada Lovelace Institute customization to support the
8A  I hallucination is a phenomenon wherein a large language model (such as a genAI chatbot) perceives company’s use case?
patterns or objects that are nonexistent or imperceptible to human observers, creating outputs that are
nonsensical or altogether inaccurate. See further discussion at What are AI hallucinations? | IBM.
6
Explainability and Interpretability of GenAI
CONSIDERATIONS FOR
There is an increasing desire for genAI users to understand how and AUDITORS
why the technology arrives at certain conclusions, which relates to The impact of the black box
the explainability and interpretability of genAI. Explainability refers to concept on the audit generally
explaining or understanding the underlying mechanisms in the genAI depends on the factors
technology’s behavior – in other words, how the technology made the described to the left. Effective
decision.9 Interpretability refers to when humans can readily understand human oversight to address
explainability and interpretability
the output of the genAI technology through the reasoning behind
risks becomes important
predictions and decisions made – in other words, why the technology specifically as companies
made the decision.10 place heavier reliance on genAI
technologies, use cases in
A challenge of AI is that it can be a “black box,” meaning that the process financial reporting processes and
to arrive at a specific output is not readily explainable or interpretable, ICFR become more sophisticated,
resulting from the inherent complexity of AI algorithms and the nonlinearity and outputs from the technology
of the relationships between the underlying data and the outputs or are unable to be independently
decisions made. While this challenge exists for all types of AI, including replicated. The following
genAI, explainability and interpretability needs will vary depending on questions may be helpful for
a number of factors, including the level of reliance on the technology auditors to consider:
(i.e., whether the technology is used to augment work performed by an + Is the company placing
employee or replacing the employee), the nature or type of the output (i.e., reliance on genAI technology to
whether the output can be independently replicated by a human reviewer), generate outputs that are not, or
and the level of human in the loop involvement (see further discussion cannot be, verified or replicated
by employees?
in the Responding to Identified Risks section). Additionally, the ability to
explain and interpret outputs may be impacted by whether the technology + If the company is placing
is built on a foundation model or a model developed by the company (i.e., reliance on employees to
whether the company controls the underlying algorithms). These factors review the output from genAI
are important for auditors to consider how the use of genAI technologies technology, how has the
company determined that
impacts the company’s financial reporting processes or ICFR and the
employees have the appropriate
related audit response in tests of controls or substantive procedures. knowledge and skills to do so?

Explainable AI (XAI) is an emerging area of research focused on + If the company is placing
reliance on genAI technology,
techniques to enhance the explainability and interpretability of AI
how has the company
(including genAI). Some of these techniques include embedding features
determined that the genAI
that can provide information regarding the AI technology’s confidence technology is sufficiently
in its outputs or decisions or to document the key elements of the input explainable and interpretable
that the AI technology focused on to make its decision. While embedding for the intended use?
such features may not be feasible for existing technologies, particularly
those genAI technologies built on a foundation model, it may be possible
to add certain features on top of genAI foundation models to enhance
explainability and interpretability.

WHY AND HOW ARE COMPANIES DEPLOYING GenAI?

Companies are noting significant opportunities from deploying genAI,

particularly from using genAI to enable knowledge workers to perform
Auditing in the Age of Generative AI

their jobs more efficiently and effectively. GenAI can help employees
streamline certain activities such as those that involve drafting content,
summarizing data, and working with unstructured data, among others,
which frees them up to focus on more challenging, analytical, or higher-
risk tasks. Further, genAI can uncover trends, patterns, and anomalies
in large amounts of data that would otherwise be difficult or time-
consuming for human employees to uncover manually.

9A
 rtificial Intelligence Risk Management Framework (AI RMF 1.0) (nist.gov)
10 Ibid.
7
Generally, companies deploying genAI within financial reporting
processes will initially use it to augment processes (rather than fully CONSIDERATIONS FOR
automate them), which enables efficiency but does not eliminate AUDITORS
human judgment and decision-making. Particularly in financial reporting Automation technologies and
processes and ICFR, humans continue to be involved to oversee, genAI have different risks;
understand, and evaluate the relevance and reliability of the outputs therefore, it is important that
from genAI technology. In the future, companies may evolve to deploy auditors understand the type of
more advanced and complex use cases or decrease the level of human technology that a company is
using. Auditors may ask:
involvement.
+ Is the technology rules-based
HOW DOES GenAI COMPARE TO OTHER AUTOMATION (i.e., performs the task the
TECHNOLOGIES? same way each time) or is it
probabilistic (i.e., involves a
degree of variation and is not
Automation technologies, such as robotic process automation (RPA),
programmed to perform the
have been used for several years by accounting and financial reporting task the same way each time)?
professionals to automate routine and repetitive tasks. While automation
technologies can be beneficial to automate tasks that are performed the +D
 oes the technology only
accept inputs in a specific
same way every time, they typically cannot handle situations where the
format, or can it accept
format or structure of data is different from how it was programmed. unstructured inputs?
GenAI can address these limitations by providing the ability to accept
unstructured inputs with greater variation. Since genAI has the potential +F
 or processes that have been
automated, is the company
to integrate with other technologies, task automation may look very
using RPA or similar automation
different when using genAI compared to traditional automation using
technologies, genAI, or a
RPA that is focused on replicating repetitive tasks. combination?

Auditing in the Age of Generative AI

8
The Regulatory Environment

As the use of AI technologies, including genAI, evolves, there have

been increased calls globally for stronger regulations related to the CONSIDERATIONS FOR
safe and responsible development and use of AI, including genAI. AUDITORS
Although existing regulations in many countries already govern the use New regulations may result in
and protection of data or emerging technologies and are applicable changes to a company’s ICFR
to AI, many countries have also begun to adopt new regulations and that could impact the audit. For
frameworks specifically to mitigate security and safety risks of AI as well example, new regulations may
as to advance the ethical and responsible use of AI. The regulations and necessitate additional entity-level
controls for the governance of AI
voluntary frameworks discussed herein are not all inclusive.
as well as updated policies and
procedures related to the training,
WHITE HOUSE EXECUTIVE ORDER ON SAFE, SECURE, AND development, use, and ongoing
TRUSTWORTHY ARTIFICIAL INTELLIGENCE monitoring of AI technologies.
Auditors are also responsible
In October 2023, President Biden issued an executive order (Executive for considering certain laws and
Order 14410) focused on seizing opportunities presented by AI and regulations and the possibility of
managing the related risks.11 Among other things, the executive order illegal acts by companies.13
directs federal agencies to establish new standards for AI safety and
security, protect data privacy, advance equity and civil rights, support
workers, promote innovation and competition, and establish the
government’s own responsible AI program. It requires the developers of
certain powerful AI systems to share safety test results and other critical
information about those AI systems with the federal government and
directs the establishment of rigorous standards for testing AI systems to
ensure their safety before public release. In addition to existing state and
federal laws that govern the use of data or AI technologies, the executive
order is a significant step towards more robust AI regulation in the US.

EU ARTIFICIAL INTELLIGENCE ACT (AI ACT)

In March 2024, Members of the European Parliament approved the EU AI

Act, a comprehensive legal framework for regulation of the development
and use of AI systems, including general purpose AI.12 It employs a risk-
based approach that prohibits certain uses of AI such as social scoring
based on social behavior or personal characteristics and requires the
development of appropriate guardrails to mitigate risks to society
for high risk AI systems around data quality, transparency, testing,
monitoring, reporting, security, human oversight, and accountability. This
regulation will impact US companies that operate in the EU or develop AI
models that are used in the EU.
Auditing in the Age of Generative AI

VOLUNTARY RISK MANAGEMENT FRAMEWORKS

In addition to complying with new regulations, some companies are

applying the principles of voluntary AI risk management frameworks
to responsibly use AI, including genAI. The National Institute of
Standards and Technology (NIST) published the AI Risk Management
Framework, which can be voluntarily used by organizations to incorporate

11 FACT SHEET: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence | The White House
12 Regulatory framework proposal on artificial intelligence | Shaping Europe’s digital future (europa.eu)
13 As required by PCAOB AS 2405, Illegal Acts by Clients.
9
trustworthiness considerations into the design, development, use,
and evaluation of AI products, services, and systems.14 It provides a
framework to manage risks arising from AI that could affect individuals,
organizations, and broader society. The framework is structured around
four key pillars – govern, map, measure, and manage – designed to
help organizations identify and assess potential risks associated
with AI. Further, in response to the executive order described above,
NIST is expected to develop additional guidelines, standards, and
processes for AI safety and security, including topics related to genAI
risk management, AI evaluation, and security testing. Frameworks and
guidelines in this area are rapidly evolving.

Additionally, COSO released the Realize the Full Potential of AI: Applying
the COSO Framework and Principles to Help Implement and Scale AI
guidance, which is designed to help companies apply the COSO ERM
Framework to the use of AI.15 Specifically, the guide focuses on the
need for organizations to design and implement governance, risk
management, and oversight strategies and structures to realize the
potential of humans collaborating with AI. The International Organization
for Standardization (ISO) also published several voluntary standards
related to mitigating risks arising from AI, including ISO/IEC 23894, which
provides guidance on AI-related risk management for organizations,
and ISO/IEC 42001, which specifies requirements for establishing,
implementing, maintaining, and continually improving an Artificial
Intelligence Management System (AIMS).16

Finally, some companies are developing their own principles for ethical
and responsible use of AI. These principles focus on key concepts such
as accountability, reliability, transparency, explainability, and security,
among others.

Auditing in the Age of Generative AI

14 AI Risk Management Framework | NIST

15 A rtificial Intelligence | COSO. The COSO ERM Framework differs from the COSO Internal Control – Integrated Framework. The ERM framework focuses on
broader strategic objectives than the Internal Control – Integrated Framework.
16 ISO/IEC 23894:2023 - Information technology — Artificial intelligence — Guidance on risk management and ISO/IEC 42001:2023 - Artificial intelligence —
Management system
10
Considerations When
Auditing Companies That Are
Deploying GenAI

POTENTIAL RISKS ARISING FROM DEPLOYING GenAI

As the auditor obtains an understanding of how genAI is used in financial reporting processes and ICFR and
the overall governance and oversight of genAI, the considerations described in the table below may help the
auditor determine how the company’s use of genAI technologies may impact the auditor’s identification and
assessment of risks of material misstatement, including the identification of process level risks or risks arising
from IT. The considerations described below are not all-inclusive and will vary based on the company’s facts and
circumstances.17

Potential Risk Area Example Risks or Sources of Risks Questions for Auditor Consideration

Governance + AI solutions are not identified + Who (individual or group) in the company is responsible
and managed appropriately and for oversight of the use of genAI?
consistently across the company. + Has the company developed a framework for responsible
use of genAI?
+ Has the company established policies regarding the
acceptable and ethical use of genAI?
+ How are policies regarding acceptable and ethical use
of genAI documented and communicated to appropriate
individuals throughout the company?
+ How does the company monitor compliance with policies
regarding acceptable and ethical use of genAI?
+ Does the company have a process to track and monitor
the use of genAI throughout the company, including use
by third-party service providers?
+ How does the company evaluate the impact (nature and
affected groups) of genAI technologies being deployed?
+ How does the company track risks arising from the use
of genAI technologies and mitigating responses?
Auditing in the Age of Generative AI

17 The considerations described herein are not necessarily unique to genAI technologies and may also be applicable for other types of artificial intelligence.
11
 Potential Risk Area Example Risks or Sources of Risks Questions for Auditor Consideration

Regulatory + The company’s use of genAI + What are the applicable laws and regulations impacting
technologies violates contractual the company’s use of genAI technologies?
agreements, laws, or regulations. + Do the company’s policies and procedures to monitor
compliance with laws and regulations include newly
enacted and changes to existing laws and regulations
related to genAI?
+ Does the company have contractual agreements that may
impact how the company can use genAI technologies?
+ Has the company performed a regulatory, legal, and
contractual compliance assessment to understand
considerations for the design, deployment, and use of
genAI technologies?
+ If the company uses genAI technologies developed by
a third party, is the company able to obtain sufficient
information from the third-party provider regarding
compliance with applicable laws, regulations, and
contractual obligations?
+ How does the company monitor genAI technologies over
time to determine if bias has been introduced through the
algorithms or the data that could result in noncompliance
with laws, regulations, and contractual obligations?

Knowledge and + Individuals in governance or + Has the company identified specialized skills or
Skills management positions do not have knowledge needed to assist with oversight, development,
the appropriate knowledge and skills deployment, operation, and monitoring of genAI
to provide effective oversight of the technologies?
company’s approach to deploying
+ How does the company provide training for employees
genAI.
and management who are responsible for oversight,
+ The company does not have skilled developing, deploying, operating, or monitoring genAI
resources to successfully oversee, technologies?
develop, deploy, operate, and monitor
+ How does the company educate employees and
genAI technologies.
management on responsible use of genAI, including
+ The company does not provide an understanding of the risks for AI hallucinations and
sufficient training to employees to guardrails on the ability to rely on the outputs?
use genAI technologies effectively
+ Does the company provide resources (such as user
and as designed or employees
manuals and real-time support) for specific genAI
inappropriately rely on genAI
technologies to employees?
technologies (automation bias).18
+ Does the company hire or engage third-party resources
with the required expertise to help ensure successful
oversight, development, deployment, operation, and
monitoring of genAI technologies?
Auditing in the Age of Generative AI

Fraud + GenAI technologies are used by +H

 ow has the company considered genAI technologies in
employees, management, or third its fraud risk assessment?
parties to perpetrate and conceal +H
 as the company identified new incentives, opportunities,
fraud. or pressures to commit fraud due to the deployment of
genAI technologies?

18 A
 utomation bias is a tendency to favor outputs generated from automated systems, even when human reasoning or contradictory information raises questions
about whether such output is reliable or fit for purpose.
12
 Potential Risk Area Example Risks or Sources of Risks Questions for Auditor Consideration

Data Privacy + The company’s confidential data is + How does the company consider data privacy risks when
mismanaged because it is entered selecting or developing genAI technologies?
into a genAI technology (some + Does the company use a public instance of genAI
third-party genAI technologies track technologies that tracks and saves inputs and data
and save all inputs to use for further that are accessible by third parties or a private instance
development of the technology). where inputs and data are tracked and saved only by the
company?

Security + The company’s genAI technology + How does the company consider cybersecurity risks
is susceptible to cyber-attacks, when selecting or developing genAI technologies?
including data poisoning,19 malicious
+ Has the company performed a cybersecurity risk
prompt injections,20 or malicious assessment to evaluate threats and safeguards?
overriding of prompts.

Selection and + The company selects or develops + How does the company identify appropriate processes
Design of GenAI a genAI technology that does not that are suited for augmentation by genAI?
Technologies achieve the desired objective. + How does the company design genAI technologies,
including determining which genAI technologies to
use (such as, selecting an existing genAI technology,
using a foundation model with added customizations,
or developing the company’s own model) and the data
needed for those technologies?
+ How does the company select third-party genAI
technologies for use?
+ Has the company developed clear objectives and related
success criteria for genAI technologies?
+ How are genAI technologies configured within the
company’s IT environment?

Use of a Foundation + The foundation model is unreliable + How does the company consider whether the foundation
Model resulting in repeated errors or a model is appropriately suited for the company’s needs?
favoring of certain results or outputs + How does the company evaluate the model for bias?
For genAI
by the model.
technologies + How does the company determine whether to add
that use a customizations to the foundation model to meet the
foundation model company’s specific needs?
(with or without
customizations from
the company) Auditing in the Age of Generative AI

19 D
 ata poisoning involves deliberately providing genAI technologies with unreliable data to influence the initial training, ongoing learning, or future retrieval,
leading the technology to provide unreliable outputs.
20 M
 alicious prompt injections involve prompting genAI technologies to provide unreliable outputs. Malicious prompt injections can be direct (a user provides a
malicious prompt to the genAI technology) or indirect (malicious prompts are hidden in or disguised as data).
13
 Potential Risk Area Example Risks or Sources of Risks Questions for Auditor Consideration

Model Training and + The methods used to train the genAI + How does the company evaluate the sufficiency of
Development model are insufficient or otherwise training of the genAI model?
not appropriate resulting in repeated + How does the company evaluate the model for bias?
Applicable for
errors or a favoring of certain results
genAI technologies + How does the company evaluate the training data for
or outputs by the model.
that use a model reliability and data quality?
developed by the + The training of the genAI model
company and introduces biases of the human
for incremental programmer resulting in repeated
customizations to a errors or a favoring of certain results
foundation model by or outputs by the model.
the company + The data used by the company to
train the model is biased or otherwise
not reliable resulting in repeated
errors or a favoring of certain results
or outputs by the model.

Model Performance + GenAI technologies do not + How does the company test genAI technologies prior to
consistently operate in accordance deployment to determine that they operate as designed?
with their intended purpose and at an
+ How does the company assess the relevance and
appropriate level of precision.
reliability of genAI outputs for the intended purpose?
+ GenAI technologies provide
+ Does the company measure, track, and communicate
incomplete, inaccurate, or unreliable
performance metrics related to the functioning of
outputs (AI hallucinations).
the genAI technologies, including the precision of the
+ GenAI technologies provide outdated technology?
or other information that is not
relevant.21

Prompts + Prompts entered into genAI + How has the company trained employees operating genAI
technology by employees are not technologies about appropriate prompts?
appropriate to achieve the intended +D
 oes the company have standardized prompts for
output from the genAI technology. employees to use when operating genAI technologies?
+ If prompts include data, how has the company
considered the reliability of data used in the prompt?

Ongoing Reliability + GenAI technologies are not + How does the company monitor the ongoing
and Monitoring monitored after deployment effectiveness of genAI technologies for the intended
to determine whether they are purpose?
functioning appropriately. + Does the company have a process to periodically
+ After deployment, genAI technologies reevaluate genAI technologies to determine whether they
do not continue to function as are functioning as intended?
designed due to the technologies’
Auditing in the Age of Generative AI

+ How does the company monitor changes to genAI

evolution over time or to intentional technologies?
or unintentional changes to genAI
technologies.

21 G
 enAI technologies often do not have access to real time data and information (as the data that the model is trained on is only through a specific point in time),
and therefore, genAI technologies may state information that is correct based on its training data but is not currently relevant.
14
RESPONDING TO IDENTIFIED RISKS
CONSIDERATIONS FOR
Human in the Loop AUDITORS
Auditors may consider the
For many current genAI use cases in financial reporting processes following questions related to
and ICFR, keeping a human involved in the process (“a human in the human in the loop involvement:
loop”) may address some of the risks arising from its use. Keeping +H
 ow does the company
a human in the loop means that employees are responsible for determine the appropriate
performing the following, as appropriate, (a) reviewing the accuracy and level of human in the loop
completeness of company inputs entered into the genAI technology, involvement with genAI
(b) understanding the explainability and interpretability of the outputs technologies?
from the genAI technology, and (c) reviewing the outputs from the genAI +H
 ow does the company
technology to determine their quality, reliability, and appropriateness. develop processes to
Generally, keeping a human in the loop can support the identification of promote appropriate human
inaccuracies, including incomplete output from the genAI technology. in the loop involvement in
The level of human involvement, including the review of inputs and reviewing outputs from genAI
outputs, may vary depending on the genAI use case, is commensurate technologies?
with the risk profile and environment that the genAI technology operates +H
 ow does the company
in, and may evolve over time. Human involvement with genAI technology consider explainability and
requires a high degree of vigilance and professional skepticism. interpretability needs of users
to enable effective human in the
loop involvement with the genAI
Audit Response: Internal Control Considerations
technology?
Based on the auditor’s risk assessment, the auditor may determine
whether it is necessary to test certain control activities related to
the company’s use of genAI.22 When there is a human in the loop, an
auditor’s evaluation of the design and operating effectiveness of such
control activities may include the sufficiency and appropriateness of the
employee’s review of the completeness, accuracy, and relevancy of the
output from the genAI technology. For example, auditors may consider
if the reviewer appropriately considered the unique risks related to
genAI when performing their review. It is important to note that control
activities related to the human involvement with genAI are supported by
appropriate entity-level controls or general IT controls.

Auditing in the Age of Generative AI

22 W
 hen performing an integrated audit in accordance with PCAOB AS 2201.39, “[t]he auditor should test those controls that are important to the auditor’s
conclusion about whether the company’s controls sufficiently address the assessed risk of misstatement to each relevant assertion.”
15
Example Use Cases

While use cases will vary based on a company’s operations, processes,

and specific facts and circumstances, the following examples While genAI
demonstrate how auditors may encounter genAI in a company’s financial
reporting processes and ICFR. is used in the
Drafting Financial Statement Disclosures process, there
Company X includes required property, plant, and equipment (PP&E) is still a high
disclosures in its annual financial statements. Previously, an employee
involved in the company’s financial reporting process prepared the draft level of human
disclosure based on underlying supporting schedules and general ledger
data. To enhance the efficiency of the process, Company X deployed involvement as
genAI technology to prepare the first draft of the disclosure using the
prior year disclosure, underlying schedules, and data from the general the employee
ledger. The financial reporting employee is now responsible for reviewing
the draft disclosure prepared by the genAI technology. The disclosure reviews and
then goes through the existing review process in which the assigned
reviewer considers that the work was prepared by genAI and the verifies the
associated risks. While genAI is used in the process, there is still a high
level of human involvement as the employee reviews and verifies the disclosure drafted
disclosure drafted by the genAI technology.
by the genAI
In its risk assessment, Company X may identify risks related to model
performance, among other risks. For example, as it relates to model technology.
performance, Company X identified two risks:

1. The disclosure prepared by the genAI technology is not complete or

accurate based on the underlying data.

2. The disclosure prepared by the genAI technology is not appropriate

because there have been updates to US GAAP PP&E disclosure
requirements after the cut-off of the model’s training data.

Company X determined that both risks are mitigated by the following

existing control, with certain enhancements to address that the work was
performed using genAI:

1. C
 ompany X has a control in which an employee with appropriate
authority and competence (i.e., knowledge of US GAAP and the
disclosure requirements related to PP&E) reviews the draft PP&E
Auditing in the Age of Generative AI

disclosure to validate that the disclosure is complete and accurate,

agrees to the underlying schedules, and includes all information
required by US GAAP.

Note that there may be additional ICFR considerations addressed

through entity-level controls or general IT controls.

16
Drafting Code for Reports
FUTURE STATE
Company Y has an internal control (control A) whereby all significantly In the future, as Company Y
aged receivables are reviewed by an individual with appropriate increases reliance on genAI, there
authority and competence to evaluate their collectability and assess is a potential for this example
the appropriateness of the related allowance, or lack thereof. In the to evolve to the point where
performance of control A, the control operator relies on report A, which the control operator prompts
lists key attributes for all outstanding receivables aged greater than 30 a genAI technology to make
desired changes to the report
days, including customer name, receivable amount, days outstanding,
(e.g., additional fields added) and
and allowance amount (if applicable), among other attributes. the genAI technology prepares
the code and, with additional
The control operator generates report A using a report writer tool based programming or interfaces, puts
on SQL code. Previously, if the control operator needed modifications to it into production without any
the report, an employee would write the code. Now, when updates are human involvement.
needed, an employee uses genAI to write the SQL code. The employee,
who has appropriate expertise in SQL, reviews the code drafted by the
genAI technology before the updates are made. Humans remain in the
loop to review and verify the appropriateness of the code drafted by
genAI, and the code goes through the normal testing protocols prior to
those changes being finalized.

With respect to the SQL code, Company Y may identify risks arising from
the use of genAI, such as model performance. Specifically, Company
Y identified a risk that the code prepared by genAI does not produce
complete and accurate results. Company Y determined that this risk is
mitigated by the following existing controls:

1. C
 ompany Y has a control in which an employee with appropriate
authority and competence (i.e., knowledge of SQL code) reviews
and approves any changes to report codes prior to implementing the
changes.

2. C
 ompany Y has a control in which the report code is tested in a non-
production environment prior to the code being implemented into
production.

Note that there may be additional ICFR considerations addressed

through entity-level controls or general IT controls.

Auditing in the Age of Generative AI

17
Additional Audit Considerations

KNOWLEDGE AND SKILL OF THE AUDIT ENGAGEMENT TEAM

When auditing companies that are deploying genAI technologies,

auditors will consider whether the audit engagement team has the
appropriate knowledge and skills to identify, evaluate, and respond to
risks of material misstatement, including process level risks or risks
arising from IT, related to the company’s use of genAI. Based on the
skillset of the audit engagement team members, it may be necessary
to complete additional training related to genAI technologies. In other
cases, the audit engagement team may determine that it is appropriate
to involve an individual with the requisite knowledge, skill, and ability
related to genAI.

FUTURE STATE

As the use of genAI technologies evolves and companies place more

reliance on genAI technologies, audit procedures will likely need to
evolve as well. For example, the nature of outputs from genAI technology
may be complex or otherwise unable to be independently replicated or
verified by a human in the loop or tested by an auditor. In these cases,
it will be critical for companies to have appropriate processes and
controls surrounding the use of genAI, including human oversight of
the genAI technology, which are designed to respond to the associated
risks. To obtain sufficient appropriate audit evidence, tests of controls
are required when substantive procedures alone are not sufficient.23
Auditors will need to consider the implications of the company’s use of
genAI technologies on tests of controls and substantive procedures in
making a conclusion about whether sufficient appropriate audit evidence
has been obtained. Obtaining sufficient appropriate audit evidence when
companies place increased reliance on genAI technologies will be an
area of continued focus and potential challenge for auditors.

Conclusion Auditing in the Age of Generative AI

The use of genAI in financial reporting processes or ICFR by companies

introduces new risk considerations for auditors. It is important for
auditors to be mindful of the risks and challenges that can arise from
a company using genAI. Auditors are well-suited to apply and build on
their expertise in identifying and assessing risks, exercising professional
skepticism, and developing appropriate audit responses.

23 Refer to PCAOB AS 2301.17.

18
 We welcome
your feedback!
Please send your comments or
www.thecaq.org questions to hello@thecaq.org