Advanced in AI Audit AAIA Isaca Study Notes

Exploratory Data Analysis (EDA) is critical during the initial stages of AI model development. According to the AAIA™ Study Guide, performing EDA—including identifying null values, incorrect data types, or duplicates—ensures that the data fed into the model is clean and reliable.

“Initial data frames should be subject to thorough EDA to uncover data quality issues. These issues, if not addressed early, can severely affect model training and predictive accuracy.”

While separating data sets (B) and visualizations (A) are important steps in later phases, C is foundational to ensure readiness for model training. Risk assessments (D) are necessary but not the first operational step.

Large Language Models (LLMs) have the capacity to memorize and unintentionally reveal sensitive information if not properly managed. As per the AAIA™ Study Guide, data masking is a critical technique that prevents the exposure of personally identifiable information (PII) or confidential content by obscuring or replacing sensitive parts of the data during training or interaction.

“Data masking ensures that training data used for LLMs does not contain real sensitive identifiers. Unlike sanitization, masking modifies data to maintain utility while eliminating exposure risk.”

Manual monitoring and access controls are supportive security measures, and data sanitization helps remove content but may not preserve the data’s structure. Data masking offers the most proactive and technically robust solution.

Ensuring that AI tools are trained on properly licensed and documented data sets is critical to avoiding copyright infringement and legal exposure. The AAIA™ Study Guide emphasizes using platforms with certified and traceable training data to meet ethical and legal standards.

“Organizations must verify the provenance and licensing of data used to train AI systems. Platforms that certify data sources reduce the risk of using protected intellectual property without consent.”

Manual review (A) is resource-intensive and may not detect embedded copyright violations. Labeling (C) is not sufficient for legal protection. Suspension (D) may be excessive without first attempting remediation. Thus, B is the most strategic and effective recommendation.

Prompt engineering manipulation often involves disguising inappropriate questions in hypothetical or conditional formats, tricking the AI system into bypassing its filters. The AAIA™ Study Guide highlights this as a common attack vector in generative AI misuse.

“Adversaries may reframe queries as academic or theoretical scenarios to exploit AI models’ reasoning patterns, bypassing ethical constraints. Auditors should test for these circumvention techniques.”

Other methods (A, C, D) may work under specific conditions, but B is the most widely effective and documented prompt-engineering technique.