Artificial intelligence (AI) can be a valuable tool in the course design process. It can help instructors brainstorm ideas, generate learning activities, develop assessments, and organize course content. However, AI should be viewed as a tool that supports course design rather than a replacement for the expertise of the instructor. AI systems can make mistakes, generate inaccurate information, misunderstand context, and overlook the unique characteristics of a particular group of learners.

Empirical studies examining the use of generative AI for course design provide evidence of these limitations and illustrate why human oversight remains essential. For example, Choi et al. (2024) found that ChatGPT occasionally failed to adhere to user instructions. In one instance, despite being informed that a course would be delivered online, the tool generated learning activities that were best suited for face-to-face interaction. The researchers also found that ChatGPT sometimes proposed unrealistic learning tasks, such as requiring students to summarize an entire book within a single week, suggesting a limited ability to consider the time demands placed on learners. In addition, the tool occasionally recommended course readings that did not exist, raising concerns about the accuracy and reliability of its outputs.

Similarly, Farrokhnia et al. (2024) observed that ChatGPT may lack a deeper understanding of complex concepts and therefore produce responses that are superficial, insufficiently nuanced, or only loosely connected to the intended topic.  Additional weaknesses identified by Choi et al. (2024) include vague instructions, repetitive learning activities, and assessments that lacked depth and rigor.

These findings highlight the importance of approaching AI as a support tool rather than an autonomous course designer. To maximize its benefits while minimizing potential risks, course designers should be mindful of several common pitfalls:

Avoid Overreliance on AI

One of the greatest risks is treating AI as if it can design a course independently. AI should be viewed as an assistant rather than the primary designer. Although it can generate ideas and draft content, it does not possess the instructor’s disciplinary expertise, pedagogical knowledge, or understanding of the students who will take the course. As a result, all AI-generated content should be reviewed for accuracy, quality, relevance, and alignment with course objectives.

Avoid Letting AI Replace Human Creativity

AI can generate ideas, but it should not replace the instructor’s creativity and vision. Course designers should enter the process with their own goals and preferences and use AI to help develop and refine those ideas. For example, an instructor may already know that they want students to complete a group project, create a video presentation, or participate in collaborative discussions. AI can then be used to elaborate on these ideas, suggest improvements, or help organize activities into a coherent learning experience.

Similarly, instructors can enrich AI-generated activities by incorporating real-world examples, professional experiences, local contexts, and knowledge of their students’ backgrounds and interests. These are elements that AI often cannot provide effectively.

Ignoring Potential Bias

Because AI systems are trained on existing data, they may reflect or reproduce biases present in those data (Zhai, 2022). Biases can appear in examples, case studies, images, language, or assumptions about learners. For instance, AI-generated images or scenarios may not adequately represent the diversity of a particular classroom. Course designers should therefore critically evaluate AI-generated content to ensure it is inclusive and equitable.

Avoid Accepting AI Output Uncritically

Not every recommendation generated by AI should be used. AI may suggest activities that are unrealistic, assessments that lack rigor, or content that does not align with the instructor’s goals. Course designers should exercise professional judgment and feel comfortable revising, refining, or even discarding AI-generated suggestions. AI can provide options, but instructors remain responsible for selecting what best supports student learning.

Avoid Privacy and Data Security Risks

When using AI tools, course designers should be careful not to share confidential, proprietary, or personally identifiable information. Entering sensitive student information or institutional data into AI platforms may create privacy and security concerns. Instructors should familiarize themselves with institutional policies and the terms of use of AI tools before sharing any information.

Unethical or Non-Transparent Use of AI

Course designers should also consider the ethical implications of AI use. Instructors should model responsible AI practices and be transparent about how AI is used in course development when appropriate. Transparency can help build trust with students and demonstrate ethical engagement with emerging technologies. Responsible use of AI involves recognizing its limitations, critically evaluating its outputs, and ensuring that human judgment remains central to the design process.

Ultimately, AI should be viewed as a collaborator rather than a course designer. While it can improve efficiency and support idea generation, responsibility for creating a meaningful, accurate, and effective learning experience remains with the instructor. In other words, course designers should always remain the “human in the loop.”