AI IN PHARMACY
Implementation Guide
April 2025 | White Paper
Nicholas Hui
Cofounder & CPO
MedMe Health
MedMe Health
Table of Content
1. Executive Summary
2. The Implementation Journey: 8-Step Overview
2.1 Start With Low-Risk, High-Value Use Cases
2.2 Assemble Your Implementation Champion(s) & Assess Your Readiness
2.3 Select the Right Tool or Vendor
2.4 Design for Workflow Integration
2.5 Train and Prepare Your Team
2.6 Launch and Iterate on the Pilot
2.7 Full Rollout and Continuous Monitoring
2.8 Create a Long-Term AI Strategy
2.2 Assemble Your Implementation Champion(s) & Assess Your Readiness
2.3 Select the Right Tool or Vendor
2.4 Design for Workflow Integration
2.5 Train and Prepare Your Team
2.6 Launch and Iterate on the Pilot
2.7 Full Rollout and Continuous Monitoring
2.8 Create a Long-Term AI Strategy
3. Conclusion
4. Appendices
4.1 Appendix A: Quick-Start Readiness Checklist (People, Process, Tech)
4.2 Appendix B: AI Vendor Evaluation Scorecard & Checklist
4.3 Appendix C: Works Cited
4.2 Appendix B: AI Vendor Evaluation Scorecard & Checklist
4.3 Appendix C: Works Cited
Executive Summary
The pharmacy landscape is shifting rapidly. Once primarily focused on dispensing, pharmacies are transforming into vital hubs for clinical services, patient counseling, and key access points for primary care. However, this evolution is faced with significant operational pressures, including workforce shortages, staff burnout, and the growing consumer expectations for convenient, digitally-enabled experiences. To stay ahead, pharmacy leaders must explore innovative but practical solutions to enhance efficiency and unlock clinical capacity, while maintaining the highest standards for care safety and quality.
Artificial intelligence (AI), supercharged by recent advancements in large language models (LLMs), presents a powerful set of tools to address these needs. Our previous white paper, "AI in PharmacyAn Overview" 1, aimed to illuminate the why–highlighting the diverse opportunities for AI across pharmacy operations and building the case for its adoption. However, for most pharmacy teams, the real challenge isn’t understanding AI’s potential, but putting it into practice.
This white paper serves as the essential follow-up: the tactical, step-by-step how-to guide. It moves beyond conceptual discussions to provide actionable advice on successfully integrating AI into your pharmacy environment.
Artificial intelligence (AI), supercharged by recent advancements in large language models (LLMs), presents a powerful set of tools to address these needs. Our previous white paper, "AI in PharmacyAn Overview" 1, aimed to illuminate the why–highlighting the diverse opportunities for AI across pharmacy operations and building the case for its adoption. However, for most pharmacy teams, the real challenge isn’t understanding AI’s potential, but putting it into practice.
This white paper serves as the essential follow-up: the tactical, step-by-step how-to guide. It moves beyond conceptual discussions to provide actionable advice on successfully integrating AI into your pharmacy environment.
The insights and recommendations presented here are not merely theoretical; they are derived from real-world experience in successfully productionizing AI tools within pharmacy workflows.
Figure 1: Breakdown of service delivery time across appointment types
Figure 1: Breakdown of service delivery time across appointment types
.png)
Avg. RPh Admin Time and Avg. Other Admin Time represent key areas where AI efficiency gains can be made
Page 3 of 33
The Implementation Journey: An Overview of the Steps:
Successfully implementing AI requires a structured approach.
This paper outlines an 8-step framework designed to guide pharmacies of all sizes – from independent owner-operators to large enterprise health systems – through the process, ensuring a thoughtful, strategic, and ultimately successful adoption journey:
Successfully implementing AI requires a structured approach.
This paper outlines an 8-step framework designed to guide pharmacies of all sizes – from independent owner-operators to large enterprise health systems – through the process, ensuring a thoughtful, strategic, and ultimately successful adoption journey:
1. Start With Low-Risk, High-Value Use Cases:
Identify initial applications that offer significant efficiency gains with minimal risk, building foundational experience and confidence.
2. Assemble Your Implementation Champion(s) & Assess Your Readiness:
Assemble the right stakeholders and evaluate your organization's preparedness across people, processes, and technology
3. Select the Right Tool or Vendor:
Navigate the options to choose an AI solution and partner that aligns with your needs, capabilities, and values.
4. Design for Workflow Integration:
Thoughtfully integrate the AI tool into existing routines to minimize disruption and maximize usability.
5. Train and Prepare Your Team
A Equip your staff with the knowledge and confidence to effectively utilize AI-enabled workflows
6. Launch and Iterate on the Pilot:
Test the AI solution in a controlled environment, measure its impact, and refine it based on real-world feedback
7. Full Rollout and Continuous Monitoring:
Scale the successful pilot across the organization while establishing processes for ongoing performance tracking and improvement.
8. Create a Long-Term AI Strategy:
Move beyond point solutions and build systems, embedding AI into workflows rather than layering it on top, so it becomes a core part of how your pharmacy operates
Page 4 of 33
By following these steps, pharmacies can turn their AI ambitions to tangible results, harnessing its value responsibly and effectively to enhance operations, empower staff, and improve patient care.
Step 1: Start With LowRisk, High-Value Use Cases
The first step to using AI effectively in pharmacy is choosing where to start. While the potential applications of AI in pharmacy are broad, ranging from administrative support to complex clinical decision-making, the most prudent approach begins with use cases that pair substantial value with minimal risk. This strategy not only delivers early wins but also builds crucial organizational capacity for future, more advanced AI implementations.
Clinician-Facing Admin:
In healthcare, risk should drive where AI is deployed first. A simple way to size up the risk is to consider the "distance from the patient" – the closer an AI tool operates to direct clinical decision-making and patient interaction, the higher the potential risk if the AI errors or "hallucinates" (produces incorrect or nonsensical output).3 Conversely, AI tools focused on administrative tasks performed by clinicians, but not requiring core clinical judgment, are inherently lower risk. Errors in these areas, while needing correction, are less likely to directly result in patient harm.3
These administrative tasks often represent significant capacity drains on pharmacists and technicians.4 Activities like documentation, managing refills, handling routine inquiries, or navigating prior authorizations consume hours that could otherwise be dedicated to patient counseling, clinical service delivery, medication therapy management, and other activities requiring specialized skills.5 Automating or augmenting these tasks with AI directly addresses operational inefficiencies and contributes to mitigating staff burnout, a critical issue exacerbated by workforce shortages
Furthermore, starting with clinician-facing administrative tasks provides a safe environment for the organization to learn how to work with AI. Traditional software is deterministic: the same input always produces the same result. AI, by contrast, is probabilistic and can produce variable outputs depending on context.1 Staff need to develop new skills in reviewing, validating, and sometimes correcting AI-generated content. Practicing these skills on lower-risk administrative outputs, such as draft documentation or inventory forecasts, allows the team to build comfort and proficiency with AI's non-deterministic nature without jeopardizing patient safety.1 This foundational learning is crucial before deploying AI in higher-stakes clinical scenarios.
Page 5 of 33
Successfully implementing AI for low-risk, highimpact administrative tasks delivers tangible benefits quickly, saving time, reducing frustration, and lowering operational costs. These early successes are vital for building momentum and securing buy-in from staff and leadership for future AI initiatives.8
Sample Entry Points:
Several administrative areas are ripe for AI augmentation in the early stages:
▶ Documentation
Pharmacists spend considerable time documenting clinical encounters, patient education, and communications. AI, particularly leveraging natural language processing and generative capabilities, can significantly streamline this. Examples include drafting patient education materials, generating summaries of complex medical information, or drafting responses to common patient inquiries.
MedMe Example: MedMe's AI Clinical Assistant utilizes AI scribe functionality during patient consultations such as medication reviews, minor ailment prescribing, and chronic disease management.It listens to the conversation and pre-populates structured clinical documentation, such as SOAP notes and medication reviews, identifies follow-up tasks, creates patient-friendly summaries, and even drafts notification letters to physicians.This directly tackles a major administrative bottleneck associated with delivering clinical services, freeing the pharmacist to focus more on the patient during the encounter.
MedMe Example: MedMe's AI Clinical Assistant utilizes AI scribe functionality during patient consultations such as medication reviews, minor ailment prescribing, and chronic disease management.It listens to the conversation and pre-populates structured clinical documentation, such as SOAP notes and medication reviews, identifies follow-up tasks, creates patient-friendly summaries, and even drafts notification letters to physicians.This directly tackles a major administrative bottleneck associated with delivering clinical services, freeing the pharmacist to focus more on the patient during the encounter.
▶ Inventory Forecasting/Management
Optimizing inventory is a constant challenge. AI can analyze historical dispensing data, seasonality, and local health trends to predict demand for specific medications more accurately than traditional methods.4 This helps optimize stock levels, reduce waste from expired medications, minimize carrying costs, and prevent stockouts of critical drugs.4 Key metrics such as inventory turnover rate, days/weeks on hand, stock-to-sales ratio, and forecast accuracy can be tracked and improved using AIdriven insights.12
▶ Automation of Data Entry/Repetitive Tasks
Many routine pharmacy tasks are rule-based and highvolume, making them ideal candidates for AI-powered automation.4 Examples include:
Communication Handling: AI-powered phone systems can manage prescription refill requests, conduct routine information gathering, and answer frequently asked questions about hours or services, thus reducing the pharmacy’s call burden.4 AI can also assist in sorting incoming electronic faxes or reading invoices.6
Prior Authorizations: AI tools can assist in initiating and tracking prior authorization requests, a notoriously timeconsuming process.5
Data Analysis: AI can analyze operational data to identify workflow bottlenecks or opportunities for efficiency improvements.5
Communication Handling: AI-powered phone systems can manage prescription refill requests, conduct routine information gathering, and answer frequently asked questions about hours or services, thus reducing the pharmacy’s call burden.4 AI can also assist in sorting incoming electronic faxes or reading invoices.6
Prior Authorizations: AI tools can assist in initiating and tracking prior authorization requests, a notoriously timeconsuming process.5
Data Analysis: AI can analyze operational data to identify workflow bottlenecks or opportunities for efficiency improvements.5
Page 6 of 33
Matching Use Cases to Your Pharmacy’s Maturity Level
Not all pharmacies possess the same level of readiness for AI adoption. Factors such as , staff technical literacy, existing technologies and processes, and budget can impact the pharmacy’s AI maturity.14 Aligning the initial AI use case with the pharmacy's current maturity level is critical to avoid overreach and ensure success. Attempting a complex, highly integrated project in a low-maturity environment often leads to failure and can demotivate staff from future AI efforts.The AI maturity journey in pharmacy can be understood across three progressive stages: Awareness/Exploration, Adoption, and Optimization/Transformation.
Table 1: AI Maturity Stages
▶ Awareness/Exploration Stage
Pharmacies at this stage are just beginning to learn about AI. They likely have limited data infrastructure, minimal in-house technical expertise, and a low tolerance for complex implementations. The goal here is to build comfort and familiarity with AI technologies without requiring significant operational changes.14
Simple tools that demonstrate clear value such as MedMe's Clinical Assistant, are particularly valuable for independents expanding clinical services, as they directly reduce the documentation burden without requiring complex integration. MedMe’s ROI for pharmacies can be easily calculated here
These tools should deliver quick, visible value with minimal disruption, making them ideal for pharmacies with limited resources and high operational sensitivit
▶ Adoption Stage
Pharmacies at this stage have experience with digital tools, some IT or vendor support, and are open to deeper workflow integration. They are ready to move from experimenting with AI to embedding it within select operations.
Page 7 of 33
At this stage, it’s important to define clear ROI metrics and establish initial AI governance processes.14
▶ Optimization/Transformation Stage
Pharmacies here have robust technical and governance infrastructures and experience with implementing complex systems. They are capable of leveraging AI at scale and can aim for enterpriselevel efficiency and transformation.14
Initiatives at this stage require significant crossfunctional collaboration and mature change management processes.
Initiatives at this stage require significant crossfunctional collaboration and mature change management processes.
Step 2: Assemble Your Implementation Champion(s) & Assess Your Readiness:
Once a suitable low-risk, high-value use case is identified, the next critical step involves assembling the right team and conducting a realistic assessment of the organization's readiness for AI implementation. AI projects are rarely successful when driven solely by the IT department or a single enthusiast. They require collaboration, diverse expertise, and organizational preparedness across people, processes, and technology. This phase is fundamentally about laying the groundwork for effective change management, recognizing that AI adoption transforms not just tools, but ways of working.9
Mapping Roles and Responsibilities(Scaled by Org Size)
The composition of the AI implementation team will vary significantly based on the size and complexity of the pharmacy organization. However, certain core functions are necessary regardless of scale.
▶ Core Needs (All Sizes)
The AI Champion/Project Lead: Every successful AI initiative needs a dedicated champion – someone with enthusiasm for the technology's potential, credibility within the organization, and the empowerment to drive the project forward.19 This individual acts as the bridge between the clinical or operational need and the technical solution, coordinating efforts and maintaining momentum.
If you're reading this white paper and finding yourself excited by the possibilities, there's a good chance you might be exactly the kind of leader this role calls for.
If you're reading this white paper and finding yourself excited by the possibilities, there's a good chance you might be exactly the kind of leader this role calls for.
▶ Independent Pharmacy
In smaller settings, the owner or manager often naturally assumes the role of AI champion, decisionmaker, and lead user, juggling multiple responsibilities. They will likely rely heavily on the chosen AI vendor for technical support and guidance.
Page 8 of 33
Crucially, key staff members, such as the lead pharmacy technician or the most experienced pharmacist, must be involved early in the selection process and provide ongoing feedback during implementation. Their buy-in is essential for adoption in a small team.
▶ Mid-Market Pharmacy
(Small chains, Groups, Associations)
These organizations might have a dedicated operations manager, a regional manager, or an IT point person who can serve as the project lead. Clear roles need to be defined for pharmacy managers at the specific sites chosen for piloting the AI tool. Lead technicians or technician supervisors 21 should be involved to represent the technician workflow perspective. A designated contact person for coordinating communication and support with the AI vendor is also advisable.
▶ Enterprise Pharmacy
(Large Chains/Health Systems)
Implementing AI in an enterprise setting demands a formal, cross-functional team structure to navigate the inherent complexities of scale, integration, governance, and regulatory compliance.9
Not sure where your organization stands? Use the Quick-Start Readiness Checklist in Appendix A to assess your organizations preparedness across people, process, and technology.
Enterprise Needs: Governance, Legal, IT Involvement
While the checklist provides a starting point for all, enterprise-level pharmacies face significantly higher hurdles related to governance, legal compliance, and IT integration. Addressing these comprehensively before widespread deployment is not just recommended, it's essential to mitigate substantial risks.9
Governance: Enterprises cannot rely on informal processes. They must establish formal AI governance frameworks.9 This involves:
Governance: Enterprises cannot rely on informal processes. They must establish formal AI governance frameworks.9 This involves:
1. Creating an AI Steering Committee or similar oversight body with cross-functional representation to guide strategy, approve projects, set policies, and monitor performance.9
2. Developing clear, documented policies covering acceptable AI use, ethical guidelines (addressing fairness, bias, transparency), risk assessment procedures, data handling standards, and accountability structures.23
3. Implementing a structured process for vetting AI project proposals, ensuring alignment with strategic goals and assessing feasibility, risk, and ROI.9
4. Enforcing robust data stewardship and data governance principles to ensure the quality, integrity, privacy, and security of data used by AI systems.9
2. Developing clear, documented policies covering acceptable AI use, ethical guidelines (addressing fairness, bias, transparency), risk assessment procedures, data handling standards, and accountability structures.23
3. Implementing a structured process for vetting AI project proposals, ensuring alignment with strategic goals and assessing feasibility, risk, and ROI.9
4. Enforcing robust data stewardship and data governance principles to ensure the quality, integrity, privacy, and security of data used by AI systems.9
Page 9 of 33
Legal & Compliance: The legal and regulatory landscape for AI in healthcare is complex and evolving, requiring dedicated legal scrutiny.
1. Data Privacy Laws: Strict adherence to patient privacy regulations like PHIPA and PIPEDA in Canada 31 , GDPR in the UK, and HIPAA in the US 31 is paramount. This governs how Personal Health Information (PHI) is collected, used, stored, disclosed, and protected by both the pharmacy and its AI vendors. Key aspects include access controls, encryption, data minimization, audit trails, and breach notification protocols.31
2. Cross-Border Data Transfers: Special attention is required when using AI vendors that process data outside the pharmacy's home country (e.g., a Canadian pharmacy using a US-based AI model). Organizations must ensure the vendor provides a comparable level of data protection as required by local laws (like PIPEDA) and understand the legal implications of transferring PHI across borders.31 Contractual safeguards are essential.
MedMe Example: MedMe addresses this specific concern for its Canadian clients by contractually ensuring its US-based AI model partners only process PHI temporarily and do not store it long-term, aligning with PHIPA/PIPEDA requirements regarding data residency and cross-border transfers. This requires careful vendor vetting and specific contractual language.
3. Contractual Diligence: Legal teams must meticulously review vendor contracts, focusing on clauses related to data processing agreements (DPAs), Business Associate Agreements (BAAs under HIPAA 31), liability allocation, intellectual property rights, data security obligations, service level agreements (SLAs), compliance warranties, and exit strategies.24
4. Emerging AI Regulations: Enterprises need processes to monitor and adapt to new AI-specific regulations being enacted globally (e.g., EU AI Act) and domestically (e.g., various US state laws).27 These regulations often impose requirements related to risk classification, transparency, human oversight, and data quality
2. Cross-Border Data Transfers: Special attention is required when using AI vendors that process data outside the pharmacy's home country (e.g., a Canadian pharmacy using a US-based AI model). Organizations must ensure the vendor provides a comparable level of data protection as required by local laws (like PIPEDA) and understand the legal implications of transferring PHI across borders.31 Contractual safeguards are essential.
MedMe Example: MedMe addresses this specific concern for its Canadian clients by contractually ensuring its US-based AI model partners only process PHI temporarily and do not store it long-term, aligning with PHIPA/PIPEDA requirements regarding data residency and cross-border transfers. This requires careful vendor vetting and specific contractual language.
3. Contractual Diligence: Legal teams must meticulously review vendor contracts, focusing on clauses related to data processing agreements (DPAs), Business Associate Agreements (BAAs under HIPAA 31), liability allocation, intellectual property rights, data security obligations, service level agreements (SLAs), compliance warranties, and exit strategies.24
4. Emerging AI Regulations: Enterprises need processes to monitor and adapt to new AI-specific regulations being enacted globally (e.g., EU AI Act) and domestically (e.g., various US state laws).27 These regulations often impose requirements related to risk classification, transparency, human oversight, and data quality
IT Involvement: Deep IT engagement is crucial for secure and effective integration within the enterprise environment.9
1. Infrastructure Readiness: IT must assess and potentially upgrade compute resources, storage capacity, and network bandwidth to handle the demands of AI workloads.9
2. Security Architecture Design: Implementing a multi-layered security strategy is vital. This includes robust firewalls, identity and access management (IAM) systems, next-generation endpoint protection, network segmentation to isolate AI systems, and potentially AI-specific threat detection tools.23
3. Integration Strategy: IT needs to evaluate the technical feasibility of integrating the AI solution with core enterprise systems like the EHR, PMS, data warehouses, or communication platforms, planning API usage or other data exchange mechanisms.
4. Data Management & Quality: IT plays a key role in ensuring data pipelines are efficient, data quality meets the requirements of the AI models, and data handling complies with established governance policies.9
2. Security Architecture Design: Implementing a multi-layered security strategy is vital. This includes robust firewalls, identity and access management (IAM) systems, next-generation endpoint protection, network segmentation to isolate AI systems, and potentially AI-specific threat detection tools.23
3. Integration Strategy: IT needs to evaluate the technical feasibility of integrating the AI solution with core enterprise systems like the EHR, PMS, data warehouses, or communication platforms, planning API usage or other data exchange mechanisms.
4. Data Management & Quality: IT plays a key role in ensuring data pipelines are efficient, data quality meets the requirements of the AI models, and data handling complies with established governance policies.9
Failing to address these enterprise-specific needs for governance, legal review, and IT integration early in the process can lead to significant project delays, budget overruns, compliance failures, security breaches, and ultimately, failed AI implementations.29 Building the right team and assessing readiness thoroughly are foundational steps that prevent downstream problems and set the stage for a successful AI journey.
Page 10 of 33
Step 3: Select the Right Tool or Vendor
With a clear initial use case identified (Step 1) and a foundational understanding of team structure and organizational readiness (Step 2), the focus shifts to selecting the appropriate AI tool or vendor partner. This decision is critical, as the right choice can accelerate success, while the wrong one can lead to wasted resources, frustration, and potential risks. The selection process involves understanding the type of AI needed, evaluating potential vendors, realistically assessing the buildversus-buy trade-off, defining rigorous testing procedures, and being aware of potential red flags.
Deciding What Type of AI Tool You Need:
The specific problem defined in Step 1 dictates the type of AI technology required. Understanding the underlying technology helps assess potential capabilities, inherent risks (e.g., generative AI's potential for "hallucination" or factual inaccuracies ), and the kind of testing and validation required.
Here are key AI technologies relevant to pharmacy and examples of their applications:
Here are key AI technologies relevant to pharmacy and examples of their applications:
Natural Language Processing (NLP) & Generative AI (e.g., LLMs):
Technologies that understand, interpret, and generate human language.
Examples: AI scribes summarizing consultations; drafting patient education materials or communications; chatbots answering patient FAQs; analyzing clinical notes for insights; assisting with literature searches.
Machine Learning (ML) / Predictive Analytics:
Algorithms that learn patterns from historical data to make predictions or classifications.
Examples: Inventory forecasting and management; predicting patient risk for adverse drug events; identifying potential drug interactions; optimizing medication adherence programs; detecting potential drug diversion.
Computer Vision:
Technology that enables AI to interpret and understand information from images.
Examples: AI-powered pill identification tools; potentially reading information from scanned documents or labels (related to Optical Character Recognition-OCR).
Page 11 of 33
Robotic Process Automation (RPA) (often combined with AI):
Technology that automates rulebased, repetitive digital tasks.
Examples: Automating data entry from faxes or invoices; streamlining parts of the prior authorization workflow.
Speech Recognition (Voice AI / Speech-to-Text):
Algorithms that learn patterns from historical data to make predictions or classifications.
Examples: Dictation programs for notes; AI-powered phone systems handling refill requests or providing status updates; voice assistants for medication information; enabling AI scribes to capture conversations.
This breakdown helps match the pharmacy's specific needs with the appropriate AI technology, setting the stage for effective vendor evaluation and implementation planning.
Evaluating Potential AI Vendors
Selecting the right vendor goes far beyond comparing surface-level features or pricing. AI doesn’t work like traditional software; its outputs are probabilistic, often running in the background, making it harder to evaluate based on demos alone. In healthcare, a trustworthy partner demonstrates a combination of industry knowledge, technical competence, transparency, and a commitment to safety and compliance. Key areas to evaluate include:
(✓) Domain Expertise
Does the vendor understand pharmacy workflows, regulatory requirements (HIPAA, PHIPA/PIPEDA, GDPR), and real-world constraints? Prioritize vendors with a proven record of success in comparable healthcare settings.
(✓) Transparency and Control
A critical aspect is the vendor's willingness to explain how their AI works. They should be transparent about their model's capabilities, limitations, and what outputs require human review. Staff must always retain the ability to override AI suggestions or intervene in AI processes if their judgment dictates.
Does the vendor understand pharmacy workflows, regulatory requirements (HIPAA, PHIPA/PIPEDA, GDPR), and real-world constraints? Prioritize vendors with a proven record of success in comparable healthcare settings.
(✓) Transparency and Control
A critical aspect is the vendor's willingness to explain how their AI works. They should be transparent about their model's capabilities, limitations, and what outputs require human review. Staff must always retain the ability to override AI suggestions or intervene in AI processes if their judgment dictates.
Page 12 of 33
(✓) Testing
Testing must go beyond model performance claims to include internal validation against real-world use cases, with attention to output accuracy, consistency, and safety. Ask whether the vendor conducts in-house testing and how they measure performance in production settings.
MedMe Example: Sharing details about the rigorous testing process for the AI Clinical Assistant, including the diverse range of scenarios tested (different accents, complexities, etc.), helps build trust by demonstrating due diligence and a commitment to accuracy and fairness
(✓) WorkFlow Integration
Is the AI purpose-built to fit into the platform’s workflows, or added on as an isolated feature? Tools that don’t integrate with the broader product ecosystem often create disjointed experiences, both for staff and patients. Look for solutions where AI complements core workflows and connects meaningfully with the surrounding feature set.
(✓) Built to Evolve
AI is changing fast. Choose a vendor whose infrastructure, models, and roadmap are built to keep up, so you’re not stuck with yesterday’s tech in a year.
Testing must go beyond model performance claims to include internal validation against real-world use cases, with attention to output accuracy, consistency, and safety. Ask whether the vendor conducts in-house testing and how they measure performance in production settings.
MedMe Example: Sharing details about the rigorous testing process for the AI Clinical Assistant, including the diverse range of scenarios tested (different accents, complexities, etc.), helps build trust by demonstrating due diligence and a commitment to accuracy and fairness
(✓) WorkFlow Integration
Is the AI purpose-built to fit into the platform’s workflows, or added on as an isolated feature? Tools that don’t integrate with the broader product ecosystem often create disjointed experiences, both for staff and patients. Look for solutions where AI complements core workflows and connects meaningfully with the surrounding feature set.
(✓) Built to Evolve
AI is changing fast. Choose a vendor whose infrastructure, models, and roadmap are built to keep up, so you’re not stuck with yesterday’s tech in a year.
MedMe Example: MedMe’s AI assistants use LangChain to easily swap in newer AI models and partnering only with vendors that support ongoing access to the latest model versions.
(✓) Security and Compliance
The vendor must demonstrate robust security measures and clear processes for ensuring compliance with healthcare data privacy laws. This includes data handling practices, encryption, access controls, and willingness to enter into necessary agreements (like BAAs under HIPAA ).
(✓) Support, Training, and Partnership
Evaluate the quality and responsiveness of their customer support and the effectiveness of their training programs. A good vendor acts as a partner, invested in your success and open to feedback.
(✓) Auditability and Version Control
The vendor should support auditability of AI outputs and maintain clear version control with transparent change logs for their software and models.
(✓) Fail-Safes and Backup Protocols
Technology, including AI, can fail or produce unexpected results. A strong vendor should be able to clearly articulate how their solution accounts for potential failure modes (e.g., inaccurate system outages, failure to trigger correctly).
(✓) Security and Compliance
The vendor must demonstrate robust security measures and clear processes for ensuring compliance with healthcare data privacy laws. This includes data handling practices, encryption, access controls, and willingness to enter into necessary agreements (like BAAs under HIPAA ).
(✓) Support, Training, and Partnership
Evaluate the quality and responsiveness of their customer support and the effectiveness of their training programs. A good vendor acts as a partner, invested in your success and open to feedback.
(✓) Auditability and Version Control
The vendor should support auditability of AI outputs and maintain clear version control with transparent change logs for their software and models.
(✓) Fail-Safes and Backup Protocols
Technology, including AI, can fail or produce unexpected results. A strong vendor should be able to clearly articulate how their solution accounts for potential failure modes (e.g., inaccurate system outages, failure to trigger correctly).
Page 13 of 33
For example, ask whether they’ve built in manual override capabilities if the AI inventory system is offline, preventing the Completion -> Next Step of ordering? Having documented manual overrides and backup procedures ensures operational continuity and safety when the AI is unavailable or unreliable.
(✓) Feedback Loops for Continuous Improvement
Vendors should demonstrate how they continuously learn from real-world use. Effective solutions include mechanisms for staff to provide inthe-moment feedback, especially around review points or error escalation. Ask whether the tool includes built-in feedback features (e.g., postinteraction surveys, thumbs up/down), and how that data informs model or workflow improvements.
MedMe Example: MedMe’s Clinical Assistant collects 1–5 rating feedback post-consultation to support iterative refinement of its AI-generated documentation.
(✓) Feedback Loops for Continuous Improvement
Vendors should demonstrate how they continuously learn from real-world use. Effective solutions include mechanisms for staff to provide inthe-moment feedback, especially around review points or error escalation. Ask whether the tool includes built-in feedback features (e.g., postinteraction surveys, thumbs up/down), and how that data informs model or workflow improvements.
MedMe Example: MedMe’s Clinical Assistant collects 1–5 rating feedback post-consultation to support iterative refinement of its AI-generated documentation.
In contrast, the following traits should be watched for and avoided when assessing potential vendors.
(x) Vague Claims / "AI Washing"
Vendors using buzzwords like "AI-powered" but cannot articulate precisely what the AI does, how it works, or what specific value it provides. This suggests the AI component may be superficial or non-existent.
(x) Ignoring Compliance & Privacy
The vendor cannot clearly explain how they ensure compliance with HIPAA/PHIPA/PIPEDA/GDPR, protect patient data, handle data security, or manage data usage rights. This poses significant legal and reputational risks.
(x) Unrealistic Promises
Be cautious of vendors who make claims that seem too good to be true, guarantee perfection, or fail to acknowledge the inherent limitations and potential biases of AI. Trustworthy vendors are realistic about what their AI can and cannot do
(x) Poor Support or Documentation
Difficulty getting clear answers, unresponsive support, or inadequate user documentation during the evaluation phase is a strong indicator of future problems.
Vendors using buzzwords like "AI-powered" but cannot articulate precisely what the AI does, how it works, or what specific value it provides. This suggests the AI component may be superficial or non-existent.
(x) Ignoring Compliance & Privacy
The vendor cannot clearly explain how they ensure compliance with HIPAA/PHIPA/PIPEDA/GDPR, protect patient data, handle data security, or manage data usage rights. This poses significant legal and reputational risks.
(x) Unrealistic Promises
Be cautious of vendors who make claims that seem too good to be true, guarantee perfection, or fail to acknowledge the inherent limitations and potential biases of AI. Trustworthy vendors are realistic about what their AI can and cannot do
(x) Poor Support or Documentation
Difficulty getting clear answers, unresponsive support, or inadequate user documentation during the evaluation phase is a strong indicator of future problems.
Page 14 of 33
(x) Unclear Scalability
The vendor cannot provide a convincing explanation or evidence of how their solution will scale to meet your anticipated volume or user base.
(x) Hidden Costs
Lack of clarity regarding the total cost of ownership, including implementation fees, integration costs, data preparation efforts, training, and ongoing maintenance or support fees.
The vendor cannot provide a convincing explanation or evidence of how their solution will scale to meet your anticipated volume or user base.
(x) Hidden Costs
Lack of clarity regarding the total cost of ownership, including implementation fees, integration costs, data preparation efforts, training, and ongoing maintenance or support fees.
A comprehensive Vendor Evaluation Scorecard & Checklist is provided within Appendix B to guide a structured assessment across these critical domains.
Build vs. Buy: A Realistic Assessment
Pharmacy organizations face a fundamental choice: develop solutions internally on top of existing AI models and platforms ("Build") or purchase solutions from external vendors ("Buy"). A hybrid approach also exists.
▶ Buy (Vendor Solution):
Generally the most practical option for the vast majority of independent and mid-market pharmacies due to team size and resources needed to build in-house.. Also suitable for enterprises starting with non-core AI applications, needing rapid deployment, or lacking deep internal AI expertise.
▶ Build (In-House Development)
Primarily an option for large enterprises or pharmacy groups with established digital infrastructure, product teams, and a desire to retain more control. Most feasible when using marketavailable models (e.g., OpenAI, Google, or industryspecific LLMs) to build tailored solutions rather than training models from scratch.
▶ Hybrid Approach
Involves the vendor building custom applications or integrations on top of an existing AI solution to tailor it to your specific needs. This typically involves paying a custom development fee and can offer a balance between speed/leveraging existing tech and achieving some customization. The hybrid approach still requires internal product ownership and careful management of the vendor relationship.
Page 15 of 33
Table 2: Build vs. Buy AI Solution: Key Considerations
The decision hinges on a realistic assessment of the organization's strategic goals, urgency, available resources (talent, time, budget), risk appetite, and long-term vision for AI. For most pharmacies embarking on their AI journey, particularly with administrative tasks, leveraging vendor solutions is the more pragmatic path. However, understanding the trade-offs is crucial.
How to Test AI Solutions
Regardless of build or buy, rigorous testing is nonnegotiable before deploying AI in a pharmacy setting, and this responsibility falls on the shoulders of the project champion and clinical leads. The goal is to verify performance, identify limitations, and ensure safety.
1. Define Test Scenarios and Expected Behavior
Based on the intended use case, create specific, realistic test scenarios. Some questions you could ask are:
1. Define Test Scenarios and Expected Behavior
Based on the intended use case, create specific, realistic test scenarios. Some questions you could ask are:
• What are the common inputs?
• What are challenging or edge-case inputs?
• What is the desired output or range of acceptable outputs?
• What are challenging or edge-case inputs?
• What is the desired output or range of acceptable outputs?
For example, for inventory forecasting, test scenarios might involve predicting demand during holidays, flu season, or after promotional events.
MedMe Example: For an AI scribe such as MedMe’s Clinical Assistant, this means testing with various consultation types, complexities, speaker accents, background noise levels, and specific terminology (drug names, clinical terms).
MedMe Example: For an AI scribe such as MedMe’s Clinical Assistant, this means testing with various consultation types, complexities, speaker accents, background noise levels, and specific terminology (drug names, clinical terms).
2. Establish Performance Metrics
Define objective metrics to evaluate the AI during testing. These may differ slightly from the pilot KPIs focused on business impact.
Define objective metrics to evaluate the AI during testing. These may differ slightly from the pilot KPIs focused on business impact.
Page 16 of 33
Think of testing AI like evaluating a student after learning a new topic, the AI needs to “pass the exam.”
Start by creating a set of test cases that reflect realworld scenarios the AI is expected to handle. Then, develop a manual “answer key” representing the ideal outputs for each case.
The AI’s responses are compared against these gold-standard answers to assess how well it performs. Focus the evaluation on the areas below:
Start by creating a set of test cases that reflect realworld scenarios the AI is expected to handle. Then, develop a manual “answer key” representing the ideal outputs for each case.
The AI’s responses are compared against these gold-standard answers to assess how well it performs. Focus the evaluation on the areas below:
Accuracy: Is the information correct and clinically sound? For AI scribes, this means checking whether clinical notes reflect what was actually said, without adding or omitting important details.
Thoroughness: Does the response include all the relevant and necessary information?
Bias: Run test cases across diverse patient scenarios (e.g., age, gender, ethnicity, language) to ensure the AI performs equitably and doesn’t favor or overlook any groups.
Response Time: Is the AI fast enough to be practical in a pharmacy workflow?
Thoroughness: Does the response include all the relevant and necessary information?
Bias: Run test cases across diverse patient scenarios (e.g., age, gender, ethnicity, language) to ensure the AI performs equitably and doesn’t favor or overlook any groups.
Response Time: Is the AI fast enough to be practical in a pharmacy workflow?
Step 4: Design for Workflow Integration
Selecting the right AI tool (Step 3) is a significant milestone, but its true value is only realized when it becomes a seamless part of your pharmacy's daily operations. Simply layering AI onto existing processes often leads to friction, frustration, and ultimately, abandonment of the technology. Effective AI implementation hinges on thoughtful workflow design: integrating the tool so intuitively that it feels like a natural extension of your team's capabilities, rather than an obstacle.
Weaving AI into the Fabric of Daily Routines
Before introducing any AI tool, it's essential to deeply understand the workflow it aims to enhance. Imagine mapping out the current process – perhaps visually, like a flowchart – detailing every step, who performs it, the time involved, and any existing pain points or bottlenecks. This "as-is" map becomes the foundation for designing the "to-be" workflow incorporating AI.
The goal is to pinpoint exactly where the AI fits and how it interacts with the human user at key moments:
The goal is to pinpoint exactly where the AI fits and how it interacts with the human user at key moments:
Page 17 of 33
1. Trigger: When is the AI activated? Is it automatic based on an event (e.g., starting a patient consultation), or does the user manually initiate it? Defining the Trigger ensures the AI engages at the right moment without being intrusive.
2. Action: What does the user need to do to trigger the AI? What is required from the human as input to the AI and workflow?
3. Review Point: Where in the process must a human intervene to check, validate, edit, or approve the AI's output? This is a critical safety and quality control step.
4. Completion: Is it clear to users how to complete the workflow? Once the AI task is completed and reviewed, what happens next? How does the output flow into the subsequent step of the broader pharmacy workflow?
2. Action: What does the user need to do to trigger the AI? What is required from the human as input to the AI and workflow?
3. Review Point: Where in the process must a human intervene to check, validate, edit, or approve the AI's output? This is a critical safety and quality control step.
4. Completion: Is it clear to users how to complete the workflow? Once the AI task is completed and reviewed, what happens next? How does the output flow into the subsequent step of the broader pharmacy workflow?
Consider an AI tool that requires pharmacists to log into a separate system, copy patient details, paste them into the AI, wait for output, and then copy it back into the Pharmacy Management System (PMS). This workflow has multiple points of friction and poorly defined transitions. Contrast this with a well-integrated solution.
Minimizing context switching and unnecessary clicks is paramount. Ideally, AI functionalities should surface within the team's primary software environment (PMS/EHR) whenever possible. Remember, the initial design during the pilot phase (Step 6) is a starting point; user feedback will be crucial for refining the integration and ensuring it truly enhances, rather than hinders, the workflow.
Figure 2: Visualizing the Workflow of MedMe’s Clinical AI Assistant
Page 18 of 33
Keeping People In Control: Oversight, FailSafes, and Escalation
In the high-stakes environment of pharmacy, AI should serve as a capable assistant, not the ultimate authority. Implementing human oversight is fundamental to ensuring safety, accountability, and user trust, whether the AI is supporting staff internally or interacting with patients. This involves designing workflows with clear points for human intervention (review points) and proactively planning for potential AI failures or unexpected outcomes through fail-safes and escalation pathways.
▶ Human Judgment Remains Central
The core principle is that AI assists, but final clinical judgment and responsibility rest with qualified humans. For clinician-facing tools like AI scribes or inventory predictors, this typically means a mandatory review point where a pharmacist or manager validates, edits, and approves the AI's output before it's finalized or acted upon. For patient-facing AI (e.g., chatbots), oversight might involve post-interaction audits or, designing the AI to recognize its limitations and escalate complex or sensitive interactions to human staff in real-time. That said, not every use case needs a formal review step. For simple tasks like answering store hours or processing a refill request, it’s usually fine for the AI to act without immediate human review, as long as there are audit processes in place. The key is matching the level of review to the level of risk.
▶ Structured Escalation for Issue Resolution:
A formal escalation pathway is needed to handle AIrelated issues, with processes that scale based on severity of erorrs. While minor usability quirks which can be expected in probabilistic systems may not warrant formal escalation, more significant issues such as repeated errors flagged during review or critical failures identified through monitoring should trigger defined responses. This pathway should define:
Reporting: How users report issues (e.g., feedback button, help desk ticket, direct supervisor)?
Triage & Investigation: Who is responsible for assessing the issue's severity and investigating its root cause (e.g., AI champion, IT support, vendor, governance committee)?
Resolution & Response: Timelines and procedures for addressing the issue based on risk level, ranging from logging minor bugs for future updates to immediate intervention for critical safety concerns.
Triage & Investigation: Who is responsible for assessing the issue's severity and investigating its root cause (e.g., AI champion, IT support, vendor, governance committee)?
Resolution & Response: Timelines and procedures for addressing the issue based on risk level, ranging from logging minor bugs for future updates to immediate intervention for critical safety concerns.
Page 19 of 33
Enterprise Considerations: Scaling Workflow Integration
Integrating AI workflows within large enterprises introduces unique challenges. Ensuring that AI tools function consistently and effectively across diverse settings requires careful planning that extends beyond the technical aspects.
▶ Collaborative by Design
Designing workflows for enterprise scale cannot happen in isolation. It demands active participation from across the organization – IT , Legal and Compliance, Operations, Finance, human resources, and, crucially, diverse end-users representing different sites and roles. Holding dedicated crossdepartmental design workshops early ensures that triggers, actions, review points, and completion points are feasible, compliant, and user-friendly system-wide. This collaborative approach prevents silos and fosters shared ownership.
▶ Workflow Meets Change Management
The design of the AI-enabled workflow, including how staff are trained on its use and oversight responsibilities, must be tightly interwoven with the overall change management strategy.
• What scalable training methods (Step 5) will be employed?
• How will user support be structured during the transition?
• How will the changes be communicated consistently across potentially thousands of staff members?
• How will user support be structured during the transition?
• How will the changes be communicated consistently across potentially thousands of staff members?
Addressing these questions during the design phase leads to smoother adoption.
▶ Navigating Change Fatigue
Enterprise staff often face numerous ongoing initiatives. AI implementation must acknowledge and plan for potential change fatigue. Strategies include phased rollouts rather than "big bang" launches across the entire organization; consistent communication emphasizing the "why" and the benefits for staff and patients; providing exceptionally strong training and accessible support; celebrating early wins from pilot sites to build enthusiasm; and strategically prioritizing and sequencing AI initiatives to avoid overwhelming the organization.
Ultimately, the success of AI in pharmacy depends less on the algorithm's brilliance and more on how seamlessly it integrates with human processes, marked by clear triggers, simple actions, effective review points, and smooth transitions at completion.
Ultimately, the success of AI in pharmacy depends less on the algorithm's brilliance and more on how seamlessly it integrates with human processes, marked by clear triggers, simple actions, effective review points, and smooth transitions at completion.
Page 20 of 33
Step 5: Train and Prepare Your Team
Introducing AI into the pharmacy workflow requires preparing the people who will interact with it to build user confidence, ensure safe and appropriate use, manage expectations, and drive successful adoption. The focus should be less on the intricacies of AI algorithms and more on practical application within the pharmacy context, addressing user concerns head-on, and reinforcing the role of AI as a supportive assistant.6
Building Confidence in AI-Enabled Workflows
User confidence is paramount for AI adoption. Staff who are hesitant or distrustful of an AI tool are unlikely to use it effectively, if at all. Several strategies can help build this crucial confidence:
▶ Address Fears and Concerns Directly
Openly acknowledge that staff may have concerns about AI, such as fears of job replacement, deskilling, the reliability of the technology, or data privacy.6 Create forums or leverage existing staff gatherings (town halls) for discussion where these concerns can be voiced and addressed transparently.
▶ Promote Transparency
Explain clearly what the specific AI tool does, provide a high-level overview of how it works, discuss its known limitations, and share information about how it was tested and validated.43
▶ Highlight Tangible BenefitsFocus communications and training on how the AI tool directly benefits the pharmacy staff. Emphasize time savings on administrative tasks, reduction in tedious work, potential for improved accuracy, alleviation of workload pressures, and the opportunity to spend more time on rewarding patient-facing activities.5 Use data and testimonials from pilot users (Step 6) to provide concrete evidence of these benefits.8
▶ Emphasize the Human-in-the-Loop (HITL) ConceptContinuously reinforce that the AI is a tool designed to assist, not replace, their professional judgment and expertise.6 Stress that humans remain in control, responsible for reviewing, validating, and making the final decisions based on AI-provided information.6
Page 21 of 33
▶ Provide Ample Hands-on Practice
Confidence comes from experience. Allow staff sufficient time to practice using the AI tool in a safe, simulated environment before they are expected to use it in live situations.59 This allows them to become comfortable with the interface, understand its outputs, and learn how to handle different scenarios without pressure.
MedMe Example: MedMe incorporates a simulated consultation into its onboarding process. This allows pharmacists to experience the AI scribe capturing information and generating documentation in a controlled setting, ask questions, and build familiarity and confidence before using the tool during actual patient interactions.
Building Future-Ready Skills
AI will shift the nature of pharmacy work. Proactively developing new competencies ensures staff can confidently collaborate with AI while continuing to excel in human-centric areas of care.
Identify Emerging Skill Needs: As AI automates routine tasks⁷, pharmacy staff will increasingly need to interpret AI outputs, validate AI suggestions, oversee AI performance, understand ethical implications, and incorporate AI insights into clinical decision-making.⁵
Invest in Continuous Learning: Offer ongoing professional development to build these skills. This may include internal training programs, partnerships with pharmacy schools to embed AI education into curricula⁷², resources from organizations like ASHP⁷², and AI-powered simulation tools.⁵⁹
Emphasize Human Strengths: Continue cultivating qualities AI cannot replicate—critical thinking, empathy, complex judgment, nuanced communication, and problem-solving.⁴ These remain core to the pharmacist’s role.⁵
Evolve Career Paths: Recognize AI fluency as a valuable skill. Integrate it into career advancement criteria to encourage upskilling and reflect the changing nature of pharmacy roles.²¹
Emphasize Human Strengths: Continue cultivating qualities AI cannot replicate—critical thinking, empathy, complex judgment, nuanced communication, and problem-solving.⁴ These remain core to the pharmacist’s role.⁵
Evolve Career Paths: Recognize AI fluency as a valuable skill. Integrate it into career advancement criteria to encourage upskilling and reflect the changing nature of pharmacy roles.²¹
Simple Training for Pharmacists and Techs
Training content should be practical, relevant, and easily digestible.
▶ Workflow-Centric Approach
Training should focus on the practicalities of using the AI tool within the established pharmacy workflow.
Key training objectives should include:
Key training objectives should include:
Page 22 of 33
• How to initiate and interact with the AI tool.
• How to interpret the AI's outputs correctly
• How to identify potential errors or inconsistencies in AI suggestions
• How to perform necessary validation, correction, or approval steps
• How to utilize built-in feedback mechanisms effectively
• How to follow established fail-safe and escalation protocols when issues arise
• How to interpret the AI's outputs correctly
• How to identify potential errors or inconsistencies in AI suggestions
• How to perform necessary validation, correction, or approval steps
• How to utilize built-in feedback mechanisms effectively
• How to follow established fail-safe and escalation protocols when issues arise
While developing these new skills may feel daunting at first, they quickly become second nature with regular use, and the return on investment is clear in the form of improved day-today efficiency and workflow ease.
▶ Concise and Clear Language
Avoid overly technical AI jargon. Use simple, direct language that focuses on the actions users need to take and the information they need to know to use the tool safely and effectively.64
▶ Diverse Training Methods
Employ a blended learning approach. Combine hands-on practice sessions with quick reference guides (e.g., checklists, laminated cards), short instructional videos, readily accessible SOPs,64 and opportunities for peer-to-peer learning and coaching.59
▶ Competency Validation (If Needed)
Depending on the criticality of the task the AI supports, consider implementing a competency check or validation process (similar to tech-checktech validation)2 to ensure users demonstrate proficiency before using the tool independently in live workflows
Enterprise Path: Role-Based Training, Internal Champions
Training and preparation in large enterprise settings require additional structure and scalability.
Role-Based Training Curricula: Develop distinct training modules tailored to the specific needs and responsibilities of different user groups.59 For example, end-user pharmacists and technicians need practical workflow training, IT support staff need technical troubleshooting knowledge, compliance officers need to understand auditing features, and managers need to know how to interpret performance reports.
Formalized Internal Champions Program: Establish a structured program to identify, train, and empower internal AI champions within various departments, sites, or regions.19 These champions serve as enthusiastic early adopters, local subject matter experts, peer coaches, and conduits for feedback.20 Providing them with advanced training ("train-thetrainer") enables them to deliver localized support and training, which is often more effective and scalable than relying solely on central resources.20 Initiatives like innovation challenges can help identify potential champions.66
Formalized Internal Champions Program: Establish a structured program to identify, train, and empower internal AI champions within various departments, sites, or regions.19 These champions serve as enthusiastic early adopters, local subject matter experts, peer coaches, and conduits for feedback.20 Providing them with advanced training ("train-thetrainer") enables them to deliver localized support and training, which is often more effective and scalable than relying solely on central resources.20 Initiatives like innovation challenges can help identify potential champions.66
Page 23 of 33
Effective training and preparation are investments that pay dividends in smoother adoption, more confident users, safer utilization, and ultimately, the realization of AI's potential benefits.
It requires addressing the human element of change proactively and thoughtfully, ensuring that staff feel supported and empowered as they learn to work alongside their new digital assistants. Training should be viewed not as a single event, but as an ongoing process of skill development and adaptation in the evolving landscape of AI in pharmacy getting easier over time.
It requires addressing the human element of change proactively and thoughtfully, ensuring that staff feel supported and empowered as they learn to work alongside their new digital assistants. Training should be viewed not as a single event, but as an ongoing process of skill development and adaptation in the evolving landscape of AI in pharmacy getting easier over time.
Step 6: Launch and Iterate on the Pilot
Following thorough preparation and training, the pilot launch represents the first real-world test of the AI solution within the pharmacy environment. This is an important phase to validate the technology, gather crucial data, refine workflows, incorporate user feedback, and build the case for a broader rollout. A well-executed pilot minimizes risks associated with large-scale deployment and provides invaluable learning opportunities.8
The mantra for this stage should be:start small, measure diligently, listen intently, and iterate rapidly.
The mantra for this stage should be:start small, measure diligently, listen intently, and iterate rapidly.
How to Plan a Pilot: Start Small and Focused
The key to a successful pilot is careful planning and a tightly defined scope. Trying to do too much too soon increases complexity and the likelihood of failure.
▶ Define Scope Clearly
Resist the temptation to pilot everything at once. Select the specific, manageable use case identified in Step 1. Choose a limited, representative group of users, perhaps a single pharmacy location, a specific clinical service, or a small group of enthusiastic early adopters.8 Define a clear start and end date for the pilot period (e.g., 4-12 weeks, depending on complexity). Keeping the initial scope contained makes the pilot easier to manage, monitor, and evaluate.
▶ Set Clear Objectives & Success Metrics Scope ClearlyRevisit the goals established earlier (Step 2) and translate them into specific, measurable, achievable, relevant, and time-bound (SMART) objectives for the pilot.8 What quantifiable outcomes will define success? Examples: "Reduce documentation time per patient encounter by 20% within 8 weeks," or "Achieve a user satisfaction score of 4 out of 5".8 Clarity on success metrics is essential for objective evaluation.
Page 24 of 33
▶ Involve Pilot Users Early
Engage the selected pilot users during the planning phase. Solicit their input on workflow integration, potential challenges, and success criteria.
Early involvement fosters a sense of ownership and increases their commitment to the pilot's success.17
▶ Confirm ReadinessEarly involvement fosters a sense of ownership and increases their commitment to the pilot's success.17
Before launching, double-check that all prerequisites are in place: the team is assembled and understands their roles (Step 2), the technology is configured and tested (Step 3), the initial workflow design is implemented (Step 4), and the pilot users have received adequate training (Step 5).
▶ Plan for Data CollectionDetermine how you will collect the data needed to measure your KPIs and gather user feedback throughout the pilot.17
This might involve time tracking, system logs, audits of AI outputs, inventory reports, surveys, interviews, or feedback forms. Ensure data collection methods are feasible and minimally disruptive.
This might involve time tracking, system logs, audits of AI outputs, inventory reports, surveys, interviews, or feedback forms. Ensure data collection methods are feasible and minimally disruptive.
MedMe Example: MedMe strategically initiated its AI Clinical Assistant beta program with independent pharmacies for a period of 6 months. This represents a "start small, focused" approach, targeting a user segment known for agility and direct feedback loops before scaling to more complex enterprise environments.
What to Measure: Time Saved, Error Reduction, Satisfaction
Pilot measurement should encompass both quantitative and qualitative aspects to provide a holistic view of the AI tool's impact and usability.
Quantitative Metrics provide objective data on performance and efficiency.
Efficiency Gained: Measure time saved on the specific task the AI addresses (e.g., minutes per documentation using an AI scribe vs. manual methods8, time to process a prior authorization10). Track reductions in manual steps or increases in task throughput.
Accuracy/Quality Improvement: Evaluate the accuracy of AI outputs compared to a baseline or gold standard. This could involve measuring documentation error rates (e.g., using metrics like Faithfulness and Thoroughness for scribes51), improvements in inventory forecast accuracy (e.g. reduced forecast error
Accuracy/Quality Improvement: Evaluate the accuracy of AI outputs compared to a baseline or gold standard. This could involve measuring documentation error rates (e.g., using metrics like Faithfulness and Thoroughness for scribes51), improvements in inventory forecast accuracy (e.g. reduced forecast error
Page 25 of 33
percentage12) ), reduction in specific types of process errors (e.g., fewer denials due to lack of prior authorization10), or improved compliance rates.
Cost Reduction/Return on Investment (ROI): Quantify financial benefits where possible. This could include cost savings from reduced inventory waste, the monetary value of staff time saved8, reduced costs associated with correcting errors, or revenue impact from reduced payer denials.10 Compare benefits against the pilot's costs.8
Cost Reduction/Return on Investment (ROI): Quantify financial benefits where possible. This could include cost savings from reduced inventory waste, the monetary value of staff time saved8, reduced costs associated with correcting errors, or revenue impact from reduced payer denials.10 Compare benefits against the pilot's costs.8
Qualitative Metrics capture user perceptions and experiences, which are crucial for adoption.
User Satisfaction: Use surveys (e.g., Likert scale questions), interviews, or focus groups to gauge user satisfaction with the AI tool.8 Assess perceived ease of use, usefulness, confidence in the technology, and its impact on job satisfaction or stress levels.
Workflow Integration Fit: Gather feedback on how well the AI tool integrates into daily routines. Are there points of friction? Does it feel natural or disruptive? Does it require awkward workarounds?
Perceived Patient Impact (Indirect): While the pilot might focus on administrative tasks, ask users if they perceive that the time saved or efficiency gained has allowed them to improve patient interactions, provide better counseling, or enhance service delivery. Initial assessments may be subjective but provide valuable context.
Workflow Integration Fit: Gather feedback on how well the AI tool integrates into daily routines. Are there points of friction? Does it feel natural or disruptive? Does it require awkward workarounds?
Perceived Patient Impact (Indirect): While the pilot might focus on administrative tasks, ask users if they perceive that the time saved or efficiency gained has allowed them to improve patient interactions, provide better counseling, or enhance service delivery. Initial assessments may be subjective but provide valuable context.
Relying solely on quantitative data can mask significant usability issues, while relying only on qualitative feedback might overlook a lack of tangible performance improvement. A balanced approach using both types of metrics provides the most comprehensive understanding of the pilot's success.63
Incorporating Team Feedback
The pilot phase is a prime opportunity to learn directly from users interacting with the AI in a realworld context. Actively soliciting and responding to their feedback is essential for refinement and building buy-in.
▶ Establish Multiple Feedback Channels
Don't rely on a single method. Use regular check-in meetings (formal or informal), structured surveys, suggestion boxes, dedicated email aliases, or, ideally, feedback mechanisms built directly into the AI tool itself.
MedMe Example: MedMe utilizes both recurring check-in calls with beta testers and in-app feedback tools, ensuring multiple avenues for users to share their experiences and suggestions.
▶ Listen Actively and Be ResponsiveTreat user feedback as valuable data. Acknowledge receipt of feedback promptly, demonstrate that concerns are being heard, and communicate back to users about how their input is being considered or used to make improvements. Ignoring feedback quickly erodes trust and discourages future engagement.
Page 26 of 33
▶ Iterate Rapidly During the Pilot
The pilot is not just about evaluation; it's about refinement.17 Use the feedback gathered to identify problems with the AI model, workflow integration, training materials, or support processes. Work with the vendor or internal team to make necessary adjustments during the pilot phase, rather than waiting until the end. This iterative approach allows for course correction and improves the likelihood of a successful outcome
Enterprise Track: Multi-Site Pilots, A/B Testing, Real-Time Dashboards
Enterprises often require more rigorous pilot methodologies due to their scale and complexity.
1. Multi-Site Pilots: Instead of piloting at a single location, run parallel pilots in multiple, diverse settings (e.g., a high-volume urban pharmacy vs. a lower-volume rural one, a hospital outpatient pharmacy vs. a community retail site).17 This helps assess how the AI performs under different operational conditions, patient demographics, and staffing models, revealing potential challenges related to scalability or variability.
2. A/B Testing (Controlled Comparison): For a more statistically robust evaluation, consider implementing A/B testing where feasible.67 This involves randomly assigning users or sites into two groups: Group A uses the new AI-enabled workflow, while Group B (the control group) continues with the traditional workflow. Measure the predefined KPIs for both groups over the pilot period. Comparing the results allows for a more definitive
2. A/B Testing (Controlled Comparison): For a more statistically robust evaluation, consider implementing A/B testing where feasible.67 This involves randomly assigning users or sites into two groups: Group A uses the new AI-enabled workflow, while Group B (the control group) continues with the traditional workflow. Measure the predefined KPIs for both groups over the pilot period. Comparing the results allows for a more definitive
assessment of the AI's true impact, isolating its effect from other confounding factors. While more resource-intensive to set up and manage, A/B testing provides strong evidence for decision-making.
3. Realtime Monitoring Dashboards: Leverage business intelligence or analytics tools to create dashboards that track the key pilot KPIs automatically and in near real-time.8 These dashboards provide immediate visibility into performance trends, potential issues (e.g., sudden drop in accuracy, spike in user errors), and progress towards objectives. This allows the project team and leadership to monitor the pilot closely and react quickly to emerging situations, rather than waiting for periodic manual reports.
3. Realtime Monitoring Dashboards: Leverage business intelligence or analytics tools to create dashboards that track the key pilot KPIs automatically and in near real-time.8 These dashboards provide immediate visibility into performance trends, potential issues (e.g., sudden drop in accuracy, spike in user errors), and progress towards objectives. This allows the project team and leadership to monitor the pilot closely and react quickly to emerging situations, rather than waiting for periodic manual reports.
Page 27 of 33
Table 3: Key Metrics for AI Pilot Success in Pharmacy
An example of a structured evaluation framework for AI implementation in pharmacy from a MedMe Health and University of Waterloo Research Study.
The pilot phase is a critical opportunity for learning and refinement. Starting small with clear success metrics, structured feedback, and strong evaluation helps pharmacies assess effectiveness and make informed decisions about scaling. Uncovering issues during a pilot is not a failure, it's a success in preventing a larger, costlier missteps.
Step 7: Full Rollout and Continuous Monitoring
Once a pilot program (Step 6) demonstrates value based on the predefined metrics, it builds the confidence to move ahead with a full rollout. However, scaling an AI solution from a limited pilot group to standard practice across an organization, especially a large enterprise, requires careful planning, strong support systems, and clear mechanisms for continuous monitoring and improvement. AI implementation is an ongoing process that demands sustained attention to ensure lasting value, safety, and alignment with evolving needs.60
Moving From Pilot to Standard Practice
The transition from pilot to full deployment needs a structured approach:
▶ Develop a Phased Rollout Plan
Based on the learnings and successes of the pilot, create a detailed plan for expanding the AI tool's use. Avoid a "big bang" approach, especially in larger organizations. Consider phasing the rollout geographically , functionally (e.g., service line by service line), or based on site readiness.68 The plan should outline the sequence, specific timelines for each phase, required resources (personnel, budget), key milestones, and a clear communication strategy.
▶ Refine Training and Support MaterialsUpdate all training materials and user guides based on the feedback and insights gathered during the pilot phase. Ensure these materials are easily accessible and reflect the finalized workflow. Scale up user support resources – whether internal help desk staff, vendor support, or the internal champion network – to handle the increased volume of users during the rollout.20
Page 28 of 33
▶ Confirm Infrastructure Scalability
Work closely with IT (if applicable) and the vendor to ensure that the underlying technical infrastructure – servers, databases, network bandwidth, API capacity, vendor systems – can reliably handle the increased load associated with full deployment across the intended user base.9 Address any potential bottlenecks identified during pilot testing.
▶ Execute Clear Communication StrategyCommunicate proactively and clearly to all affected staff about the upcoming rollout. Explain the rationale, reiterate the benefits observed during the pilot, set clear expectations for adoption, detail the rollout schedule, provide information on training resources, and outline available support channels. Consistent communication helps manage change and reduce anxiety.
Ongoing Monitoring and Improvement: The Work After Go-Live
Deploying an AI solution is not the end of the implementation journey; it's the beginning of an ongoing cycle of monitoring, evaluation, and refinement. Unlike traditional software, AI models can evolve, and their performance can change over time. The work doesn't stop after go-live.
▶ Continuous Vigilance is Key:
Establish formal processes for ongoing monitoring of the AI system's performance against key metrics. This includes actively soliciting user feedback through channels proven effective during the pilot (e.g., in-app tools, regular check-ins, champion networks).
Establish formal processes for ongoing monitoring of the AI system's performance against key metrics. This includes actively soliciting user feedback through channels proven effective during the pilot (e.g., in-app tools, regular check-ins, champion networks).
▶ Beware of Drift and Regression:
AI models trained on historical data can experience "performance drift" as real-world conditions change. Furthermore, vendor updates or retraining, while intended to improve the model, can sometimes introduce regression – meaning performance on certain tasks might unexpectedly get worse. Continuous monitoring helps detect these issues early.
AI models trained on historical data can experience "performance drift" as real-world conditions change. Furthermore, vendor updates or retraining, while intended to improve the model, can sometimes introduce regression – meaning performance on certain tasks might unexpectedly get worse. Continuous monitoring helps detect these issues early.
Page 29 of 33
▶ Investigate and Escalate:
Have clear procedures to promptly investigate anomalies or unexpected AI outputs reported by users or flagged by monitoring. Utilize the escalation protocols defined in Step 4 to manage issues based on severity, ranging from logging minor bugs to immediate action for critical errors. Conducting Root Cause Analysis (RCA) for significant issues helps prevent recurrence.
Have clear procedures to promptly investigate anomalies or unexpected AI outputs reported by users or flagged by monitoring. Utilize the escalation protocols defined in Step 4 to manage issues based on severity, ranging from logging minor bugs to immediate action for critical errors. Conducting Root Cause Analysis (RCA) for significant issues helps prevent recurrence.
▶ Stay Informed and Spot-Check:
Keep up-to-date with vendor platform updates and changes. Don't rely solely on automated monitoring; perform occasional manual spotchecks yourself. For example, if you've implemented an AI-powered Interactive Voice Response (IVR) system, call into your pharmacy periodically to interact with it as a patient would, assessing its current performance and user experience firsthand.
Keep up-to-date with vendor platform updates and changes. Don't rely solely on automated monitoring; perform occasional manual spotchecks yourself. For example, if you've implemented an AI-powered Interactive Voice Response (IVR) system, call into your pharmacy periodically to interact with it as a patient would, assessing its current performance and user experience firsthand.
▶ Iterate and Communicate:
Use the insights from monitoring and feedback to drive data-driven improvements, working with the vendor or internal team. Crucially, close the loop by communicating back to users about the improvements made based on their input, reinforcing the value of their feedback and fostering continued engagement.
Use the insights from monitoring and feedback to drive data-driven improvements, working with the vendor or internal team. Crucially, close the loop by communicating back to users about the improvements made based on their input, reinforcing the value of their feedback and fostering continued engagement.
This continuous cycle of monitoring, feedback, investigation, and iteration ensures the AI solution remains effective, safe, relevant, and continues to deliver value over its entire lifecycle.
Enterprise Focus: Governance Oversight, SLA Adherence, Performance Auditing
Maintaining control and ensuring accountability for AI systems deployed across a large enterprise requires robust ongoing governance and oversight mechanisms.
▶ Sustained Governance Oversight
The AI governance committee or equivalent body established in earlier steps should provide continuous oversight for all deployed AI systems. Their responsibilities include reviewing regular performance reports, monitoring identified risks, ensuring ongoing ethical use, approving major updates or changes, and ensuring alignment with organizational policies and evolving regulations.23
▶ SLA Adherence MonitoringEnterprises must actively track and manage vendor performance against the SLAs defined in the contract.56This includes monitoring metrics like system uptime, application performance, support ticket response times, and resolution times. Establish processes for documenting SLA breaches and enforcing contractual remedies (e.g., service credits).56 AI-powered CLM tools can automate aspects of SLA tracking and reporting.56
Page 30 of 33
▶ Periodic Performance Auditing
Beyond routine monitoring, conduct periodic, potentially independent, audits of the AI system's performance, fairness, security, and compliance.23 Audits provide a more in-depth, objective assessment than standard monitoring. Maintaining comprehensive audit trails of AI decisions and system changes is crucial to support these audits.23
▶ Ongoing Regulatory Compliance ChecksRegularly verify that the deployed AI system, its data handling practices, and its use within workflows remain compliant with current healthcare regulations (HIPAA, PHIPA/PIPEDA, GDPR, etc.) and data privacy laws, which may evolve over time.23
The rollout phase marks the beginning of AI becoming part of routine operations, but full integration often takes time. It’s normal for adoption to evolve gradually as teams adapt and workflows adjust. To support this transition, it's important to establish systems for continuous monitoring, feedback collection, iterative improvement, and, particularly for enterprises, formal governance and auditing.
Step 8: Create a LongTerm AI Strategy
To truly harness the transformative potential of AI, pharmacies need to move beyond adopting individual point solutions and develop a comprehensive, long-term AI strategy. This involves fostering an AI-ready culture, proactively upskilling the workforce, staying abreast of rapid technological advancements, and integrating AI thinking into the core strategic planning processes of the organization.6
From Tool Adoption to Cultural Shift
The ultimate goal is to embed AI capabilities strategically across the organization where they can deliver the most value, moving from isolated implementations to a more holistic integration.
1. Strategic Integration, Not Just Tools: Shift the organizational mindset from "implementing an AI tool" to "strategically leveraging AI capabilities" to solve business problems and create new opportunities.68 This means looking for synergies between different AI applications and considering AI as a component of broader digital transformation efforts.
Page 31 of 33
2. Fostering an AI-Ready Culture: Cultivate an environment that encourages curiosity, experimentation, and data-driven decision-making related to AI.19 Leadership must actively champion this cultural shift, promoting AI literacy and celebrating innovation.9 Encourage staff at all levels to think about how AI could improve their work.
3. Embedding AI in Process Improvement: Make AI considerations a standard part of process analysis and workflow redesign initiatives. Proactively ask: "Could AI automate parts of this process?", "Could AI provide insights to improve decisions here?", "How can we design this new service with AI capabilities in mind from the start?".
Enterprise Strategy: Incorporating AI Into Strategic Planning and Innovation Roadmaps
For larger organizations, integrating AI into the formal strategic planning and innovation processes is crucial for sustained, impactful adoption.
▶ Align AI Initiatives with Business Goals:
Ensure that the long-term AI strategy and specific initiatives are explicitly linked to the organization's overarching strategic objectives, such as improving specific patient outcomes, enhancing operational efficiency targets, reducing specific costs, expanding clinical service lines, or improving market competitiveness.9 AI should be a tool to achieve strategic ends, not an end in itself.
▶ Develop a Multi-Year AI Roadmap:
Create a forward-looking roadmap (e.g., 3-5 years) that outlines planned AI initiatives, potential areas of exploration, and required foundational investments (e.g., data infrastructure, talent).14 Prioritize initiatives based on strategic value, feasibility, potential ROI, dependencies, and alignment with overall business priorities.
▶ Integrate AI into Budgeting and Planning:
Ensure that AI initiatives are incorporated into the standard organizational budgeting, capital planning, and resource allocation processes. Secure dedicated funding for strategic AI projects identified in the roadmap.
▶ Establish an Innovation Pipeline:
Create a structured process for identifying, evaluating, prioritizing, and piloting promising new AI opportunities as they emerge.9 This might involve internal ideation platforms, partnerships with external innovators 69, or participation in industry challenges.66
▶ Adopt a Platform-Driven Approach:
For enterprises planning multiple AI deployments, consider investing in or building scalable AI platforms.69 These platforms provide standardized infrastructure, data pipelines, development tools, and governance frameworks that allow for the reuse of components across different use cases, accelerating development, ensuring consistency, and reducing duplication of effort.69
Page 32 of 33
▶ Embed Ethical Considerations:
Ensure that the entire long-term AI strategy is underpinned by a strong, clearly articulated ethical framework that guides the responsible development, deployment, and use of AI technologies, prioritizing patient safety, fairness, transparency, and accountability.23
Developing a long-term AI strategy requires a shift in culture, investment in staff development, continuous learning, and embedding AI into the organization's planning and innovation processes. Pharmacy leaders who take this approach will be better equipped to adapt to navigate the future successfully, enhance operational efficiency, support their teams, and deliver higher-quality care.9
Developing a long-term AI strategy requires a shift in culture, investment in staff development, continuous learning, and embedding AI into the organization's planning and innovation processes. Pharmacy leaders who take this approach will be better equipped to adapt to navigate the future successfully, enhance operational efficiency, support their teams, and deliver higher-quality care.9
Conclusion
AI presents a practical opportunity for pharmacies to improve efficiency, reduce administrative burden, and support clinical services. While implementation can be complex, it is manageable with a clear, structured approach.
This guide has outlined an 8-step framework to help pharmacies begin with low-risk, high-value use cases and build internal readiness. Success requires a capable cross-functional team, realistic assessment of infrastructure, and careful vendor selection based on trust, security, and long-term alignment.
Effective adoption depends on rigorous testing, human-in-the-loop design, and clear training. Piloting solutions enables validation and iteration before full rollout, while ongoing monitoring ensures continued performance and risk management.
As pharmacies face growing operational pressures, AI offers a viable path forward. With disciplined planning and execution, organizations can begin integrating AI in a way that adds value today and scales with future needs
This guide has outlined an 8-step framework to help pharmacies begin with low-risk, high-value use cases and build internal readiness. Success requires a capable cross-functional team, realistic assessment of infrastructure, and careful vendor selection based on trust, security, and long-term alignment.
Effective adoption depends on rigorous testing, human-in-the-loop design, and clear training. Piloting solutions enables validation and iteration before full rollout, while ongoing monitoring ensures continued performance and risk management.
As pharmacies face growing operational pressures, AI offers a viable path forward. With disciplined planning and execution, organizations can begin integrating AI in a way that adds value today and scales with future needs
Page 33 of 33
Appendix A: Quick-Start Readiness Checklist (People, Process, Tech)
Before diving deep into vendor selection or implementation planning, a quick readiness check across key domains can highlight potential roadblocks or areas needing attention. This isn't an exhaustive assessment but a starting point:
▶ People:
- Leadership Buy-in: Is there genuine, visible support from the top? Lack of leadership commitment is a common reason for AI initiatives stalling.9
- AI Champion Identified: Has a specific individual been designated and empowered to lead this effort?19
- Staff Openness to Change: What is the general attitude towards new technologies and process changes within the team? High resistance needs to be addressed proactively.10
- Basic Digital Literacy: Are staff generally comfortable using existing pharmacy software and basic computer functions? Significant gaps here will require foundational training first.
▶ Process:
- Defined Problem/Use Case: Is the specific business problem or opportunity that the AI aims to address clearly articulated? 9
- Current Workflow Understanding: Is the existing manual process well understood and perhaps even documented? Implementing AI effectively requires knowing what you're changing?
- Existing Governance (Basic): Are there any established procedures for approving new software or technology, however informal?
- Preliminary Success Metrics: Have initial thoughts been given to how success will be measured? (e.g., time saved, errors reduced)8
▶ Technology:
- Basic Infrastructure: Does the pharmacy have reliable internet connectivity and workstations capable of handling modern software applications?25
- Data Availability (Initial): Is the data needed for the chosen AI use case (e.g., dispensing history for inventory, consultation notes for documentation) actually being captured and accessible, even if it requires cleaning?9
- Existing Systems Awareness: What is the current Pharmacy Management System (PMS), Electronic Health Record (EHR), or other key software? Are there known integration capabilities or limitations?
- Security Fundamentals: Are basic security hygiene practices followed (e.g., unique user passwords, basic access controls, antivirus software)?25
This checklist helps identify immediate red flags. Significant gaps, particularly in leadership support or basic infrastructure, need to be addressed before proceeding further. The results should inform the implementation strategy, influencing the choice of use case (Step 1) and the criteria for vendor selection (Step 3).
Ready to Enhance Your Pharmacy’s Clinical Practice?
Get started
Subscribe to our newsletter
Thank you! Your submission has been received!
© 2025 MedMe Health. All rights reserved.
MedMe Health is NOT a Pharmacy nor a Drug Manufacturer. We provide software solutions for Pharmacies to conduct clinical services.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.