I've spent the last several years building AI products across three very different industries: consumer health at Mamaearth, personalized learning at Edxcare, and now clinical GenAI at HCLTech. The industries look nothing alike on the surface - regulatory environments, user personas, data assets, and success metrics all vary dramatically.
But here's what I've learned: the hard problems in AI product management are fundamentally the same. Stakeholders who don't trust the model. Engineers who optimize for accuracy instead of business outcomes. Executives who want AI without understanding what it actually requires. Users who abandon features the moment they get one bad result.
The frameworks I'm sharing below are the ones I keep reaching for. They work whether you're building a clinical decision support tool or a recommendation engine for fast-moving consumer goods.
1. RICE for AI: Prioritization That Accounts for Model Uncertainty
The classic RICE framework (Reach, Impact, Confidence, Effort) needs modification for AI products because confidence means something different when a model is involved.
In standard RICE, confidence captures your certainty about the estimate. In AI product work, I split confidence into two components:
- Business confidence: How certain are you this will drive the outcome you want?
- Technical confidence: How certain are you the model can actually do this reliably?
The modified score becomes: (Reach × Impact × Business Confidence × Technical Confidence) / Effort
At Edxcare, we had a feature idea to auto-generate personalized study plans using student assessment data. Business confidence was high - we knew personalized plans drove retention. Technical confidence was low - our data was sparse and noisy for new students. The combined score correctly flagged this as a deprioritized item until we had enough data density.
At HCLTech, we used the same logic to triage GenAI features for clinical documentation. Features requiring high-confidence medical extraction from unstructured notes got low technical confidence scores until we had validated the extraction pipeline on a representative sample.
Practical tip: Score technical confidence by running a quick feasibility spike - 2-3 days of engineering time to assess data availability, model capability, and latency constraints. Don't let engineering do a full proof-of-concept before prioritization. The spike is enough.
2. AI Readiness Assessment
Before committing to any AI feature, run your organization through a five-dimension readiness check. I use this as a pre-mortem tool - before building, not after.
- Data readiness: Do you have labeled data? How much? How clean? Who owns it?
- Infrastructure readiness: Can you serve model inferences at the latency your UX requires?
- Talent readiness: Do you have ML engineers or are you relying purely on third-party APIs?
- Process readiness: Does your organization have workflows to handle model errors gracefully?
- Governance readiness: Do you have policies for data use, model bias, audit trails, and user consent?
At Mamaearth, when we explored AI-powered skin analysis, governance readiness was the blocker - we didn't have a clear policy for using customer selfie data. We paused the feature and invested three months in building the policy framework first. That decision prevented what could have been a significant brand and legal problem.
In healthcare, governance readiness is almost always the longest pole. In CPG, it's usually data quality. In edtech, it's infrastructure - schools often have terrible connectivity and device heterogeneity.
3. The Build vs Buy Matrix
I'll cover this in detail in a dedicated post, but the quick version: stop treating build vs buy as a binary. There are at least four options.
- Build from scratch: Full control, maximum effort, competitive moat potential
- Buy and customize: Vendor foundation, your differentiation layer on top
- Buy and configure: Pure vendor, your data and prompts only
- Open-source and self-host: Community model, your infrastructure, middle-ground on control
The decision axis isn't just cost - it's where your differentiation lives. If differentiation lives in the model itself, build. If it lives in the data, buy and fine-tune. If it lives in the workflow, buy and configure.
4. LLM Evaluation Framework
Most teams evaluate LLMs on benchmarks that have nothing to do with their actual use case. I use a four-layer evaluation framework instead:
- Intrinsic quality: How good is the raw output? (Accuracy, fluency, factual grounding)
- Task performance: Does it complete the specific task you need? (Custom test sets, not generic benchmarks)
- User acceptance: Do real users trust and act on the output?
- Business impact: Does the feature actually move the metric it was designed to move?
Most teams only measure layer 1. Layer 4 is the only one that matters to the business. I've seen products with impressive intrinsic quality scores fail at layer 3 because users couldn't understand the output format, or at layer 4 because the feature addressed the wrong problem.
Real example: At HCLTech, we had a clinical summarization model that scored well on ROUGE and BERTScore. But nurses weren't using it. Layer 3 audit revealed they didn't trust summaries that didn't cite the source note. We added inline citations. Adoption jumped 40% in the next sprint cycle.
5. Stakeholder Alignment Canvas
AI projects fail at the stakeholder layer more often than the technical layer. I use a canvas with six columns for every major AI initiative:
- Stakeholder: Name/role
- What they care about: Their primary success metric
- Their AI fear: What keeps them up at night about this feature
- Their AI hope: What they're most excited about
- Evidence they need: What would make them a champion vs a blocker
- Engagement mode: How often and in what format do they want updates?
The AI fear column is the most important. In healthcare, clinicians fear liability. In fintech, compliance officers fear regulatory exposure. In edtech, teachers fear replacement. Name the fear explicitly - then address it directly in your product narrative.
6. The Data Flywheel Audit
Before launching any AI feature, map your data flywheel: what data does the feature generate, how does that data feed back into improving the model, and how long until the flywheel creates a defensible moat?
A flywheel that takes five years to spin up is a vulnerability, not a moat. A flywheel that generates signal within the first week of launch is a compounding advantage.
At Edxcare, adaptive learning created a flywheel immediately - every student interaction taught the recommendation engine which content worked for which learning style. At HCLTech, clinical AI flywheels are slower because labeling requires physician review, which is expensive and slow.
The audit question: What's your flywheel spin time, and can you survive until it's meaningful?
7. Failure Mode Mapping
Every AI feature has at least three failure modes you should document before launch:
- False positive failure: The model is wrong in the affirmative. Cost?
- False negative failure: The model misses something it should catch. Cost?
- Confidence calibration failure: The model is confidently wrong. Cost?
In healthcare, a false negative in sepsis detection can cost a life. A false positive in fraud detection can freeze a customer's account. A confidently wrong learning recommendation can waste a student's time and erode their confidence in the platform.
Map each failure mode, estimate the cost (financial, reputational, legal), and design explicit mitigation into the product - not as an afterthought.
8. The AI Trust Ladder
Users don't trust AI systems immediately. They climb a trust ladder, and products that skip rungs fail.
- Awareness: User knows the AI feature exists
- Curiosity: User tries it once
- Skepticism: User tests it, checks its work
- Calibrated trust: User knows when to trust it and when not to
- Habitual use: AI is the default workflow
Design features to accelerate the climb. Explainability features (show your work) help users move from skepticism to calibrated trust. Graceful degradation (acknowledge uncertainty) prevents trust collapse. Progressive disclosure (start with low-stakes tasks) lets users build confidence safely.
9. Regulatory Complexity Index
Not all AI products face the same regulatory overhead. Before committing to a roadmap, score your regulatory complexity on three dimensions:
- Data sensitivity: Is the data PHI, PII, financial records, biometric? (1-5 scale)
- Decision stakes: Is the AI output informational or decision-making? Does a bad output harm a person? (1-5 scale)
- Jurisdictional complexity: How many regulatory bodies have jurisdiction? FDA, HIPAA, GDPR, state laws? (1-5 scale)
A score above 10 means you need a dedicated regulatory track running in parallel to your product track. I've seen teams learn this the hard way - launching a feature and then spending nine months retrofitting compliance.
10. The North Star Metric Selection Framework
AI products tend to attract the wrong north stars. Teams optimize for accuracy, F1 score, or model performance - metrics that don't map to business outcomes.
My framework for selecting a north star: it must be (a) measurable within your sprint cycle, (b) directly influenced by the AI feature, and (c) correlated with revenue or retention in your business model.
In clinical AI, my north star is often workflow time saved per clinician per day - measurable, AI-influenced, correlated with hospital cost reduction. In edtech, it's learning outcome improvement rate - measurable via assessments, AI-influenced through personalization, correlated with renewal. In CPG, it's recommendation click-through rate leading to conversion - tight causal chain from AI to revenue.
Accuracy is an input metric, not a north star. Pick the output metric that maps to why the business is investing in AI in the first place.
Putting It Together
These frameworks aren't meant to be applied all at once. I typically use the AI Readiness Assessment and Stakeholder Alignment Canvas before a project starts, RICE for AI and Failure Mode Mapping during discovery, the LLM Evaluation Framework during development, and the Trust Ladder and North Star Selection Framework during launch and growth.
The through-line is this: AI product management requires you to hold two very different skill sets simultaneously. You need to understand the probabilistic, non-deterministic nature of ML systems. And you need to translate that into deterministic business outcomes that stakeholders can plan around.
These frameworks are the bridge between those two worlds. Use them.
