Non-deterministic agentic AI systems present fundamental challenges for traditional performance testing methodologies that rely on deterministic metrics and reproducible measurements. We propose a novel probabilistic performance profiling framework that models agent performance as probability distributions rather than point estimates. Our approach leverages Monte Carlo sampling to generate comprehensive performance distribution profiles across diverse execution contexts, while employing Bayesian inference for continuous model refinement based on observed system behavior. The framework provides confidence intervals, performance bounds, and probabilistic guarantees that enable robust decision-making under uncertainty. Extensive evaluation on multiple agent frameworks demonstrates that our approach captures performance variability more accurately than traditional methods, providing 95% confidence intervals with mean absolute errors below 8% across different task complexities. This work establishes the foundational framework for probabilistic performance assessment in agentic systems, enabling more reliable deployment and monitoring of non-deterministic AI agents.