Với \(a;b\ge0\) đặt \(\left\{{}\begin{matrix}\sqrt{a}=x\\\sqrt{b}=y\end{matrix}\right.\) cho dễ nhìn
\(P=x^2-2xy+3y^2-2x+1\)
\(3P=3x^2-6xy+9y^2-6x+3\)
\(3P=\left(x-3y\right)^2+2\left(x-\frac{3}{2}\right)^2-\frac{3}{2}\ge-\frac{3}{2}\)
\(\Rightarrow P\ge-\frac{1}{2}\)
Dấu "=" xảy ra khi \(\left\{{}\begin{matrix}x=\frac{3}{2}\\y=\frac{1}{2}\end{matrix}\right.\) hay \(\left\{{}\begin{matrix}a=\frac{9}{4}\\x=\frac{1}{4}\end{matrix}\right.\)