ĐK: \(x;y\ge0\) đặt \(\left\{{}\begin{matrix}\sqrt{x}=a\ge0\\\sqrt{y}=b\ge0\end{matrix}\right.\)
\(P=a^2-2ab+3b^2-2a+1=3\left(\frac{1}{9}a^2-\frac{2}{3}ab+b^2\right)+\frac{2}{3}\left(a^2-3a+\frac{9}{4}\right)-\frac{1}{2}\)
\(P=3\left(\frac{a}{3}-b\right)^2+\frac{2}{3}\left(a-\frac{3}{2}\right)^2-\frac{1}{2}\ge-\frac{1}{2}\)
\(\Rightarrow P_{min}=-\frac{1}{2}\) khi \(\left\{{}\begin{matrix}a=\frac{3}{2}\\b=\frac{1}{2}\end{matrix}\right.\) hay \(\left\{{}\begin{matrix}x=\frac{9}{4}\\y=\frac{1}{4}\end{matrix}\right.\)