Lời giải:
Ta có: \(P=\frac{1}{x^2+xy+y^2}+4xy+\frac{1}{xy}=\frac{1}{x^2+xy+y^2}+\frac{1}{3xy}+4xy+\frac{1}{4xy}+\frac{5}{12xy}\)
Áp dụng BĐT AM-GM: \(1=x+y\geq 2\sqrt{xy}\Rightarrow xy\leq \frac{1}{4}\)
\(4xy+\frac{1}{4xy}\geq 2\sqrt{4xy.\frac{1}{4xy}}=2(1)\)
\(\frac{5}{12xy}\geq \frac{5}{12.\frac{1}{4}}=\frac{5}{3}(2)\)
Áp dụng BĐT Cauchy-Schwarz:
\(\frac{1}{x^2+xy+y^2}+\frac{1}{3xy}\geq \frac{4}{x^2+xy+y^2+3xy}=\frac{4}{(x+y)^2+2xy}=\frac{4}{1+2xy}\geq \frac{4}{1+2.\frac{1}{4}}=\frac{8}{3}(3)\)
Từ \((1);(2);(3)\Rightarrow P\geq \frac{8}{3}+2+\frac{5}{3}=\frac{19}{3}\)
Vậy \(P_{\min}=\frac{19}{3}\Leftrightarrow x=y=\frac{1}{2}\)