Đặt \(\left\{{}\begin{matrix}x=a^2\\y=b^2\end{matrix}\right.\) \(\Rightarrow12\ge\left(a^2+b^2\right)^3+4a^2b^2\ge8a^3b^3+4a^2b^2\)
\(\Rightarrow2a^3b^3+a^2b^2-3\le0\Rightarrow ab\le1\)
\(P=\frac{1}{1+a^2}+\frac{1}{1+b^2}+2018a^2b^2\le\frac{2}{1+ab}+2018a^2b^2\)
Ta sẽ chứng minh \(P\le2019\)
Thật vậy, đặt \(ab=t\Rightarrow0< t\le1\)
\(\frac{2}{1+t}+2018t^2\le2019\Leftrightarrow2+2018t^2\left(1+t\right)\le2019\left(1+t\right)\)
\(\Leftrightarrow2018t^3+2018t^2-2019t-2017\le0\)
\(\Leftrightarrow\left(t-1\right)\left(2018t^2+4036t+2017\right)\le0\) (luôn đúng)
(Do \(2018t^2+4036t+2017>0\) \(\forall t>0\) và \(t-1\le0\) \(\forall t\le1\))
\(\Rightarrow P_{max}=2019\) khi \(x=y=1\)
