\(P=\left(x^2+y^2\right)^2-2x^2y^2-4xy+3=\left[\left(x+y\right)^2-2xy\right]^2-2x^2y^2-4xy+3\)
\(=\left(16-2xy\right)^2-2x^2y^2-4xy+3=2x^2y^2-68xy+259\)
\(4=x+y\ge2\sqrt[]{xy}\Rightarrow0\le xy\le4\)
Đặt \(xy=a\Rightarrow0\le a\le4\)
\(P=2a^2-68a+259=259-2a\left(34-a\right)\le259\)
\(P_{max}=259\) khi \(a=0\) hay \(\left(x;y\right)=\left(4;0\right);\left(0;4\right)\)
\(P=\left(2a^2-68a+240\right)+19=2\left(4-a\right)\left(30-a\right)+19\ge19\)
\(P_{min}=19\) khi \(a=4\) hay \(x=y=2\)