Gọi \(A=x^2+y^2+xy-3x-3y-3\)
\(=\left(x^2-2x+1\right)+\left(y^2-2y+1\right)+\left(xy-x-y+1\right)-6\)
\(=\left(x-1\right)^2+\left(y-1\right)^2+\left(x-1\right)\left(y-1\right)-6\)
\(=\left(x-1\right)^2+2\cdot\left(x-1\right)\cdot\dfrac{1}{2}\left(y-1\right)+\dfrac{1}{4}\left(y-1\right)^2+\dfrac{3}{4}\left(y-1\right)^2-6\)
\(=\left[\left(x-1\right)+\dfrac{1}{2}\left(y-1\right)\right]^2+\dfrac{3}{4}\left(y-1\right)^2-6\ge-6\) Có GTNN là - 6
Dấu "=" xảy ra \(\Leftrightarrow\left\{{}\begin{matrix}\left[\left(x-1\right)+\dfrac{1}{2}\left(y-1\right)\right]^2=0\\\dfrac{3}{4}\left(y-1\right)^2=0\end{matrix}\right.\Rightarrow x=y=1\)
Vậy GTNN của A là - 6 tại \(x=y=1\)