Lời giải:
\(A=-x^2+2xy-4y^2+2x+10y+5\)
\(\Leftrightarrow -A=x^2-2xy+4y^2-2x-10y-5\)
\(-A=(x^2-2xy+y^2)-2x-10y+3y^2-5\)
\(-A=(x-y)^2-2(x-y)+3y^2-12y-5\)
\(-A=(x-y)^2-2(x-y)+1+3(y^2-4y+4)-18\)
\(-A=(x-y-1)^2+3(y-2)^2-18\)
\(\Rightarrow -A\geq 0+3.0-18\)
\(\Rightarrow -A\geq -18\Rightarrow A\leq 18\)
Vậy GTLN của $A=18$ khi \(\left\{\begin{matrix} x-y-1=0\\ y-2=0\end{matrix}\right.\Leftrightarrow x=3; y=2\)