Áp dụng BĐT bunhia: \(2xy\le x^2+y^2\)\(\Leftrightarrow xy\le4\)
\(P=2\left(x^2+y^2\right)+2xy-x-y-13\)\(\le2.8+2xy-2\sqrt{xy}-13\)
\(\Leftrightarrow P\le2xy-2\sqrt{xy}+3\)
Đặt \(f=2xy-2\sqrt{xy}+3\)
BBT: \(f=2xy-2\sqrt{xy}+3\), \(0< \sqrt{xy}\le2\)
=> \(f_{max}=7\) \(\Leftrightarrow\sqrt{xy}=2\Leftrightarrow xy=4\)
\(\Rightarrow P_{max}=7\)
Dấu = xảy ra <=> \(\left\{{}\begin{matrix}x^2+y^2=8\\x=y\\xy=4\end{matrix}\right.\)\(\Leftrightarrow x=y=2\)


