Đặt \(\left\{{}\begin{matrix}\dfrac{1}{x}=a\\\dfrac{1}{y}=b\end{matrix}\right.\) thì bài toán trở thành
Cho \(a+b+ab=3\)
Tìm GTLN của: \(M=\dfrac{3b}{a+1}+\dfrac{3a}{b+1}-a^2-b^2=\dfrac{ab}{a+1}+\dfrac{ab}{b+1}\)
Ta có: \(3=a+b+ab\ge3\sqrt[3]{a^2b^2}\)
\(\Leftrightarrow ab\le1\)
Ta lại có: \(M=\dfrac{ab}{a+1}+\dfrac{ab}{b+1}=ab.\dfrac{a+1+b+1}{ab+a+b+1}=ab.\dfrac{5-ab}{4}\)
\(=\dfrac{5ab-a^2b^2}{4}=\dfrac{\left(-a^2b^2+2ab-1\right)+3ab+1}{4}=\dfrac{-\left(ab-1\right)^2+3ab+1}{4}\le\dfrac{3+1}{4}=1\)
Vậy GTLN là \(M=1\) khi \(a=b=1\) hay \(x=y=1\)