Ta có: \(a\sqrt{b+1}=\frac{a\sqrt{\left(b+1\right)2}}{\sqrt{2}}\le a\frac{b+1+2}{2\sqrt{2}}=\frac{ab+3a}{2\sqrt{2}}\)
Tương tự: \(b\sqrt{a+1}\le\frac{ab+3b}{2\sqrt{2}}\)
\(\Rightarrow M\le\frac{3\left(a+b\right)+2ab}{2\sqrt{2}}\le\frac{6+\frac{\left(a+b\right)^2}{2}}{2\sqrt{2}}=\frac{8}{2\sqrt{2}}=2\sqrt{2}\)
Dấu = khi a=b=1
Ta có: \(a+b=2\Rightarrow b=2-a\)
\(\Rightarrow a\sqrt{b+1}=a\sqrt{3-a}\)
Lại có: \(\hept{\begin{cases}a;b>0\\a+b=2\end{cases}}\Rightarrow0\le a;b\le2\)
Mặt khác: \(a\le2\Rightarrow3-a\ge1\)
\(\Rightarrow\sqrt{3-a}\ge1\)
\(\Rightarrow a\sqrt{3-a}\ge a\) Do \(a\ge0\)
Tương tự suy ra \(M\ge a+b=2\)
Dấu = khi \(\left(a;b\right)=\left(0;2\right);\left(2;0\right)\)
Vậy \(M_{Max}=2\sqrt{2}\Leftrightarrow a=b=1\)
\(M_{Min}=2\Leftrightarrow\left(a;b\right)=\left(0;2\right);\left(2;0\right)\)
