Lời giải:
Áp dụng BĐT Cauchy-Schwarz:
\(\text{VT}=\frac{1}{1+ab}+\frac{a^2}{a+ab}+\frac{b^2}{b+ab}\geq \frac{(1+a+b)^2}{1+ab+a+ab+b+ab}\)
\(\Leftrightarrow \text{VT}\geq \frac{(a+b+1)^2}{a+b+1+3ab}\)
\(\Leftrightarrow \text{VT}\geq \frac{(a+b+1)^2}{a+b+1+3(3-a-b)}=\frac{(a+b+1)^2}{10-2(a+b)}\)
Theo giả thiết:
\(3=a+b+ab\Leftrightarrow 4=a+b+ab+1=(a+1)(b+1)\)
\(\leq \left (\frac{a+b+2}{2}\right)^2\) (theo BĐT AM-GM)
suy ra \(a+b+2\geq 4\Leftrightarrow a+b\geq 2\) (với \(a,b>0\) )
Do đó: \((a+b+1)^2\geq 9\) (1)
\(10-2(a+b)\leq 10-2.3=4; 10-2(a+b)=4+2ab>0\)
\(\Rightarrow \frac{1}{10-2(a+b)}\geq \frac{1}{6}\) (2)
Từ \((1);(2)\Rightarrow A\geq \frac{(a+b+1)^2}{10-2(a+b)}\geq \frac{9}{6}=\frac{3}{2}\)
Ta có đpcm
Dấu bằng xảy ra khi \(a=b=1\)