Lời giải:
\(Q=\frac{ab}{c+ab}+\frac{ac}{b+ac}+\frac{bc}{a+bc}-\frac{1}{4abc}=\frac{ab}{c(a+b+c)+ab}+\frac{ac}{b(a+b+c)+ac}+\frac{bc}{a(a+b+c)+bc}-\frac{1}{4abc}\)
\(=\frac{ab}{(c+a)(c+b)}+\frac{ac}{(b+a)(b+c)}+\frac{bc}{(a+b)(a+c)}-\frac{1}{4abc}\)
\(=\frac{ab(a+b)+ac(a+c)+bc(b+c)}{(a+b)(b+c)(c+a)}-\frac{1}{4abc}\)
\(=\frac{(a+b)(b+c)(c+a)-2abc}{(a+b)(b+c)(c+a)}-\frac{1}{4abc}\) (đẳng thức quen thuộc \((a+b)(b+c)(c+a)=ab(a+b)+bc(b+c)+ca(c+a)+2abc\) )
\(=1-\left(\frac{2abc}{(a+b)(b+c)(c+a)}+\frac{1}{4abc}\right)\)
Áp dụng BĐT AM-GM:
\(\frac{2abc}{(a+b)(b+c)(c+a)}+\frac{1}{108abc}\geq 2\sqrt{\frac{1}{54(a+b)(b+c)(c+a)}}\).
Mà \(2=(a+b)+(b+c)+(c+a)\geq 3\sqrt[3]{(a+b)(b+c)(c+a)}\Rightarrow (a+b)(b+c)(c+a)\leq \frac{8}{27}\)
\(\Rightarrow \frac{2abc}{(a+b)(b+c)(c+a)}+\frac{1}{108abc}\geq \frac{1}{2}\)
\(1=a+b+c\geq 3\sqrt[3]{abc}\Rightarrow abc\leq \frac{1}{27}\)
\(\Rightarrow \frac{13}{54abc}\geq \frac{13}{2}\)
Do đó: \(\frac{2abc}{(a+b)(b+c)(c+a)}+\frac{1}{4abc}\geq 7\)
\(\Rightarrow Q\leq 1-7=-6=Q_{\max}\)
Dấu "=" xảy ra khi $a=b=c=\frac{1}{3}$