Áp dụng bđt cosi ta có:
\(M=\frac{1}{a^2+2b^2+3}+\frac{1}{b^2+2c^2+3}+\frac{1}{c^2+2a^2+3}=\frac{1}{a^2+b^2+b^2+1+2}+\frac{1}{b^2+c^2+c^2+1+2}+\frac{1}{c^2+a^2+a^2+1+2}\le\frac{1}{2\sqrt{ab}+2\sqrt{b}+2}+\frac{1}{2\sqrt{bc}+2\sqrt{c}+2}+\frac{1}{2\sqrt{ac}+2\sqrt{a}+2}=\frac{1}{2}\left(\frac{1}{\sqrt{ab}+\sqrt{b}+1}+\frac{1}{\sqrt{bc}+\sqrt{c}+1}+\frac{1}{\sqrt{ac}+\sqrt{a}+1}\right)=\frac{1}{2}\left(\frac{1}{\sqrt{ab}+\sqrt{b}+1}+\frac{\sqrt{abc}}{\sqrt{bc}+\sqrt{c}+\sqrt{abc}}+\frac{\sqrt{b}}{\sqrt{abc}+\sqrt{ab}+\sqrt{b}}\right)=\frac{1}{2}\left(\frac{1}{\sqrt{ab}+\sqrt{b}+1}+\frac{\sqrt{ab}}{\sqrt{b}+1+\sqrt{ab}}+\frac{\sqrt{b}}{1+\sqrt{ab}+\sqrt{b}}\right)=\frac{1}{2}\left(\frac{1+\sqrt{ab}+\sqrt{b}}{\sqrt{ab}+\sqrt{b}+1}\right)=\frac{1}{2}\Rightarrow M\le\frac{1}{2}\)
Vậy GTLN của M là \(\frac{1}{2}\)
\(M=\sum\frac{1}{a^2+b^2+b^2+1+2}\le\frac{1}{2}\sum\frac{1}{ab+b+1}\)
Maặt khác, ta có bài toán quen thuộc, cho \(abc=1\Rightarrow\sum\frac{1}{ab+b+1}=1\)
\(\Rightarrow M\le\frac{1}{2}\)
Dấu "=" xảy ra khi \(a=b=1\)
