Ta cóa : \(20x^6-\left(8-40y\right)x^3+25y^2-5\)
\(=20x^6-8x^3+40x^3y+25y^2-5\)
\(=16x^6+40x^3y+25y^2+4x^6-8x^3+4-9\)
\(=\left(4x^3+5y\right)^2+4\left(x^3-1\right)^2-9\)
Ta thấy ngay \(\left(4x^3+5y\right)^2\ge0;4\left(x^3-1\right)^2\ge0\)
\(\Rightarrow\left(4x^3+5y\right)^2+4\left(x^3-1\right)^2-9\ge-9\)
\(\Rightarrow M=\frac{6}{20x^6-\left(8-40y\right)x^3+25y^2-5}\le\frac{6}{-9}=-\frac{2}{3}\)
Dấu "=" xảy ra khi \(\hept{\begin{cases}4x^3+5y=0\\x^3-1=0\end{cases}\Leftrightarrow x=1;y=-\frac{4}{5}}\)