By C-S and AM-GM's inequality
\(M=a\left(9b\left(a+8b\right)\right)^{\dfrac{1}{2}}+b\left(9a\left(b+8a\right)\right)^{\dfrac{1}{2}}\)
\(\le\left(\left(a^2+b^2\right)\left(9b\left(a+8b\right)+9a\left(b+8a\right)\right)\right)^{\dfrac{1}{2}}\)
\(=\left(\left(a^2+b^2\right)\left(18ab+72b^2+72a^2\right)\right)^{\dfrac{1}{2}}\)
\(=\left(\left(a^2+b^2\right)\left(18\cdot\dfrac{a^2+b^2}{2}+72b^2+72a^2\right)\right)^{\dfrac{1}{2}}\)
\(=\left(16\cdot\left(18\cdot\dfrac{16}{2}+72\cdot16\right)\right)^{\dfrac{1}{2}}=144\)
\("="\Leftrightarrow a=b=2\sqrt{2}\)