\(ab=\frac{\left(\sqrt{10}+\sqrt{16}\right)\left(\sqrt{10}-\sqrt{16}\right)}{4}=-\frac{3}{2}\)
\(a+b=\sqrt{10}\)
\(a^2+b^2=\left(a+b\right)^2-2ab=13\)
\(a^3+b^3=\left(a+b\right)\left(a^2+b^2-ab\right)=\sqrt{10}\left(13+\frac{3}{2}\right)=\frac{29\sqrt{10}}{2}\)
\(S=\frac{a^5+b^5}{\left(ab\right)^5}=\frac{\left(a^2+b^2\right)\left(a^3+b^3\right)-a^2b^3-a^3b^2}{\left(ab\right)^5}=\frac{\left(a^2+b^2\right)\left(a^3+b^3\right)-\left(ab\right)^2\left(a+b\right)}{\left(ab\right)^5}\)
\(=\frac{13.\frac{29\sqrt{10}}{2}-\frac{9}{4}.\sqrt{10}}{\left(-\frac{3}{2}\right)^5}=-\frac{5960\sqrt{10}}{243}\)