Ta có \(\left(a^3-3ab^2\right)^2\) =\(a^6-6a^4b^2+9a^2b^4=25\)
\(\left(b^3-3a^2b\right)^2=b^6-6a^2b^4+9a^4b^2=100\)
\(=>\left(a^3-3a^2b\right)^2-\left(b^3-3a^2b\right)^2=a^6-6a^4b^2+9a^2b^4+b^6-6a^2b^4+9a^4b^2=125\)
\(< =>a^6+3a^4b^2=3a^2b^4+b^6=125\)
\(< =>\left(a^2+b^2\right)^3=125\)
\(=>a^2+b^2=5\)