Ta có: \(\dfrac{3}{5}a=\dfrac{2}{3}b\)
\(\Leftrightarrow\dfrac{a}{\dfrac{5}{3}}=\dfrac{b}{\dfrac{3}{2}}\)
Đặt \(\dfrac{a}{\dfrac{5}{3}}=\dfrac{b}{\dfrac{3}{2}}=k\)
\(\Leftrightarrow\left\{{}\begin{matrix}a=\dfrac{5}{3}k\\b=\dfrac{3}{2}k\end{matrix}\right.\)
Ta có: \(a^2-b^2=38\)
\(\Leftrightarrow k^2\cdot\dfrac{25}{9}-k^2\cdot\dfrac{9}{4}=38\)
\(\Leftrightarrow k^2=72\)
Trường hợp 1: \(k=6\sqrt{2}\)
\(\Leftrightarrow\left\{{}\begin{matrix}a=\dfrac{5}{3}k=10\sqrt{2}\\b=\dfrac{3}{2}k=9\sqrt{2}\end{matrix}\right.\)
Trường hợp 2: \(k=-6\sqrt{2}\)
\(\Leftrightarrow\left\{{}\begin{matrix}a=\dfrac{5}{3}k=-10\sqrt{2}\\b=\dfrac{3}{2}k=-9\sqrt{2}\end{matrix}\right.\)