Lời giải:
Đặt \(\frac{\sqrt{ab}-1}{3}=\frac{\sqrt{bc}-3}{9}=\frac{\sqrt{ca}-5}{-6}=t\)
\(\Rightarrow \left\{\begin{matrix} \sqrt{ab}=3t+1\\ \sqrt{bc}=9t+3\\ \sqrt{ca}=5-6t\end{matrix}\right.\)
\(\Rightarrow \sqrt{ab}+\sqrt{bc}+\sqrt{ca}=6t+9\)
\(\Leftrightarrow 11=6t+9\Leftrightarrow t=\frac{1}{3}\)
Khi đó : \(\left\{\begin{matrix} \sqrt{ab}=2\\ \sqrt{bc}=6\\ \sqrt{ac}=3\end{matrix}\right.\) \(\Rightarrow \left\{\begin{matrix} ab=4\\ bc=36\\ ac=9\end{matrix}\right.\Rightarrow abc=\sqrt{4.36.9}=36\)
\(\Rightarrow \left\{\begin{matrix} c=\frac{abc}{ab}=9\\ a=\frac{abc}{bc}=1\\ b=\frac{abc}{ac}=4\end{matrix}\right.\)
Vậy....