Từ ab = c2 + 11 suy ra ab - c2 = 11.
Ta có:
\(\dfrac{a}{5}=\dfrac{b}{3}=\dfrac{c}{2}\)
\(\Rightarrow\dfrac{ab}{15}=\dfrac{c^2}{4}\)
Áp dụng tính chất dãy tỉ số bằng nhau ta có:
\(\dfrac{ab}{15}=\dfrac{c^2}{4}=\dfrac{ab-c^2}{15-4}=\dfrac{11}{11}=1\)
\(\Rightarrow\left\{{}\begin{matrix}ab=15\\c^2=4\Rightarrow c=\pm2\end{matrix}\right.\)
+) Nếu c = 2 thì \(\dfrac{a}{5}=\dfrac{b}{3}=1\Rightarrow\left\{{}\begin{matrix}a=5\\b=3\end{matrix}\right.\) (thỏa mãn)
+) Nếu c = -2 thì \(\dfrac{a}{5}=\dfrac{b}{3}=-1\Rightarrow\left\{{}\begin{matrix}a=-5\\b=-3\end{matrix}\right.\) (thỏa mãn)
Vậy (a, b) \(\in\left\{\left(5;3\right),\left(-5;-3\right)\right\}\)