\(3x^2+10xy+8y^2=96\)
\(\Leftrightarrow3x^2+6xy+4xy+8y^2=96\)
\(\Leftrightarrow3x\left(x+2y\right)+4y\left(x+2y\right)=96\)
\(\Leftrightarrow\left(x+2y\right)\left(3x+4y\right)=96\)
Ta có: \(96=1\cdot96=2\cdot48=3\cdot32=4\cdot24=8\cdot12=6\cdot16\)
Mà \(x,y>0\Rightarrow \)\(\left\{\begin{matrix}3x+4y>7\\x+2y>3\end{matrix}\right.\)
Ta có các hệ sau: \(\left\{\begin{matrix}x+2y=4\\3x+4y=24\end{matrix}\right.\)\(\left(I\right)\Leftrightarrow\)\(\left\{\begin{matrix}x=16\\y=-6\end{matrix}\right.\left(Loai\right)\)
\(\left\{\begin{matrix}x+2y=6\\3x+4y=16\end{matrix}\right.\)\(\left(II\right)\Leftrightarrow\)\(\left\{\begin{matrix}x=4\\y=1\end{matrix}\right.\) (Thỏa mãn)
\(\left\{\begin{matrix}x+2y=8\\3x+4y=12\end{matrix}\right.\)\(\left(III\right)\Leftrightarrow\)\(\left\{\begin{matrix}x=4\\y=6\end{matrix}\right.\left(Loai\right)\)
\(\left\{\begin{matrix}x+2y=12\\3x+4y=8\end{matrix}\right.\)\(\left(IV\right)\Leftrightarrow\)\(\left\{\begin{matrix}x=-16\\y=14\end{matrix}\right.\left(Loai\right)\)
Vậy nghiệm của phương trình là \(\left\{\begin{matrix}x=4\\y=1\end{matrix}\right.\)