Ta có: \(\left\{{}\begin{matrix}x+my=3\\mx+4y=6\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}mx+m^2y=3m\\mx+4y=6\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}m^2y-4y=3m-6\\mx+4y=6\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}y\left(m^2-4\right)=3m-6\\mx+4y=6\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}y=\dfrac{3m-6}{m^2-4}\\mx=6-4y\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}y=\dfrac{3\left(m-2\right)}{\left(m+2\right)\left(m-2\right)}=\dfrac{3}{m+2}\\mx=6-4\cdot\dfrac{3}{m+2}\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}y=\dfrac{3}{m+2}\\mx=6-\dfrac{12}{m+2}=\dfrac{6\left(m+2\right)-12}{m+2}\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}y=\dfrac{3}{m+2}\\mx=\dfrac{6m+12-12}{m+2}=\dfrac{6m}{m+2}\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}x=\dfrac{6m}{m+2}:m=\dfrac{6m}{m+2}\cdot\dfrac{1}{m}=\dfrac{6}{m+2}\\y=\dfrac{3}{m+2}\end{matrix}\right.\)
Để phương trình có nghiệm x>1 và y>0 thì \(\left\{{}\begin{matrix}\dfrac{6}{m+2}>1\\\dfrac{3}{m+2}>0\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}\dfrac{6}{m+2}-1>0\\m+2>0\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}\dfrac{6}{m+2}-\dfrac{m+2}{m+2}>0\\m>-2\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}\dfrac{6-m-2}{m+2}>0\\m>-2\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}4-m>0\\m>-2\end{matrix}\right.\Leftrightarrow\left\{{}\begin{matrix}-m>-4\\m>-2\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}m< 4\\m>-2\end{matrix}\right.\Leftrightarrow-2< m< 4\)
Vậy: Để hệ phương trình có nghiệm x>1 và y>0 thì -2<m<4