ĐKXĐ: ...
Đặt \(\left\{{}\begin{matrix}\sqrt{x+1}=a\ge0\\\sqrt{y+1}=b\ge0\end{matrix}\right.\)
\(\Rightarrow\left\{{}\begin{matrix}a+b=3\\\left(a^2-1\right)b+\left(b^2-1\right)a+a+b=m\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}a+b=3\\a^2b+ab^2=m\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}a+b=3\\ab\left(a+b\right)=m\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}a+b=3\\ab=\frac{m}{3}\end{matrix}\right.\)
Hệ đã cho có nghiệm khi và chỉ khi pt:
\(\left\{{}\begin{matrix}\frac{m}{3}\ge0\\\left(a+b\right)^2\ge4ab\end{matrix}\right.\) \(\Leftrightarrow\left\{{}\begin{matrix}m\ge0\\9\ge\frac{4m}{3}\end{matrix}\right.\)
\(\Rightarrow0\le m\le\frac{27}{4}\)