\(\Delta=\left(m-1\right)^2-4\left(-m^2+m-2\right)\)
\(=5m^2-6m+9=5\left(m-\frac{3}{5}\right)^2+\frac{36}{5}>0;\forall m\)
Mặt khác \(-m^2+m-2\ne0;\forall m\Rightarrow\) biểu thức đề bài luôn xác định
\(B=\left(\frac{x_1}{x_2}+\frac{x_2}{x_1}\right)^3-6\left(\frac{x_1}{x_2}+\frac{x_2}{x_1}\right)\)
Xét \(A=\frac{x_1}{x_2}+\frac{x_2}{x_1}=\frac{\left(x_1+x_2\right)^2-2x_1x_2}{x_1x_2}=\frac{\left(m-1\right)^2-2\left(-m^2+m-2\right)}{-m^2+m-2}=\frac{3m^2-4m+5}{-m^2+m-2}\)
\(\Rightarrow-Am^2+Am-2A=3m^2-4m+5\)
\(\Leftrightarrow\left(A+3\right)m^2-\left(A+4\right)m+2A+5=0\)
\(\Delta=\left(A+4\right)^2-4\left(A+3\right)\left(2A+5\right)\ge0\)
\(\Leftrightarrow7A^2+36A+44\le0\Rightarrow-\frac{22}{7}\le A\le-2\)
Thay vào B:
\(B=A^3-6A\) với \(-\frac{22}{7}\le A\le-2\)
\(B=A^2\left(A+2\right)-2\left(A+1\right)\left(A+2\right)+4\)
Do \(A\le-2\Rightarrow\left\{{}\begin{matrix}A+2\le0\\\left(A+1\right)\left(A+2\right)\ge0\end{matrix}\right.\) \(\Rightarrow B\le4\)
\(\Rightarrow B_{max}=4\) khi \(A=-2\) hay \(m=1\)
