a: \(y=-x^3+\left(m+2\right)x^2-3x\)
=>\(y'=-3x^2+2\left(m+2\right)x-3\)
=>\(y'=-3x^2+\left(2m+4\right)\cdot x-3\)
Để hàm số nghịch biến trên R thì \(y'< =0\forall x\)
=>\(\left\{{}\begin{matrix}\left(2m+4\right)^2-4\cdot\left(-3\right)\left(-3\right)< =0\\-3< 0\end{matrix}\right.\)
=>\(4m^2+16m+16-4\cdot9< =0\)
=>\(4m^2+16m-20< =0\)
=>\(m^2+4m-5< =0\)
=>\(\left(m+5\right)\left(m-1\right)< =0\)
TH1: \(\left\{{}\begin{matrix}m+5>=0\\m-1< =0\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}m>=-5\\m< =1\end{matrix}\right.\)
=>-5<=m<=1
TH2: \(\left\{{}\begin{matrix}m+5< =0\\m-1>=0\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}m>=1\\m< =-5\end{matrix}\right.\)
=>\(m\in\varnothing\)
b: \(y=x^3-3x^2+\left(1-m\right)x\)
=>\(y'=3x^2-3\cdot2x+1-m\)
=>\(y'=3x^2-6x+1-m\)
Để hàm số đồng biến trên R thì \(y'>=0\forall x\)
=>\(\left\{{}\begin{matrix}\text{Δ}< =0\\a>0\end{matrix}\right.\)
=>\(\left\{{}\begin{matrix}3>0\\\left(-6\right)^2-4\cdot3\left(1-m\right)>=0\end{matrix}\right.\)
=>\(36-12\left(1-m\right)>=0\)
=>\(36-12+12m>=0\)
=>12m+24>=0
=>m+2>=0
=>m>=-2