a) Sửa đề: \(2x^2\left(ax^2+2bx+4c\right)=6x^4-20x^3-8x^2\)

<=> \(2ax^4+4bx^3+8cx^2=6x^4-20x^3-8x^2\)

=> \(\left\{{}\begin{matrix}2a=6\\4b=-20\\8c=-8\end{matrix}\right.\) <=> \(\left\{{}\begin{matrix}a=3\\b=-5\\c=-1\end{matrix}\right.\)

b) Ta có: \(\left(ax+b\right)\left(x^2-cx+2\right)=x^3+x^2-2\)

<=> \(ax^3-acx^2+2ax+bx^2-bcx+2b=x^3+x^2+2\)

<=> \(ax^3+x^2\left(b-ac\right)+x\left(2a-bc\right)+2b=x^3+x^2-2\)

=> \(\left\{{}\begin{matrix}ax^3=x^3\\\left(b-ac\right)x^2=x^2\\\left(2a-bc\right)x=0\\2b=-2\end{matrix}\right.\) <=> \(\left\{{}\begin{matrix}a=1\\b-ac=1\\2a-bc=0\\b=-1\end{matrix}\right.\)

=> a,b,c ko có!

P/s: Đề có sai ko!

a/ \(-4x^3\cdot\left(ax^2+bx+c\right)=-8x^5+12x^4-20x^3\)

\(\Leftrightarrow-4ax^5-4bx^4-4cx^3=-8x^5+12x^4-20x^3\)

\(\Leftrightarrow\left[{}\begin{matrix}a=-\dfrac{8}{-4}=\dfrac{8}{4}=2\\b=-\dfrac{12}{4}=-3\\c=-\dfrac{20}{-4}=5\end{matrix}\right.\)

Vậy......................

b/ \(-2x^3\cdot\left(ax^2-bx-c\right)=-4x^5+6x^4+2x^3\)

\(\Leftrightarrow-2ax^5+2bx^4+2cx^3=-4x^5+6x^4+2x^3\)

\(\Leftrightarrow\left[{}\begin{matrix}a=2\\b=3\\c=1\end{matrix}\right.\)