\(\left(x^2+1-x^3\right)^8=\sum\limits^8_{k=0}C^k_8.\left(x^2-x^3\right)^k\)
\(=\sum\limits^8_{k=0}C^k_8\sum\limits^k_{i=0}C^i_k.\left(x^2\right)^{k-i}\left(x^3\right)^i\)
\(=\sum\limits^8_{k=0}\sum\limits^k_{i=0}C^k_8C^i_k.x^{2k+i}\)
\(\Rightarrow2k+i=8\)
Ta có: \(\left\{{}\begin{matrix}2k+i=8\\i\in N\\k\in N\\0\le i\le k\le8\end{matrix}\right.\)
\(\Leftrightarrow\left\{{}\begin{matrix}i=2\\k=3\end{matrix}\right.\)
\(\Rightarrow\) Hệ số của \(x^8\) trong khai triển là \(C^3_8C^2_3=168\).