a: \(=\dfrac{2}{5}\left(xy-x-y^2+1\right)\)
\(=\dfrac{2}{5}\left[x\left(y-1\right)-\left(y-1\right)\left(y+1\right)\right]\)
\(=\dfrac{2}{5}\left(y-1\right)\left(x-y-1\right)\)
b: \(=x\left(x^2+2xy+y^2-9\right)\)
\(=x\left(x+y-3\right)\left(x+y+3\right)\)