\(a.x^2-1+2xy+y^2=\left(x+y\right)^2-1=\left(x+y\right)^2-1^2\)
\(=\left(x+y+1\right)\left(x+y-1\right).\)
\(b.64y^3-1=\left(4y\right)^3-1^3=\left(4y-1\right)\left(4^2y^2+4y+1\right)\)
\(=\left(4y-1\right)\left(16y^2+4y+1\right).\)
\(c.3x^2-12x+9=3\left(x^2-4x+3\right)=3\left(x-1\right)\left(x-3\right).\)
\(d.x^2-9x+20=\left(x^2-4x\right)+\left(-5x+20\right)=x\left(x-4\right)-5\left(x-4\right)\)
\(=\left(x-4\right)\left(x-5\right).\)