\(xy+yz+zx+2xyz=1\)
\(\Leftrightarrow2xy+2yz+2zx+2x+2y+2z+2=xy+yz+zx+2x+2y+2z+3\)
\(\Leftrightarrow2\left(x+1\right)\left(y+1\right)\left(z+1\right)=\left(x+1\right)\left(y+1\right)+\left(y+1\right)\left(z+1\right)+\left(z+1\right)\left(x+1\right)\)
\(\Leftrightarrow\frac{1}{x+1}+\frac{1}{y+1}+\frac{1}{z+1}=2\)
\(\Rightarrow2\ge\frac{9}{x+y+z+3}\Rightarrow x+y+z\ge\frac{3}{2}\)
\(\Rightarrow x^2+y^2+z^2\ge\frac{1}{3}\left(x+y+z\right)^2\ge\frac{3}{4}\)
\(P^2\le3\left[3-\left(x^2+y^2+z^2\right)\right]\le3\left(3-\frac{3}{4}\right)=\frac{27}{4}\)
\(\Rightarrow P\le\frac{3\sqrt{3}}{2}\)
\(P_{max}=\frac{3\sqrt{3}}{2}\) khi \(x=y=z=\frac{1}{2}\)