\(a+b+1=8ab\Leftrightarrow\frac{1}{a}+\frac{1}{b}+\frac{1}{ab}=8\)
Đặt \(\left\{{}\begin{matrix}x=\frac{1}{a}\\y=\frac{1}{b}\end{matrix}\right.\) \(\Rightarrow x+y+xy=8\) \(\Rightarrow P=\frac{1}{a^2}+\frac{1}{b^2}=x^2+y^2\)
\(8=x+y+xy\le\sqrt{2\left(x^2+y^2\right)}+\frac{x^2+y^2}{2}\)
\(\Leftrightarrow x^2+y^2+2\sqrt{2}\sqrt{x^2+y^2}-16\ge0\)
\(\Leftrightarrow\left(\sqrt{x^2+y^2}+4\sqrt{2}\right)\left(\sqrt{x^2+y^2}-2\sqrt{2}\right)\ge0\)
\(\Leftrightarrow\sqrt{x^2+y^2}\ge2\sqrt{2}\)
\(\Leftrightarrow x^2+y^2\ge8\)
\(\Rightarrow P_{min}=8\) khi \(x=y=2\) hay \(a=b=\frac{1}{2}\)