Lời giải:
Áp dụng BĐT AM-GM:
$1=x+y\geq 2\sqrt{xy}\Rightarrow xy\leq \frac{1}{4}$
$P=x^2y^2+\frac{1}{x^2y^2}+2-\frac{17}{6}$
$=x^2y^2+\frac{1}{x^2y^2}-\frac{5}{6}$
$=(x^2y^2+\frac{1}{256x^2y^2})+\frac{255}{256x^2y^2}-\frac{5}{6}$
$\geq 2\sqrt{\frac{1}{256}}+\frac{255}{256.\frac{1}{4^2}}-\frac{5}{6}=\frac{731}{48}$
Vậy $P_{\min}=\frac{731}{48}$ khi $x=y=\frac{1}{2}$