Lời giải:
Áp dụng BĐT AM-GM:
$1\geq a+b\geq 2\sqrt{ab}\Rightarrow ab\leq \frac{1}{4}$
Áp dụng BĐT Cauchy-Schwarz:
\(A=\frac{1}{1+a^2+b^2}+\frac{1}{6ab}+\frac{1}{3ab}\geq \frac{4}{1+a^2+b^2+6ab}+\frac{1}{3ab}\)
\(=\frac{4}{1+(a+b)^2+4ab}+\frac{1}{3ab}\geq \frac{4}{1+1+4.\frac{1}{4}}+\frac{1}{3.\frac{1}{4}}=\frac{8}{3}\)
Vậy $A_{\min}=\frac{8}{3}$ khi $a=b=\frac{1}{2}$