Theo BĐT Cô - si ta có :
\(M=\sqrt{3}xy+y^2\le\frac{3x^2+y^2}{2}+y^2=\frac{3\left(x^2+y^2\right)}{2}=\frac{3}{2}\)
Vậy \(MAX_M=\frac{3}{2}\) . Dấu \("="\) xảy ra khi : \(\left\{{}\begin{matrix}3x^2=y^2\\x^2+y^2=1\end{matrix}\right.\Rightarrow\left\{{}\begin{matrix}x=\frac{1}{2}\\y=\frac{\sqrt{3}}{2}\end{matrix}\right.\)
Xét \(2M+1=2\sqrt{3}xy+2y^2+1=x^2+2\sqrt{3}xy+3y^2=\left(x+\sqrt{3}y\right)^2\ge0\Rightarrow M\ge-\frac{1}{2}\)
Vậy \(MIN_M=-\frac{1}{2}\) . Dấu \("="\) xảy ra khi \(\left\{{}\begin{matrix}x-\sqrt{3}y=0\\x^2+y^2=1\end{matrix}\right.\Rightarrow\left\{{}\begin{matrix}x=\frac{\sqrt{3}}{2}\\y=\frac{1}{2}\end{matrix}\right.\)