a) ĐKXĐ: \(\left\{{}\begin{matrix}a>0\\b>0\\a\ne b\end{matrix}\right.\)

P = \(\dfrac{a-\sqrt{ab}+b+3\sqrt{ab}}{\left(\sqrt{a}+\sqrt{b}\right)\left(a-\sqrt{ab}+b\right)}.\left[\left(\dfrac{a+\sqrt{ab}+b-3\sqrt{ab}}{\left(\sqrt{a}-\sqrt{b}\right)\left(a+\sqrt{ab}+b\right)}\right):\dfrac{a-b}{a+\sqrt{ab}+b}\right]\)= \(\dfrac{\left(\sqrt{a}+\sqrt{b}\right)^2}{\left(\sqrt{a}+\sqrt{b}\right)\left(a-\sqrt{ab}+b\right)}.\left[\dfrac{\left(\sqrt{a}-\sqrt{b}\right)^2}{\left(\sqrt{a}-\sqrt{b}\right)\left(a+\sqrt{ab}+b\right)}.\dfrac{a+\sqrt{ab}+b}{a-b}\right]\)

= \(\dfrac{\sqrt{a}+\sqrt{b}}{a-\sqrt{ab}+b}.\dfrac{\sqrt{a}-\sqrt{b}}{a-b}\)

= \(\dfrac{1}{a-\sqrt{ab}+b}\)

b) có a = 16 và b = 4 (thoả mãn ĐKXĐ)

Thay a = 16, b =4 vào P có:

P = \(\dfrac{1}{16-\sqrt{16.4}+4}\)= \(\dfrac{1}{12}\)

Vậy tại a =16, b = 4 thì P = \(\dfrac{1}{12}\)

\(M=\dfrac{a\sqrt{a}-b\sqrt{b}-a\sqrt{a}+a\sqrt{b}+b\sqrt{a}+b\sqrt{b}}{\left(\sqrt{a}+\sqrt{b}\right)\left(\sqrt{a}-\sqrt{b}\right)}\\ M=\dfrac{a\sqrt{b}+b\sqrt{a}}{\left(\sqrt{a}+\sqrt{b}\right)\left(\sqrt{a}-\sqrt{b}\right)}=\dfrac{\sqrt{ab}\left(\sqrt{a}+\sqrt{b}\right)}{\left(\sqrt{a}-\sqrt{b}\right)\left(\sqrt{a}+\sqrt{b}\right)}\\ M=\dfrac{\sqrt{ab}}{\sqrt{a}-\sqrt{b}}\)

\(\left(1-a\right)\left(1-b\right)+2\sqrt{ab}=1\\ \Leftrightarrow1-a-b+ab+2\sqrt{ab}=1\\ \Leftrightarrow a+b-ab-2\sqrt{ab}=0\\ \Leftrightarrow\left(\sqrt{a}-\sqrt{b}\right)^2=ab\\ \Leftrightarrow\left[{}\begin{matrix}\sqrt{a}-\sqrt{b}=\sqrt{ab}\\\sqrt{a}-\sqrt{b}=-\sqrt{ab}\end{matrix}\right.\)

Với \(\sqrt{a}-\sqrt{b}=\sqrt{ab}\Leftrightarrow M=\dfrac{\sqrt{ab}}{\sqrt{ab}}=1\)

Với \(\sqrt{a}-\sqrt{b}=-\sqrt{ab}\Leftrightarrow M=\dfrac{\sqrt{ab}}{-\sqrt{ab}}=-1\)

\(M=\dfrac{a\sqrt{a}-b\sqrt{b}-a\left(\sqrt{a}-\sqrt{b}\right)+b\left(\sqrt{a}+\sqrt{b}\right)}{\left(\sqrt{a}-\sqrt{b}\right)\left(\sqrt{a}+\sqrt{b}\right)}\)

\(=\dfrac{a\sqrt{b}+b\sqrt{a}}{\left(\sqrt{a}-\sqrt{b}\right)\left(\sqrt{a}+\sqrt{b}\right)}=\dfrac{\sqrt{ab}\left(\sqrt{a}+\sqrt{b}\right)}{\left(\sqrt{a}-\sqrt{b}\right)\left(\sqrt{a}+\sqrt{b}\right)}=\dfrac{\sqrt{ab}}{\sqrt{a}-\sqrt{b}}\)

\(\left(1-a\right)\left(1-b\right)+2\sqrt{ab}=1\)

\(\Leftrightarrow a+b-ab-2\sqrt{ab}=0\)

\(\Leftrightarrow\left(\sqrt{a}-\sqrt{b}\right)^2=ab\Leftrightarrow\sqrt{a}-\sqrt{b}=\sqrt{ab}\)

\(M=\dfrac{\sqrt{ab}}{\sqrt{a}-\sqrt{b}}=\dfrac{\sqrt{ab}}{\sqrt{ab}}=1\)

\(=\dfrac{a+2\sqrt{ab}+b-a+2\sqrt{ab}-b}{a-b}\cdot\dfrac{\sqrt{ab}\cdot\sqrt{a-b}}{ab}\)

\(=\dfrac{4ab}{ab}\cdot\dfrac{1}{\sqrt{a-b}}=\dfrac{4}{\sqrt{a-b}}\)

\(a,\) ta có : 

\(\Leftrightarrow\left\{{}\begin{matrix}A=\sqrt{3}+\sqrt{2^2.3}-\sqrt{3^2.3}-\sqrt{6^2}\\A=\sqrt{3}+2\sqrt{3}-3\sqrt{3}-6\\A=\sqrt{3}.\left(1+2-3\right)-6\\A=-6\end{matrix}\right.\)

\(\Rightarrow A=-6\) . vậy \(A=9\sqrt{5}\)

__________________________________________________________

\(b,\) với \(x>0\) và \(x\ne1\) . ta có :

\(B=\dfrac{2}{\sqrt{x}-1}-\dfrac{1}{\sqrt{x}}+\dfrac{3\sqrt{x}-5}{\sqrt{x}\left(\sqrt{x}-1\right)}\)

\(\Leftrightarrow B=\dfrac{2\sqrt{x}-\left(\sqrt{x}-1\right)+3\sqrt{x}-5}{\sqrt{x}\left(\sqrt{x}-1\right)}\)

\(\Leftrightarrow B=\dfrac{2\sqrt{x}-\sqrt{x}+1+3\sqrt{x}-5}{\sqrt{x}\left(\sqrt{x}-1\right)}\)

\(\Leftrightarrow B=\dfrac{4\sqrt{x}-4}{\sqrt{x}\left(\sqrt{x}-1\right)}\)

\(\Leftrightarrow\) \(B=\dfrac{4\left(\sqrt{x}-1\right)}{\sqrt{x}\left(\sqrt{x}-1\right)}\)

\(\Leftrightarrow B=\dfrac{4}{\sqrt{x}}\)

vậy với \(x>0\) \(;\) \(x\ne1\) thì \(B=\dfrac{4}{\sqrt{x}}\)

để \(B=2\) thì \(\dfrac{4}{\sqrt{x}}=2\Leftrightarrow\sqrt{x}=2\Leftrightarrow x=4\left(tm\right)\)

vậy để \(B=2\) thì \(x=4\)

\(I=\dfrac{a-\sqrt{ab}+b+3\sqrt{ab}}{\left(\sqrt{a}+\sqrt{b}\right)\left(a-\sqrt{ab}+b\right)}\cdot\left[\left(\dfrac{a+\sqrt{ab}+b+3\sqrt{ab}}{\left(\sqrt{a}-\sqrt{b}\right)\left(a+\sqrt{ab}+b\right)}\right)\cdot\dfrac{a+\sqrt{ab}+b}{a-b}\right]\)

\(=\dfrac{a+2\sqrt{ab}+b}{\left(\sqrt{a}+\sqrt{b}\right)\left(a-\sqrt{ab}+b\right)}\cdot\left(\dfrac{a+4\sqrt{ab}+b}{\left(\sqrt{a}-\sqrt{b}\right)\left(a+\sqrt{ab}+b\right)}\cdot\dfrac{a+\sqrt{ab}+b}{a-b}\right)\)

\(=\dfrac{\sqrt{a}+\sqrt{b}}{a-\sqrt{ab}+b}\cdot\dfrac{a+4\sqrt{ab}+b}{\left(\sqrt{a}-\sqrt{b}\right)\left(a-b\right)}\)

\(=\dfrac{a+4\sqrt{ab}+b}{\left(\sqrt{a}-\sqrt{b}\right)^2\cdot\left(a-\sqrt{ab}+b\right)}\)

Khi a=16 và b=4 thì \(I=\dfrac{16+4+4\cdot\sqrt{16\cdot4}}{\left(4-2\right)^2\cdot\left(16-\sqrt{16\cdot4}+4\right)}=\dfrac{20+4\cdot8}{4\cdot12}\)

\(=\dfrac{20+32}{48}=\dfrac{52}{48}=\dfrac{13}{12}\)

\(A=-\dfrac{3+\sqrt{5}+3-\sqrt{5}}{\left(3-\sqrt{5}\right)\left(3+\sqrt{5}\right)}\cdot\dfrac{\sqrt{5}}{5}\\ A=\dfrac{-6}{4}\cdot\dfrac{\sqrt{5}}{5}=\dfrac{-3\sqrt{5}}{10}\)

\(B=\dfrac{a+\sqrt{a}+1-a+\sqrt{a}-1}{\sqrt{a}}+\dfrac{\sqrt{a}-1}{\sqrt{a}}\cdot\dfrac{a+2\sqrt{a}+1+a-2\sqrt{a}+1}{a-1}\)

\(=2+\dfrac{1}{\sqrt{a}}\cdot\dfrac{2a+2}{\sqrt{a}+1}\)

\(=\dfrac{2a+2\sqrt{a}+2a+2}{\sqrt{a}\left(\sqrt{a}+1\right)}=\dfrac{4a+2\sqrt{a}+2}{\sqrt{a}\left(\sqrt{a}+1\right)}\)