ĐKXĐ: \(x\ge1\)
\(\left(\sqrt{x+8}-2\sqrt{x-1}\right)^2=\left(4-x\right)^2\)
\(\Leftrightarrow\left(x+8+4x-4\right)-4\sqrt{\left(x+8\right)\left(x-1\right)}=\left(4-x\right)^2\)
\(\Leftrightarrow\left(5x+4\right)-4\sqrt{\left(x+8\right)\left(x-1\right)}=16-8x+x^2\)
\(\Leftrightarrow-4\sqrt{\left(x+8\right)\left(x-1\right)}=x^2-13x+12\)
\(\Leftrightarrow-4\sqrt{\left(x+8\right)\left(x-1\right)}=\left(x-12\right)\left(x-1\right)\)
\(\Leftrightarrow-4\sqrt{\left(x+8\right)}=x-12\) (1)
Điều kiện để phương trình có nghiệm là \(x-12\le0\) \(\Leftrightarrow x\le12\)
(1) \(\Rightarrow16\left(x+8\right)=\left(x-12\right)^2\)
\(\Rightarrow16x+128=x^2-24x+144\)
\(\Rightarrow x^2-40x+16=0\)
\(\Rightarrow\left[{}\begin{matrix}x=20+8\sqrt{6}\\x=20-8\sqrt{6}\end{matrix}\right.\) (loại)
Vậy \(S=\varnothing\)