a: A+2xy^2-x^2y-B=3x^2y-4xy^2
=>A-B=3x^2y-4xy^2-2xy^2+x^2y=4x^2y-6xy^2
=>A=4x^2y; B=6xy^2
b: 5xy^2-A-6x^2y+B=-7xy^2+8x^2y
=>-A+B=-7xy^2+8x^2y-5xy^2+6x^2y=14x^2y-12xy^2
=>A=12xy^2; B=14x^2y
c: 5xy^3-A-5/8x^3y+B=2+1/4xy^3-7/6x^3y
=>-A+B=2+1/4xy^3-7/6x^3y-5xy^3+5/8x^3y
=>B-A=-19/4xy^3-13/24x^3y+2
=>B=-19/4xy^3; A=13/24x^3y-2