How to solve quadratic equations in system of equations