What role do supermassive black holes play when galaxies merge?