The main conceptual difference is that optimization is about finding the set of parameters/weights that maximizes/minimizes some objective function (which can also include a regularization term; while regularization, is more specific, it is about limiting the values that your parameters can take during the optimization or learning or training using some penalty term.