Optimization with learning-informed models and their applications in optimal control and inverse problems