In this talk, we consider numerical simulations of an ensemble of parametrized problems, each having different initial condition data (if the problem is time dependent), boundary condition data, forcing functions, and physical parameters. For such settings, we propose ensemble methods to accelerate the solutions. The main idea is to manipulate the numerical scheme so that all the problems could share a common coefficient matrix, then, instead of solving a sequence of linear systems with one right-hand-side vector, the method solves one linear system with multiple right-hand-sides. The computational efficiency is then improved by using block iterative algorithms. Rigorous analyses are given proving the conditional stability and establishing error estimates for the proposed algorithms. Numerical experiments are presented to illustrate the analyses.