We propose a method for preconditioner construction and parallel im- plementations of the Preconditioned Conjugate Gradient algorithm on GPU platforms. The preconditioning matrix is an approximate inverse derived from an algorithm for the iterative improvement of a solution to linear equations. Using a sparse matrix-vector product, our precondi- tioner is well suited for massively parallel GPU architecture. We present numerical experiments and comparisons with CPU implementations