The static photovoltaic (PV) models simulate the current and voltage to convert solar energy to electricity. Besides, it is an optimization problem that identifies the unknown parameters of PV models. Runge Kutta optimizer (RUN) was proposed in 2021 when the source code was public. Thus, this paper proposes a reinforcement learning strategy adapted Local Gradient Nelder-Mead Runge Kutta method (RLGNMRUN) for building a novel parameter identification model. In this case, the modified local escaping operator of the gradient-based optimizer replaces the enhanced solution quality strategy of RUN, which increase convergence speed and avoid the local optima. Nelder-Mead simplex (NMs) mechanism forms simplex mining higher quality solutions. Through the reinforcement learning Q-learning reasonably switches between the above two mechanisms, and the proposed method achieves a balanced competitive advantage between global exploration and local exploitation. Experimental results conclude that the modified method has more advantages than existing algorithms under multiple optimization functions, different PV component models, and different environmental conditions in identifying unknown parameters of the PV model.